Local and Remote Type-2 Are Different Truths

Introduction to Provenance in Networking

Provenance is the traceable origin and transformation history of forwarding state—such as a MAC address, IP address, or EVPN route—from the point it is first learned locally, through control‑plane advertisements (e.g., BGP EVPN Route‑Type 2), to its final installation in the forwarding information base (FIB). It captures who originated the state, what modifications were applied, and where it currently resides.

When forwarding fails (black‑holing, loops, incorrect MAC/IP binding), the symptom appears at the data‑plane layer. Without provenance information, engineers often assume the problem lies in the local learning process, while the real cause may be a mismatch between what was advertised, imported, and installed. Comparing the four provenance checkpoints—local learning state, exported RT‑2, imported RT‑2, and installed forwarding entries—lets an operator pinpoint exactly where the state diverged from intent and avoid misdirected troubleshooting.

Local MAC or IP Learning State

Overview

• MAC learning: Switch records <MAC, VLAN, ingress port> in its MAC address (CAM) table when it sees a source MAC in an incoming Ethernet frame.
• IP learning: Via ARP or ND, the switch records <IP, MAC, VLAN, ingress port> in an ARP/ND cache or host‑binding table.

These tables constitute the local learning state—the ground truth observed directly on the device’s interfaces.

How Local Learning State Is Updated

1. Frame/Packet Reception – ASIC parses L2/L3 headers.
2. Lookup – MAC/IP is looked up in the learning table.
3. Insert/Age – If absent, create entry with timestamp; if present, refresh timestamp.
4. Aging – Entries older than the configured age timer (typically 300 s) are removed unless relearned.
5. Sync to Control Plane – Many platforms copy new/updated entries to the routing daemon (e.g., l2rib, macrib) for potential export into BGP EVPN.

CLI Commands for Viewing Local Learning State

Note: Output format varies; consult the device manual for exact column meanings.

Exported RT‑2 and Its Role in Provenance

Definition and Purpose

In EVPN, Route‑Type 2 (RT‑2) carries MAC/IP advertisement routes. An exported RT‑2 is the route a VTEP (or leaf switch) originates and sends to its BGP peers after encapsulating a locally learned MAC/IP binding in the NLRI:

NLRI: [ESI(0), Ethernet Tag ID, MAC Address Length, MAC Address,
       IP Address Length, IP Address]
Attributes: Route Target (RT), MAC Mobility Extended Community, etc.

Exporting RT‑2 propagates the local learning state to the overlay control plane so remote leaves can learn the binding without relying on data‑plane flooding.

How Exported RT‑2 Affects Provenance

Exported RT‑2 is the first control‑plane representation of the local learning state. If export fails or is mis‑configured, remote peers never receive the binding, even though the local state is correct. Conversely, spurious exports (e.g., due to sticky MAC timers or mis‑applied MAC move detection) can inject false provenance into the overlay.

CLI Examples for Exported RT‑2

Configuration (Cisco NX‑OS – VXLAN BGP EVPN)

feature bgp
feature evpn
feature nv overlay
interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10000 associate-vrf
  member vni 20000

Verification

# Show locally originated EVPN RT‑2 routes
show bgp l2vpn evpn route-type 2

# Show export status per VNI
show nve vni

Illustrative output

BGP routing table entry for [2]:[0]:[48]:[0011.2233.4455]:[32]:[10.1.1.10]/24
  Paths: (1 available, best #1)
    Advertised to update-groups:
          1
    Local
      0.0.0.0 from 0.0.0.0 (10.0.0.1)
        Origin IGP, metric 0, localpref 100, weight 32768, valid, sourced, best
        Extended Community: RT:65000:10000 MAC Mobility:flags 0x0

Note: Attribute names may differ between vendors; the presence of an RT‑2 entry with the local router as origin indicates successful export.

Imported RT‑2 and Its Impact on Provenance

Definition and Purpose

An imported RT‑2 is a MAC/IP advertisement route received from a BGP peer, processed by the inbound routing policy, and inserted into the EVPN routing table (often called the l2rib or evpn rib). Imported RT‑2 represents the remote learning state that a leaf will use to populate its forwarding table for that MAC/IP binding.

How Imported RT‑2 Affects Provenance

Imported RT‑2 is the second control‑plane checkpoint. If the import policy incorrectly filters, modifies, or drops the route, the leaf will never install a forwarding entry for that binding, even though the remote peer correctly exported it. Conversely, an over‑permissive import policy may accept stale or duplicate RT‑2s, leading to MAC move flaps or incorrect forwarding.

Example Code for Configuring and Troubleshooting Imported RT‑2

Configuration (Juniper Junos – EVPN)

set routing-instances EVPN-VRF instance-type vrf
set routing-instances EVPN-VRF vtep-source-interface lo0.0
set routing-instances EVPN-VRF interface et-0/0/1.0
set routing-instances EVPN-VRF route-distinguisher 65000:100
set routing-instances EVPN-VRF vrf-target target:65000:100
set routing-instances EVPN-VRF protocols evpn encapsulation vxlan
set routing-instances EVPN-VRF protocols evpn extended-vni-list 10000

Import policy example (accept all RT‑2s)

set policy-options community EVPN-RT2 members target:65000:100
set policy-options policy-statement IMPORT-EVPN-RT2 term accept from community EVPN-RT2
set policy-options policy-statement IMPORT-EVPN-RT2 term accept then accept
set routing-instances EVPN-VRF protocols bgp group EVPN-PEERS import IMPORT-EVPN-RT2

Verification

show route table EVPN-VRF.evpn.0 route-type 2
show bgp neighbor <peer-ip> advertised-routes | match route-type 2
show bgp neighbor <peer-ip> received-routes | match route-type 2

Troubleshooting tip: If the received‑routes count is non‑zero but the installed route table shows zero matches, check for:

• Import policy then reject or next policy mismatch.
• Duplicate route‑target mismatch (RT‑2 carries RT; the VRF’s import RT must match).
• MAC mobility community causing route flap suppression.

Note: Junos uses evpn.0 table for EVPN routes; other platforms may use l2rib or bgp.l2vpn.0. Adjust accordingly.

Installed Forwarding Entries and Provenance

Overview

The forwarding information base (FIB) or MAC forwarding table is the data‑plane structure used by the ASIC to make per‑packet forwarding decisions. An entry typically contains:

• Destination MAC (or MAC+VLAN)
• Outgoing interface (local port, VNI encapsulation, or peer VTEP IP)
• Optional metadata (e.g., VNI, ESI, MAC mobility flag)

When a MAC/IP binding is learned locally or imported via RT‑2, the control plane programs the FIB through the SDK (e.g., Cisco’s fibmgr, Juniper’s pfed, Arista’s syncd).

How Installed Forwarding Entries Relate to Provenance

Installed entries are the final provenance checkpoint. They reflect the applied state after all control‑plane processes (learning, export, import, policy) have run. A mismatch between the imported RT‑2 and the FIB entry indicates a failure in the control‑to‑data plane sync (e.g., SDK error, resource exhaustion, or explicit suppress).

Troubleshooting Installed Forwarding Entries with CLI Examples

Example of a missing FIB despite correct RT‑2 (Cisco NX‑OS):

switch# show bgp l2vpn evpn route-type 2 | include 0011.2233.4455
[2]:[0]:[48]:[0011.2233.4455]:[32]:[10.1.1.10]/24
    ... bestpath ...

switch# show mac address-table address 0011.2233.4455
% MAC address not found

The absence of the MAC in the table while the RT‑2 is present signals a control‑to‑data plane sync failure.

Comparing States to Identify Provenance Loss

Methodology

1. Capture Local Learning State

   • Run show mac address-table (or ARP) on the leaf where the host is attached.
   • Record <MAC, VLAN, port, age> and <IP, MAC, VLAN, port> tuples.

2. Verify Exported RT‑2

   • On the same leaf, query the BGP EVPN table for locally originated RT‑2 (show bgp l2vpn evpn route-type 2 with local flag).
   • Confirm that the NLRI matches the captured MAC/IP and that the route is marked as local or self‑originated.

3. Check Imported RT‑2 on Remote Leaf

   • On the peer leaf, run show bgp l2vpn evpn route-type 2 received-routes (or equivalent) to see if the RT‑2 was received.
   • Apply the inbound import policy (show policy ...) to confirm the route is accepted into the EVPN RIB.

4. Inspect Installed Forwarding Entry

   • On the peer leaf, query the MAC/FIB table for the MAC (show mac address-table address <mac>).
   • If the host is reachable via IRB, also check the ARP/ND cache and the L3 FIB (show route forwarding-table).

5. Compare Tuples
   Build a simple comparison table:

Any missing or mismatched field indicates where provenance was lost.

Tools and Techniques for Comparison

• Manual CLI diff – Capture each command output to a file and use diff or vimdiff.
• Automated scripts – Use Python/Ansible to gather the four checkpoints and compare fields programmatically.
• Visibility platforms – Tools such as Cisco DNA Center, Juniper Contrail, or Arista CloudVision can correlate control‑plane and data‑plane state across the fabric.

By systematically walking through these four provenance checkpoints, engineers can quickly isolate whether a forwarding problem stems from local learning, export, import, or control‑to‑data plane synchronization—avoiding the common pitfall of troubleshooting the wrong leaf.