VLAN Strip Features Change What XDP and tc Parse

Introduction to Fast‑Path Misclassification

A VLAN tag is a 4‑byte field (TPID = 0x8100, TCI) placed after the Ethernet source/destination addresses and before the EtherType/length field. NICs can offload VLAN handling:

• rxvlan – strip the tag on receive, presenting a plain Ethernet frame to the driver.
• txvlan – insert the tag on transmit.

When these features are enabled the driver never sees the tag; when disabled the tag remains in the packet buffer handed to the kernel stack.

Where XDP and tc Operate

• XDP runs immediately after the NIC DMA ring delivers a packet to the driver. The struct xdp_md *ctx points to the raw buffer exactly as the NIC placed it in memory, after any hardware offloads (rxvlan/txvlan, checksum, LRO, etc.).
• tc ingress runs later, after the driver calls netif_receive_skb() and the packet becomes an skb. At this point the stack may have performed VLAN demux (if a VLAN sub‑interface exists) or may have left the tag intact, depending on the presence of a matching vlan device.

A filter or XDP program that assumes the tag is always at a fixed offset (e.g., eth + 12) will misbehave when the actual layout differs because of driver offloads or stack processing.

Competing Explanations for Fast‑Path Misclassification

Vanishing VLAN Tags

Driver impact – With rxvlan enabled the NIC removes the tag before the driver sees the packet. The driver delivers an untagged Ethernet frame; an XDP program reading a VLAN TCI at offset 14 will fetch the first two bytes of the payload, causing false matches or drops.

XDP impact – XDP sees the same untagged frame as the driver because stripping occurs in the NIC before the DMA buffer is filled. An XDP program that does

u16 vlan_tci = *(u16 *)(data + offset_to_eth + 12);

will read whatever happens to be in the payload (often the first two bytes of an IPv4 header or TCP/UDP port). Using this value for XDP_PASS/XDP_DROP can lead to incorrect classification.

tc impact – tc filters attached to the netdevice receive untagged frames when rxvlan is on, so a filter matching vlan id 100 never matches. If the filter is attached to a VLAN sub‑device (e.g., eth0.100) while rxvlan strips the tag, the sub‑device is never used because the underlying device never sees a tagged packet.

Shifting VLAN Tags

Driver impact – Some NICs/firmware (e.g., certain Mellanox devices) can move the VLAN tag deeper into the packet when features like TX checksum offload, LRO, or tunneling encapsulation (VXLAN, GRE) are active. The tag may appear after an outer tunnel header instead of directly after the Ethernet MAC addresses.

XDP impact – An XDP program assuming a fixed offset will read the wrong bytes. With VXLAN encapsulation the layout is

[Eth MAC][Eth Type=0x0800][UDP][VXLAN][Inner Eth][Inner VLAN?][Inner Payload]

Looking at data+14 for a VLAN TCI fetches part of the VXLAN header (typically the VXLAN Network Identifier) and treats it as a VLAN ID, causing misclassification.

tc impact – tc runs after the driver has assembled the skb. If the tag has been shifted by hardware offloads, the skb still contains the tag at its shifted location. A tc filter that matches on vlan id uses vlan_tx_tag_get(), which reads skb->vlan_tci set by the network stack only when it detects a VLAN header at the canonical position (immediately after the MAC addresses). If the tag is shifted, the stack does not detect it, vlan_tci is zero, and the filter never matches. The packet may be treated as untagged, leading to incorrect queueing or classification.

Troubleshooting Fast‑Path Misclassification

Identifying Vanished or Shifted VLAN Tags

Verifying XDP Parsing Assumptions

1. Check XDP program attachment

   ip link show dev eth0          # look for xdp flag and program name
   bpftool prog show              # list all XDP programs and IDs
   bpftool prog dump xlated id <PROG_ID>   # inspect bytecode for offset used

2. Sniff the raw packet with XDP

   /* xdp_sniff.c – capture first 64 bytes */
   #include <linux/bpf.h>
   #include <linux/if_ether.h>
   #include <bpf/bpf_helpers.h>
   
   struct {
       __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
       __uint(max_entries, 1);
       __type(key, u32);
       __type(value, unsigned char[64]);
   } xdp_sniff SEC(".maps");
   
   SEC("xdp")
   int xdp_sniffer(struct xdp_md *ctx)
   {
       void *data = (void *)(long)ctx->data;
       void *data_end = (void *)(long)ctx->data_end;
       u32 idx = 0;
       if (data + 64 > data_end)
           return XDP_PASS;
   
       bpf_probe_read_kernel(&xdp_sniff, sizeof(xdp_sniff), data);
       return XDP_PASS;
   }
   char _license[] SEC("license") = "GPL";

   Load the program, then read the map while traffic flows:

   bpftool map dump id <MAP_ID>

   Compare the captured bytes to the expected Ethernet layout. Missing 0x8100 at offset 12 indicates vanishing; a tag appearing later indicates shifting.

Validating tc Filter Expectations

• Inspect filter configuration

  tc filter show dev eth0 ingress          # lists filters and match keys
  tc -s qdisc show dev eth0                # queue discipline statistics
  bpftrace -e 'tracepoint:tc:cls_classify { printf("tc proto=%d vlan_tci=%x\n", args->protocol, args->vlan_tci); }'

• Test with known VLAN traffic

  # reset counters
  tc -s filter show dev eth0 ingress | grep '^filter'
  # send tagged traffic
  ping -c 100 -I eth0.100 192.168.1.1
  # check counters again
  tc -s filter show dev eth0 ingress

  If the filter’s counters stay at zero while tcpdump -i eth0.100 sees packets, the tag is either stripped before reaching tc or shifted such that tc’s VLAN matcher does not fire.

Code Examples for Troubleshooting

CLI Commands for VLAN Tag Inspection

# Show offload settings
ethtool -k eth0 | grep -E 'rx-vlan-offload|tx-vlan-offload'

# Capture first 20 bytes of each packet to see VLAN presence
tcpdump -i eth0 -c 20 -vv -e

# Show tc filter counters
tc -s filter show dev eth0 ingress

XDP Program to Verify VLAN Tag Presence

/* xdp_vlan_check.c – count packets with/without VLAN at expected offset */
#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <bpf/bpf_helpers.h>

struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 2);
    __type(key, u32);          // 0 = no tag, 1 = tag present
    __type(value, u64);
} vlan_stat SEC(".maps");

SEC("xdp")
int xdp_vlan_check(struct xdp_md *ctx)
{
    void *data = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;
    struct ethhdr *eth = data;

    if ((void *)(eth + 1) > data_end)
        return XDP_PASS;

    u16 proto = eth->h_proto;
    u32 key = 0; // default: no VLAN

    if (proto == htons(ETH_P_8021Q)) {
        /* VLAN tag found at expected position */
        key = 1;
    }

    u64 *val = bpf_map_lookup_elem(&vlan_stat, &key);
    if (val)
        *val += 1;

    return XDP_PASS;
}
char _license[] SEC("license") = "GPL";

Compile and load:

clang -O2 -target bpf -c xdp_vlan_check.c -o xdp_vlan_check.o
ip link set dev eth0 xdp obj xdp_vlan_check.o sec xdp

Read results:

bpftool map dump id $(bpftool map show | grep vlan_stat | awk '{print $1}')

These steps let you pinpoint whether VLAN tags are vanishing, shifting, or present as expected, and verify what each hook (driver, XDP, tc) actually sees.