Intended Attach Graph vs the Programs Actually Live

Introduction to XDP and tc Attachments

eXpress Data Path (XDP) and Traffic Control (tc) are kernel‑level mechanisms that let operators inject BPF programs into the fast‑path of packet processing. XDP runs immediately after the NIC DMA ring delivers a packet, enabling drop, redirect, or pass decisions before the kernel allocates an skb. tc attaches BPF programs to classic queuing disciplines (qdiscs) or classful classifiers, providing shaping, policing, and complex actions after the packet has entered the networking stack.

Both mechanisms share a common lifecycle:

1. Intended state – the declarative goal (source code, policy, or command snippet).
2. Rendered state – the kernel‑loadable object produced by the toolchain (ELF .o or netlink payload).
3. Applied state – the moment the rendered object is installed and bound to a netdev or qdisc.
4. Observed state – runtime behavior visible via counters, traces, or packet captures.

Drift appears when any layer diverges from the others, often silently until a traffic anomaly surfaces.

Key Concepts: Intended, Rendered, Applied, and Observed State

Understanding where each layer lives helps answer: “Did the change I made actually take effect, and is it behaving as I expect?”

Modeling XDP Attachments

Intended State: XDP Program Definition

The intended state begins with the operator’s functional requirement—e.g., “drop all inbound packets from 10.0.0.0/8 on interface eth0”. In practice this is expressed as C (or Rust) using BPF helpers:

/* prog_drop_10_0_0_8.c */
#include <linux/bpf.h>
#include <linux/pkt_cls.h>
#include <linux/if_ether.h>
#include <linux/ip.h>

SEC("xdp")
int drop_10_0_0_8(struct xdp_md *ctx)
{
    void *data_end = (void *)(long)ctx->data_end;
    void *data     = (void *)(long)ctx->data;
    struct ethhdr *eth = data;

    if ((void *)(eth + 1) > data_end)
        return XDP_PASS;

    if (eth->h_proto != htons(ETH_P_IP))
        return XDP_PASS;

    struct iphdr *ip = data + sizeof(struct ethhdr);
    if ((void *)(ip + 1) > data_end)
        return XDP_PASS;

    if ((ip->saddr & htonl(0xFF000000)) == htonl(0x0A000000)) /* 10.0.0.0/8 */
        return XDP_DROP;

    return XDP_PASS;
}
char _license[] SEC("license") = "GPL";

Intended state artifacts:

• Source file (prog_drop_10_0_0_8.c) stored in Git.
• Build manifest (Makefile or bpftool gen skeleton) that records the desired section name (xdp) and target interface.

Rendered State: XDP Program Compilation and Loading

Compilation turns the C source into an ELF object that the BPF verifier can ingest:

# Install required packages (example for Ubuntu)
sudo apt-get install -y clang llvm libelf-dev libpcap-dev

# Compile
clang -O2 -target bpf -c prog_drop_10_0_0_8.c -o prog_drop_10_0_0_8.o

Verification of the rendered state:

# Show the ELF sections
readelf -S prog_drop_10_0_0_8.o

# Dump the BPF bytecode after verifier processing
sudo bpftool prog dump xlated prog prog_drop_10_0_0_8.o

The rendered state is immutable once the object is on disk; any source change requires a recompile. This immutability lets us treat the rendered artifact as a versioned input to the apply step.

Applied State: XDP Program Attachment to Network Interface

Attachment is performed with the ip link command (or via bpf link create for a more stable reference). The simplest, immediate form:

# Attach the program to eth0
sudo ip link set dev eth0 xdp obj prog_drop_10_0_0_8.o sec xdp

What happens under the hood:

1. Kernel loads the ELF object via the BPF syscall (bpf(BPF_PROG_LOAD)).
2. The verifier runs; if it rejects, the syscall returns -EINVAL and no attachment occurs.
3. On success, the kernel stores a reference to the program in the netdev’s xdp_prog field and increments the program’s usage count.

Applied state verification:

# Show the attached program ID
ip -d link show eth0 | grep xdp
# Expected output: xdp prog id 12345 tag abcdef1234567890

# Or via bpftool
sudo bpftool link show | grep eth0

If the command returns nothing, the applied state is detached (or the attachment failed silently—check dmesg for verifier logs).

Observed State: XDP Program Performance and Statistics

Observability comes from generic interface counters and BPF‑specific stats.

# Basic interface counters (rx/tx/drops)
ip -s link show eth0

# XDP-specific stats via bpftool
sudo bpftool prog show id 12345
# Output includes: run_time, processed, dropped

# Tracepoint example (if the program uses bpf_trace_printk)
sudo cat /sys/kernel/debug/tracing/trace | grep drop_10_0_0_8

A healthy observed state for the drop‑10 program would show:

• processed roughly equal to the inbound packet rate on eth0.
• dropped increasing for packets matching 10.0.0.0/8.
• No increase in rx_drops at the NIC level (since XDP drop occurs before the NIC’s own drop counters).

If processed stalls or dropped stays zero while traffic matching the subnet is present, the applied state likely does not match the intended state (e.g., wrong section, program not loaded, or attachment to the wrong interface).

Modeling tc Attachments

Intended State: tc Configuration and Qdisc Definition

The intended state for tc is usually expressed as a series of tc commands that define a qdisc, classes, filters, and actions. Example: “Apply an HTB root qdisc with a 100 Mbps ceiling and a leaf SFQ for fairness”.

# Intended tc script (store in version control)
tc qdisc add dev eth0 root handle 1: htb default 30
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 20mbit ceil 100mbit
tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 \
    match ip dst 10.0.0.0/8 flowid 1:10

These commands constitute the intended state; they are idempotent only if the objects already exist with the same parameters.

Rendered State: tc Configuration Rendering and Validation

When a tc command is executed, the userspace tool parses the arguments, builds an internal netlink message, and validates against the kernel’s ABI. The rendered state is the netlink payload that will be sent to the kernel.

You can inspect the rendered netlink without applying it using the -p flag:

# Show what netlink would be sent (requires tc with -p flag)
tc -p qdisc show dev eth0

In practice, the rendered state is transient; however, you can capture it via tc -s -d show or by enabling netlink logging (nlmon or tcpdump -i netlink). If the command fails validation (e.g., unknown classid), the rendered state is never sent, and the apply step does not happen.

Applied State: tc Configuration Application to Network Interface

Application occurs when the netlink message is successfully transferred to the kernel and the corresponding qdisc/class/filter objects are created in the scheduler attached to the netdev.

# Apply the intended tc script
sudo tc qdisc add dev eth0 root handle 1: htb default 30
sudo tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit
sudo tc class add dev eth0 parent 1:1 classid 1:10 htb rate 20mbit ceil 100mbit
sudo tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10
sudo tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 \
    match ip dst 10.0.0.0/8 flowid 1:10

Verification of the applied state:

# Show the full hierarchy
sudo tc -s qdisc show dev eth0
# Expected output includes the HTB root, child classes, SFQ leaf, and filter

# Show filter details
sudo tc -s filter show dev eth0 parent 1:0

If any command in the chain fails (e.g., due to a typo in classid), the kernel leaves the previously applied state untouched, and the operator sees a partial configuration—a classic drift scenario.

Observed State: tc Configuration Performance and Statistics

Observability for tc is provided by the -s (statistics) flag on tc commands, which queries the kernel’s qdisc and class counters.

# View qdisc and class stats
sudo tc -s qdisc show dev eth0
sudo tc -s class show dev eth0

# Monitor packet flow through a specific class
sudo tc -s filter show dev eth0 parent 1:0 prio 1 u32

A healthy observed state for the HTB/SFQ example:

• The 1:10 class shows sent bytes approximating the traffic destined for 10.0.0.0/8.
• The SFQ leaf (10:) shows a non‑zero backlog and drops only under congestion.
• The filter’s match counter increments in lockstep with the class’s sent counter.

If the filter counter stays zero while traffic to 10.0.0.0/8 is present, the applied state (filter) is not matching the intended state—perhaps the filter was attached to the wrong parent or the protocol field is mis‑specified.

Understanding Drift Across Interfaces

Interface Configuration Drift

Interface drift occurs when the netdev’s operational flags, MTU, offload features, or XDP/tc attachments diverge from the desired state, leading to mismatched behavior. Detecting drift requires continuously comparing the intended, rendered, applied, and observed layers across all interfaces and correcting any discrepancies before they affect production traffic.