Benchmarking RX Interrupt Moderation Before and After XDP Deployment

RX Interrupt Moderation

Definition and Purpose

RX interrupt moderation trades latency for interrupt rate. By waiting to accumulate packets or time, the NIC reduces the number of interrupts the CPU must handle, lowering interrupt‑handling CPU usage and cache pressure. The trade‑off is increased latency variance (jitter) because packets may sit in the NIC’s internal buffers until the moderation condition is met.

How It Works

1. Packet arrival – NIC DMA writes packet data to a receive ring buffer.
2. Counter/timer update – Driver increments a per‑queue packet counter and (re)starts a timeout timer.
3. Moderation check – If packet_counter >= rx-frames or elapsed_time >= rx-usecs, the NIC asserts an interrupt line.
4. Interrupt handling – The CPU enters the interrupt handler, which schedules the NAPI poll loop (netif_napi_schedule). The NAPI loop then pulls packets from the ring, processes them, and re‑enables interrupts.

If the driver works in NAPI poll mode (most modern drivers), the interrupt merely signals that there is work; the actual packet processing happens in the poll loop, which can run for a configurable weight number of packets before yielding.

Importance in Network Performance

• Interrupt rate – Directly correlates with CPU cycles spent in interrupt entry/exit and cache line bouncing.
• CPU utilization – Lower interrupt rates free cycles for application work, but only if the NAPI loop can keep up with the packet rate.
• Latency/jitter – Excessive moderation can cause bursts of packets to be delivered together, increasing tail latency and making latency‑sensitive applications see jitter.
• Core feeding – In a multi‑queue, RSS setup, each queue’s interrupt moderation influences how often a particular core is awakened. Too little work per interrupt can leave cores idle; too much can cause overload.

Impact of XDP on Interrupt Moderation

XDP Deployment Process and Configuration

1. Write XDP program – A simple C program that either passes or drops packets based on a filter.
2. Compile – Using clang with -target bpf -O2 -Wall -target bpf.
3. Load – Via ip link set dev eth0 xdp obj xdp_prog.o sec xdp.
4. Verify – ip -d link show eth0 shows the xdp flag; ethtool -k eth0 shows offload flags unchanged.

Example XDP source (xdp_drop_icmp.c):

#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/icmp.h>

SEC("xdp")
int xdp_drop_icmp(struct xdp_md *ctx)
{
    void *data_end = (void *)(long)ctx->data_end;
    void *data = (void *)(long)ctx->data;
    struct ethhdr *eth = data;

    if (data + sizeof(*eth) > data_end)
        return XDP_PASS;

    if (eth->h_proto == htons(ETH_P_IP)) {
        struct iphdr *ip = data + sizeof(*eth);
        if (data + sizeof(*eth) + sizeof(*ip) > data_end)
            return XDP_PASS;
        if (ip->protocol == IPPROTO_ICMP)
            return XDP_DROP;
    }
    return XDP_PASS;
}
char _license[] SEC("license") = "GPL";

Compile and load:

clang -O2 -target bpf -c xdp_drop_icmp.c -o xdp_drop_icmp.o
ip link set dev eth0 xdp obj xdp_drop_icmp.o sec xdp

Changes in Network Performance After XDP Deployment

• CPU cycles per packet – Drop path: ~300 cycles (XDP) vs ~1500 cycles (full kernel pass). Measured via perf stat -e cycles -p <pktgen-pid>.
• Interrupt rate – Because fewer packets reach the NAPI layer, the same moderation settings yield fewer interrupts per second.
• Potential effects – Larger interrupt intervals, increased jitter, or core under‑utilization if moderation values were