NAT can make ACL unit tests lie

Pre-NAT Classification and Translation

Packet Flow Through Pre-NAT Classification

Consider a client 10.0.0.5:54321 contacting a web service 203.0.113.10:80 behind a NAT gateway 198.51.100.1. The packet enters the gateway on eth0.

1. PREROUTING (raw) – optional early‑drop, not used here.
2. PREROUTING (mangle) – classification:

   # Mark packets destined for the virtual IP for DNAT
   iptables -t mangle -A PREROUTING -d 203.0.113.10 -p tcp --dport 80 -j MARK --set-mark 0x1

3. PREROUTING (nat) – DNAT based on mark or direct match:

   iptables -t nat -A PREROUTING -m mark --mark 0x1 -j DNAT --to-destination 10.10.10.20:80

   At this point the packet’s destination becomes 10.10.10.20:80. The conntrack entry is created:

   orig-src=10.0.0.5 orig-dst=203.0.113.10 orig-sport=54321 orig-dport=80
   repl-src=10.10.10.20 repl-dst=198.51.100.1 repl-sport=80 repl-dport=54321

4. Routing lookup – kernel routes to the internal network (eth1).

Translation Techniques and Methods

Classification can also be performed with nftables:

nft add table ip nat
nft add chain ip nat prerouting { type nat hook prerouting priority -100; }
nft add rule ip nat prerouting ip daddr 203.0.113.10 tcp dport 80 dnat to 10.10.10.20:80

Example Code for Pre-NAT Configuration

A minimal Containerlab topology that places a Linux router (vr-nat) between two hosts:

name: nat-demo
topology:
  nodes:
    h1:
      kind: linux
      image: ubuntu:22.04
    h2:
      kind: linux
      image: ubuntu:22.04
    vr-nat:
      kind: linux
      image: ubuntu:22.04
      cmd: /sbin/init
      sysctls:
        net.ipv4.ip_forward: "1"
  links:
    - endpoints: ["h1:eth1", "vr-nat:eth1"]
    - endpoints: ["h2:eth1", "vr-nat:eth2"]

On vr-nat we configure DNAT for traffic to 203.0.113.10 (assigned to vr-nat:eth1) and SNAT for outbound traffic:

# Inside vr-nat container
ip addr add 203.0.113.10/32 dev eth1
ip addr add 10.10.10.1/24 dev eth2
sysctl -w net.ipv4.ip_forward=1

# DNAT: inbound web traffic to internal server 10.10.10.20
iptables -t nat -A PREROUTING -i eth1 -d 203.0.113.10 -p tcp --dport 80 -j DNAT --to-destination 10.10.10.20:80

# SNAT: outbound traffic from internal net uses the public address
iptables -t nat -A POSTROUTING -o eth1 -s 10.10.10.0/24 -j SNAT --to-source 203.0.113.10

Post-NAT Filters and Return-Path Checks

Configuring Post-NAT Filters

After DNAT, the packet is routed to the internal host. The FORWARD chain enforces whether the translated packet may be forwarded:

# Allow only HTTP to the internal web server
iptables -A FORWARD -i eth2 -o eth1 -d 10.10.10.20 -p tcp --dport 80 -j ACCEPT
iptables -A FORWARD -i eth2 -o eth1 -j DROP

For the return path, the conntrack entry ensures the inverse DNAT/SNAT is applied. However, administrators often add explicit POSTROUTING filters to log or drop unexpected translations:

# Log any packet leaving eth1 whose source is not the expected NAT address
iptables -t nat -A POSTROUTING -o eth1 ! -s 203.0.113.10 -j LOG --log-prefix "UNEXPECTED_SNAT: "

Implementing Return-Path Checks

Return‑path validation relies on the conntrack state. The kernel checks:

1. Direction – if the packet matches orig-src/dst → apply DNAT; if it matches repl-src/dst → apply SNAT.
2. State – only ESTABLISHED or RELATED packets are allowed to traverse the reverse NAT; INVALID packets are dropped.

To inspect the conntrack table:

conntrack -L -p tcp --dport 80

Sample output (truncated):

tcp      6 431999 ESTABLISHED src=10.0.0.5 dst=203.0.113.10 sport=54321 dport=80 src=10.10.10.20 dst=198.51.100.1 sport=80 dport=54321 [ASSURED] mark=0 use=1

If a packet arrives that does not match any conntrack entry, it is treated as NEW and subjected to the NAT rules again—this is where mis‑ordered rules can cause double‑translation or leaks.

CLI Examples for Post-NAT Filter Configuration

Using nftables for symmetric filtering:

nft add table ip filter
nft add chain ip filter forward { type filter hook forward priority 0; policy drop; }
nft add rule ip filter forward iif eth2 oif eth1 ip daddr 10.10.10.20 tcp dport 80 accept

nft add chain ip filter postrouting { type nat hook postrouting priority 100; }
nft add rule ip filter postrouting oif eth1 ip saddr != 203.0.113.10 log prefix "BAD_SNAT: "

Troubleshooting NAT Translation Issues

Identifying Translation Boundaries

The first step is to locate where the packet’s address/port changes. Use tcpdump on each interface and compare:

# On eth0 (public side)
tcpdump -i eth0 -nn -s 0 -c 5 'tcp port 80'

# On eth1 (internal side)
tcpdump -i eth1 -nn -s 0 -c 5 'tcp port 80'

If the source address changes between the two captures, NAT occurred on the outbound interface; if the destination changes, DNAT occurred on the inbound interface.

Debugging Techniques for NAT Issues

1. Verify conntrack entry – conntrack -L -p tcp --dport 80. Missing entry → classification failed.
2. Check NAT counters – iptables -t nat -L -v -n shows packet/byte counts per rule.
3. Log raw packets – add a NFLOG target in the raw table to see pre‑NAT packets:

   iptables -t raw -A PREROUTING -j NFLOG --nflog-prefix "PRE-NAT: "

4. Use nfct (netfilter conntrack tool) to inject test entries and observe translation:

   echo "src=10.0.0.5 dst=203.0.113.10 sport=54321 dport=80" | nfct add

5. Enable kernel debug (temporarily):

   echo 1 > /proc/sys/net/netfilter/nf_log_all_netns

Common Pitfalls in NAT Configuration

Scaling Limitations of NAT

NAT Table Size Limitations

Linux conntrack stores each flow as a struct nf_conn. The default limit is often 65536 entries, configurable via:

sysctl -w net.netfilter.nf_conntrack_max=262144

Each entry consumes ~1 KB of kernel memory; on a 2 GB RAM system, ~2 M entries is realistic before memory pressure triggers kmalloc failures and packet drops.

Performance Impacts of NAT on Network Traffic

• Per‑packet cost: classification (mangle/raw), hash lookup in conntrack, possible NAT rewrite, checksum recalculation.
• Lock contention: the conntrack hash table uses per‑bucket spinlocks; high concurrent flow rates can cause CPU saturation.
• Checksum offload: if NIC offloads TX checksum, NAT must disable offload or recalculate in software, adding latency.

Empirical numbers (Linux 5.15, Intel Xeon E5‑2680 v4):