How NXDOMAIN and NODATA Answers Are Cached During Bootstrap

Negative DNS Responses

When a recursive resolver queries a name that either does not exist (NXDOMAIN) or exists but lacks the requested RR type (NODATA), the authoritative server returns an answer with the AA flag set.

• NXDOMAIN – RCODE = NXDOMAIN; the authority section carries the SOA of the closest enclosing zone.
• NODATA – RCODE = NOERROR with an empty answer section; the authority section holds the SOA of the zone.

Both responses are authoritative and therefore cacheable. The resolver stores a negative cache entry keyed by (query name, query type, IN). The TTL for that entry is taken from the SOA’s minimum field (or the SOA’s own TTL if the resolver follows RFC 2308’s “negative caching TTL” rule). The entry expires after now + TTL.

Why the Service Can Be Up Before Clients See It

During system or container bootstrap:

1. Network interfaces are configured.
2. The local stub resolver (systemd-resolved, dnsmasq, libc) starts and forwards queries to upstream recursors defined in /etc/resolv.conf.
3. Applications immediately perform DNS lookups for service discovery.

If the upstream recursors have not yet cached the positive RRset (or still hold a stale negative entry from a previous cycle), they will return the cached NXDOMAIN/NODATA answer even though the authoritative server is already serving the correct record. The resolver’s view of the namespace lags behind the actual service availability, causing a false failure that persists until the negative TTL expires or the cache is flushed.

How Negative Answers Are Cached

1. Extract TTL – For NXDOMAIN or NODATA, the TTL is the SOA’s minimum field (or the SOA TTL per RFC 2308).
2. Create entry – Key = (name, type, IN). Value = {type: NEGATIVE, ttl: extracted TTL, expiry: now + TTL, optional: SOA RRset}.
3. Insert/replace – If an entry for the same key exists, replace it only when the new TTL is greater than the remaining TTL (most resolvers use a “refresh on longer TTL” policy).
4. Set expiration – The resolver stores an absolute expiry time and checks it lazily on lookup or purges it via a background janitor thread.

The entry remains until the TTL elapses; the next lookup for the same (name, type) triggers a fresh upstream query.

TTL Boundaries That Prolong False Failure

The length of the false‑failure window equals the TTL stored in the negative cache entry. Because this TTL often comes from the SOA’s minimum field, zone administrators can unintentionally create long windows:

If the authoritative zone adds the missing record before the negative TTL expires, resolvers will continue to return the cached error, making the service appear unavailable to clients.

Troubleshooting Stale Negative Cache

Symptoms

• Repeated SERVFAIL or NXDOMAIN from the resolver while dig @<authoritative> returns the correct answer.
• The issue affects all clients using the same resolver but disappears when querying a different resolver (e.g., 1.1.1.1).
• Failure duration matches a round TTL value (e.g., exactly 1 hour) seen in logs or packet captures.

Verification steps

1. Confirm the authoritative zone contains the expected RRset.
2. Check the resolver’s cache for a negative entry matching the query name and type.

CLI Tools for Cache Inspection and Flush

BIND (named)

# Flush entire cache
sudo rndc flush

# Flush a specific name/type (BIND 9.11+)
sudo rndc flushname example.com A

# View statistics (includes negative cache hits/misses)
sudo rndc stats
# Then examine /var/named/named.stats for:
#   negativecache: <hits> hits, <misses> misses

Unbound

# Flush whole cache
sudo unbound-control flush_cache

# Flush a specific RRset
sudo unbound-control flush_example example.com A IN

# Dump cache for offline analysis
sudo unbound-control dump_cache > /tmp/unbound_cache.db

# Show statistics (look for negativecachehits)
sudo unbound-control stats_noreset

dnsquery (raw DNS query)

# Non‑recursive query to see if the resolver has authoritative data
dnsquery @127.0.0.1 -t A -norecurse example.com
# AA=0 and empty answer → likely a cached NODATA/NXDOMAIN.

Capture traffic with tcpdump -i any port 53 -w /tmp/dns.pcap and compare the resolver’s response to a direct query to the authoritative server. Matching TTL values and differing RCODE/AA flags confirm a cache hit.

Managing Negative Cache TTL

BIND Configuration

options {
    max-negative-ttl 300;   // cap negative TTL at 5 minutes
    channel negative_log {
        file "/var/log/bind/negative.log" versions 3 size 5m;
        severity info;
    };
    category negative { negative_log; };
};

max-negative-ttl forces named to ignore any TLA larger than the specified value when caching NXDOMAIN/NODATA, shortening the false‑failure window.

Unbound Configuration

server:
    neg-cache-size 4m;          // memory for negative cache
    neg-cache-ttl 300;          // TTL cap for negative entries (seconds)
    log-replies: yes
    log-local-actions: yes

neg-cache-ttl overrides the SOA‑derived TTL, ensuring negative entries expire sooner.

Programmatic Cache Invalidation

Python (dnspython + Unbound control socket)

import socket

def unbound_cmd(sock_path, cmd):
    with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
        s.connect(sock_path)
        s.sendall((cmd + "\n").encode())
        resp = b""
        while True:
            chunk = s.recv(4096)
            if not chunk:
                break
            resp += chunk
            if b"\n" in chunk:
                break
        return resp.decode().strip()

# Flush a specific name/type
print(unbound_cmd("/var/run/unbound.control.sock",
                  "flush_example example.com A IN"))

Node.js (unbound-control via child_process)

const { execSync } = require('child_process');

function flushUnbound(name, type = 'A', klass = 'IN') {
    const cmd = `unbound-control flush_example ${name} ${type} ${klass}`;
    try {
        const out = execSync(cmd, { encoding: 'utf8' });
        console.log('Flush output:', out.trim());
    } catch (e) {
        console.error('Flush failed:', e.message);
    }
}

flushUnbound('api.internal.example.com');

These snippets let operators instantly purge a stale negative entry, reducing the false‑failure window to near‑zero when a service becomes available.

Scaling and Operational Considerations

• Memory usage – Negative caches store only an SOA per (name, type) key, but in environments blocking millions of domains they can grow large. Monitor max-cache-size (BIND) or neg-cache-size (Unbound) and adjust to avoid LRU eviction of useful entries.
• Clustered resolvers – When multiple resolver instances share a load balancer, cache incoherency can prolong false failures. Use a push‑based invalidation mechanism (e.g., a Pub/Sub topic that each node subscribes to) to propagate flushes promptly.
• Consistent TTL caps – Ensure all nodes apply the same max-negative-ttl / neg-cache-ttl policy; divergent settings cause varied client experiences.
• Logging – Enable negative‑cache logging (as shown above) to spot unexpected long‑lived entries during troubleshooting.

Best‑Practice Summary

1. Understand the source – Negative TTL comes from the SOA’s minimum field (or SOA TTL per RFC 2308).
2. Cap the TTL – Use resolver‑specific limits (max-negative-ttl, neg-cache-ttl) to bound the false‑failure window.
3. Validate bootstrap timing – Delay service‑dependent lookups until after the resolver has had a chance to prime its cache, or prime the cache with a warm‑up query.
4. Monitor and flush – Regularly check cache statistics and flush specific negative entries when a service becomes known to be available.
5. Ensure cluster coherency – Propagate cache invalidations quickly across resolver instances to avoid staggered client views.

By controlling how long negative answers are retained and by clearing them promptly when the underlying data changes, you can eliminate the misleading “service not found” errors that often appear during system bootstrap.