Encoding Unknowns in Network Automation Agents

Introduction

In network automation and AI‑assisted operations, agents rely on command‑line outputs, parser results, and intermediate state to make decisions. When a command returns no data, a stale value, or information outside the intended scope, treating that absence as a definitive answer can lead to fake certainty—the agent acts as if it knows the truth when it actually does not. Encoding unknowns explicitly preserves the epistemic distinction between known true, known false, and unknown (or uncertain), enabling safe fallback, human‑in‑the‑loop review, and deterministic rollback.

Types of Unknowns

Each unknown type must survive parsing, aggregation, and reasoning layers without being coerced without loss of meaning.

Command Wrappers

Design Principles

A command wrapper executes a CLI or API call, captures raw output, and returns a structured result enriched with metadata about certainty. It should:

1. Distinguish three outcome classes – success with data, success with empty data, and failure (non‑zero exit or error response).
2. Timestamp the acquisition for stale‑read detection.
3. Record the command’s effective scope (VRF, namespace, RBAC role) as declared in the wrapper’s configuration.
4. Emit a standardized unknown: true flag when the output is empty or a precondition violation is detected (e.g., missing privilege).
5. Preserve raw output for diagnostics, but never treat it as authoritative when unknown is set.

Standard JSON Schema

{
  "command": "show interface eth0",
  "exit_code": 0,
  "stdout": "",
  "stderr": "",
  "timestamp": "2025-09-16T14:32:10Z",
  "scope": {"vrf": "default", "namespace": "netops"},
  "unknown": true,
  "unknown_reason": "empty_output",
  "parsed": null
}

• unknown is Boolean; when true, parsed must be null (or omitted).
• asonis an enum:empty_output, stale_read, scope_violation, parse_error`.
• scope mirrors the agent’s view of the command’s authority; a mismatch triggers scope_violation.

Python Implementation

#!/usr/bin/env python3
import json
import subprocess
import shlex
from datetime import datetime, timezone
from typing import Dict, Any, Optional

class CliWrapper:
    def __init__(self, timeout: int = 10, stale_threshold_sec: int = 30):
        self.timeout = timeout
        self.stale_threshold = stale_threshold_sec

    def run(self, cmd: str, scope: Optional[Dict[str, str]] = None) -> Dict[str, Any]:
        start = datetime.now(timezone.utc)
        try:
            proc = subprocess.run(
                shlex.split(cmd),
                capture_output=True,
                text=True,
                timeout=self.timeout,
                check=False,
            )
            exit_code = proc.returncode
            stdout = proc.stdout.strip()
            stderr = proc.stderr.strip()
        except subprocess.TimeoutExpired as e:
            return self._error_result(cmd, start, "timeout", str(e), scope)
        except Exception as e:
            return self._error_result(cmd, start, "exception", str(e), scope)

        # Determine unknown status
        unknown = False
        reason = None
        if exit_code != 0:
            unknown = True
            reason = "non_zero_exit"
        elif not stdout:
            unknown = True
            reason = "empty_output"
        else:
            age = (datetime.now(timezone.utc) - start).total_seconds()
            if age > self.stale_threshold:
                unknown = True
                reason = "stale_read"

        parsed = None if unknown else self._parse_output(cmd, stdout)

        return {
            "command": cmd,
            "exit_code": exit_code,
            "stdout": stdout,
            "stderr": stderr,
            "timestamp": start.isoformat(),
            "scope": scope or {},
            "unknown": unknown,
            "unknown_reason": reason,
            "parsed": parsed,
        }

    def _error_result(self, cmd: str, start: datetime, err_type: str, msg: str,
                      scope: Optional[Dict[str, str]]) -> Dict[str, Any]:
        return {
            "command": cmd,
            "exit_code": -1,
            "stdout": "",
            "stderr": msg,
            "timestamp": start.isoformat(),
            "scope": scope or {},
            "unknown": True,
            "unknown_reason": err_type,
            "parsed": None,
        }

    def _parse_output(self, cmd: str, raw: str) -> Any:
        # Placeholder: implement command‑specific parsers elsewhere
        return raw.splitlines()


# Usage example
if __name__ == "__main__":
    wrapper = CliWrapper(stale_threshold_sec=5)
    result = wrapper.run("show ip interface brief", scope={"vrf": "default"})
    print(json.dumps(result, indent=2))

Parser Outputs and Intermediate State Models

Parser Output Format

Parsers consume the stdout field of a wrapper result and must emit a typed unknown when they cannot produce a deterministic value. We use a tagged union (discriminated union) representation:

{
  "type": "interface_state",
  "value": "up",
  "unknown": false
}

or, when unknown:

{
  "type": "interface_state",
  "unknown": true,
  "unknown_reason": "missing_oid",
  "raw": ["", ""]
}

• type enables downstream routers to dispatch to the correct handler without inspecting the payload.
• unknown_reason enumerates parser‑specific failure modes (e.g., regex_no_match, unexpected_column_count, stale_snapshot).

Intermediate State Model

Intermediate state is the materialized view an agent maintains after each observation cycle. It must preserve unknownness across time steps to support temporal reasoning (e.g., “the interface was unknown for three consecutive polls → likely down”). A suitable model is a versioned key‑value store where each entry is:

<entity> ::= {
    "attribute": <string>,
    "value": <TypedValue>,
    "ts": <timestamp>,
    "source": <wrapper_id>,
    "confidence": <float in [0,1]>,
    "history": [<previous_entry>, ...]
}

• TypedValue follows the parser output schema (known or unknown).
• confidence can be derived from the inverse of staleness age or from the number of consecutive unknown reports.

Merge rule:

• If new.unknown == true → keep existing value only if new.ts - existing.ts < stale_grace_period; otherwise replace with unknown.
• If both known and values differ → raise a conflict event that may trigger human review.

CLI Example: Parsing BGP Summary

Assume a wrapper produced the following JSON (truncated):

{
  "command": "show ip bgp summary",
  "exit_code": 0,
  "stdout": "BGP router identifier 10.0.0.1, local AS number 65001\nNeighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd\n* 10.1.1.1       4 65002      1234    5678      12      0    0 00:12:34        100\n",
  "timestamp": "2025-09-16T14:32:10Z",
  "scope": {"vrf": "default"},
  "unknown": false,
  "unknown_reason": null,
  "parsed": null
}

A simple parser invoked via bgp-parser:

$ echo "$(cat wrapper_output.json | jq -r '.stdout')" | bgp-parser --format json
[
  {
    "type": "bgp_neighbor",
    "neighbor": "10.1.1.1",
    "state": "Established",
    "prefix_received": 100,
    "unknown": false
  }
]

If the output were empty (BGP process not running), the wrapper would set unknown: true. The parser, seeing no lines, would emit:

{
  "type": "bgp_neighbor",
  "unknown": true,
  "unknown_reason": "no_neighbor_lines",
  "raw": []
}

Encoding Stale Reads and Scope Limits

Detecting Stale Reads

Staleness is a property of the observation time, not the parsed value. The wrapper supplies a timestamp. Downstream components can compute an age:

def is_stale(entry: dict, max_age_sec: int = 60) -> bool:
    ts = datetime.fromisoformat(entry["timestamp"].replace("Z", "+00:00"))
    age = (datetime.now(timezone.utc) - ts).total_seconds()
    return age > max_age_sec

When is_stale returns true, treat the entry as unknown regardless of the wrapper’s unknown flag. This two‑layer protection catches cases where the wrapper succeeded (non‑empty output) but the data is outdated.

Enforcing Scope Limits

Scope limits are enforced at the wrapper level by comparing the requested scope (passed in by the agent) against the effective scope derived from the execution context (e.g., VRF from show vrf, network namespace from ip netns identify). A wrapper can:

1. Pre‑flight: run a scoped introspection command (e.g., show vrf | include <requested_vrf>) and abort if the VRF is not present.
2. Post‑flight: parse any scope‑indicating fields in the output (e.g., VRF: default in an interface line) and verify they match the request.
3. Emit unknown_reason: "scope_violation" with details.

Python extension:

def _validate_scope(self, stdout: str, requested: dict) -> (bool, Optional[str]):
    if "vrf" in requested:
        vrf_req = requested["vrf"]
        import re
        match = re.search(rf"VRF:\s*{re.escape(vrf_req)}", stdout, re.IGNORECASE)
        if not match:
            return False, f"VRF mismatch: expected {vrf_req}"
    return True, None

If validation fails, the wrapper returns unknown: true and unknown_reason: "scope_violation".

Troubleshooting Guide

Example troubleshooting script:

#!/usr/bin/env bash
# stale-check.sh: compare wrapper timestamp with device sysUpTime
WRAPPER_JSON=$1
MAX_AGE=30

TS=$(jq -r '.timestamp' "$WRAPPER_JSON")
AGE=$(( $(date +%s) - $(date -d "$TS" +%s) ))
if (( AGE > MAX_AGE )); then
    echo "Stale read: age $AGE seconds > $MAX_AGE"
    exit 1
else
    echo "Fresh: age $AGE seconds"
    exit 0
fi

Avoiding Fake Certainty in Agents

Converting Absence of Evidence to Unknowns

An agent must treat any observation lacking a definitive known label as unknown until evidence accumulates. Implementation steps:

• Initial state: all attributes start as unknown: true.
• Update rule: transition to known only when a parser returns unknown: false and the observation passes freshness and scope checks.
• Decay: if N consecutive observations are unknown, the attribute reverts to unknown (or triggers an alarm).

Three‑Valued Logic (3VL)

Three‑valued logic uses the truth values True, False, and Unknown (⊥). Operators follow Kleene or Łukasiewicz tables. Example conjunction:

An agent can encode each attribute as an object with a tribool field and evaluate policies using these tables, ensuring that missing information never forces a definitive True or False outcome.

Java Example: Tribool Enum

public enum Tribool {
    TRUE, FALSE, UNKNOWN;

    // Kleene conjunction
    public Tribool and(Tribool other) {
        if (this == FALSE || other == FALSE) return FALSE;
        if (this == TRUE && other == TRUE) return TRUE;
        return UNKNOWN;
    }

    // Kleene disjunction
    public Tribool or(Tribool other) {
        if (this == TRUE || other == TRUE) return TRUE;
        if (this == FALSE && other == FALSE) return FALSE;
        return UNKNOWN;
    }

    // Negation
    public Tribool not() {
        if (this == UNKNOWN) return UNKNOWN;
        return this == TRUE ? FALSE : TRUE;
    }
}

Using Tribool, an agent can safely evaluate complex conditions without converting absence of evidence into false certainty.

By explicitly encoding unknowns at the wrapper, parser, and state‑model layers, and by applying three‑valued logic in decision‑making, agents avoid the pitfalls of fake certainty and operate safely in dynamic, partially observable network environments.