Model explanations do not prove generated intent

Introduction to Trust in Automated Systems

Definition of Trust in Automated Systems

Trust in automated systems is the justified belief that a system will behave as intended under specified conditions, without causing unintended harm or violating operational policies. Unlike confidence derived from a fluent narrative (e.g., a language model’s persuasive explanation), trust is grounded in observable, repeatable evidence: artifacts that record state, tests that validate behavior, and execution bounds that limit the blast radius of changes.

Importance of Trust in Automated Systems

Network operators rely on automation to reduce manual toil, accelerate change, and improve consistency. However, each automated action expands the attack surface and the potential for silent failures. Trust enables:

• Safe delegation – operators can authorize scripts or agents to act without constant supervision.
• Auditability – every action can be traced back to a verified artifact or test result.
• Resilience – bounded execution limits the impact of a faulty change, allowing rapid containment and rollback.

Without trust, automation becomes a source of risk rather than efficiency, leading to change freezes, manual overrides, and erosion of confidence in the automation platform itself.

Limitations of Narrative Confidence

Masking Missing Constraints

A fluent rationale generated by an LLM or a well‑crafted runbook can convincingly explain why a change should succeed while omitting critical constraints such as:

• Interface‑specific MTU limits.
• VLAN pruning policies on trunk ports.
• Rate‑limit thresholds on control‑plane protocols (e.g., BGP hold timers).

When the narrative hides these constraints, the automation may issue a command that appears syntactically correct but is semantically invalid for the target device, resulting in configuration rejection or, worse, a silent misconfiguration that only surfaces under load.

Stale Inventory and Its Consequences

Automation often relies on an inventory source (CMDB, NetBox, DNS, or dynamic discovery) to determine which devices to touch. If that inventory is stale:

• Ghost devices – entries for decommissioned hardware cause attempts to SSH into non‑existent IPs, wasting time and potentially triggering lockout policies.
• Missing devices – new gear is omitted, leaving it unpatched or misconfigured.
• Incorrect attributes – stale interface descriptions or IP‑address mappings lead to applying the wrong policy (e.g., applying a QoS map to a port that no longer carries voice traffic).

The resulting drift between the automation’s view and the real network creates a false sense of correctness; the system reports success while the actual state diverges.

Impossible Rollback Paths and Their Implications

Narrative confidence may assert that a change is “easily reversible” by invoking a stored previous configuration or by re‑applying a baseline. However, rollback can become impossible when:

• The change modifies a stateful resource that cannot be restored by configuration alone (e.g., deleting a VLAN that had active ports, causing MAC‑table flushes that disrupt traffic).
• The rollback script depends on a feature or command that was deprecated or removed in the target OS version.
• The system lacks a persistent checkpoint (e.g., no configuration archive, or the archive was rotated out before the change).

If rollback is impossible, the network may remain in a degraded state until manual intervention, increasing mean‑time‑to‑recover (MTTR) and eroding trust in the automation framework.

Artifacts as a Basis for Trust

Types of Artifacts in Automated Systems

Role of Artifacts in Ensuring Trust

Artifacts provide verifiable, tamper‑evident evidence that:

1. Intent – what the operator meant to do (template, policy).
2. Pre‑condition – the state of the system before change (inventory snapshot, facts).
3. Execution – the exact commands issued and their raw output.
4. Post‑condition – the resulting state (post‑change show commands, state diff).

When each of these artifacts is cryptographically signed or stored in an append‑only log, an auditor can reconstruct the entire change lifecycle and confirm that no step deviated from the approved intent.

Examples of Artifacts in Real‑World Scenarios

• Git‑based network CI – A pull request renders Cisco IOS‑XR configs via Jinja2. The rendered files are stored as artifacts in the CI workflow; a subsequent job runs cisco_iosxr_config module and captures the device’s show running-config output as an artifact.
• Terraform state lock – When applying a change to AWS VPC resources that underlie a transit‑gateway‑based network, the Terraform state file (held in an S3 bucket with versioning) serves as the artifact proving which resources existed before the apply.
• OPA decision logs – Before allowing a BGP peer addition, an OPA policy evaluates the request against a “max‑peers‑per‑router” rule. The decision (allow/deny) and the evaluated input are logged as an artifact for later review.

Tests as a Foundation for Trust

Types of Tests in Automated Systems

Role of Tests in Ensuring Trust

Tests convert narrative confidence into empirical evidence. A passing test suite indicates that, under the exercised conditions, the automation behaves as expected. Crucially, tests must be:

• Deterministic – same inputs yield same outputs.
• Isolated – they do not rely on mutable external state unless that state is explicitly versioned.
• Automated – run on every commit or pull request in a CI pipeline.

When a test fails, the change is blocked before it reaches production, preventing the propagation of erroneous artifacts.

Code Examples for Implementing Tests

Python unit test for a Jinja2 filter that validates an IPv4 address:

# tests/test_ipaddr_filter.py
import ipaddress
import pytest
from jinja2 import Environment

def valid_ipaddr(value):
    try:
        ipaddress.IPv4Address(value)
        return value
    except ipaddress.AddressValueError:
        raise ValueError(f"Not a valid IPv4 address: {value}")

def test_valid_ipaddr():
    env = Environment()
    env.filters["valid_ipaddr"] = valid_ipaddr
    template = env.from_string("{{ ip | valid_ipaddr }}")
    assert template.render(ip="10.0.0.1") == "10.0.0.1"

def test_invalid_ipaddr():
    env = Environment()
    env.filters["valid_ipaddr"] = valid_ipaddr
    template = env.from_string("{{ ip | valid_ipaddr }}")
    with pytest.raises(ValueError):
        template.render(ip="999.999.999.999")

Ansible integration test using Molecule to verify BGP session state:

# molecule/default/converge.yml
- name: Apply BGP config and verify
  hosts: all
  become: true
  tasks:
    - name: Render and push BGP config
      ansible.builtin.template:
        src: bgp.j2
        dest: /etc/frr/frr.conf
      notify: restart frr

    - name: Wait for BGP ESTABLISHED
      ansible.builtin.wait_for:
        host: "{{ ansible_host }}"
        port: 179
        timeout: 30
      register: bgp_check
      until: bgp_check.elapsed > 0

    - name: Confirm BGP summary shows ESTABLISHED
      ansible.builtin.command: vtysh -c "show ip bgp summary"
      register: bgp_summary
      changed_when: false
      failed_when: "'Estab' not in bgp_summary.stdout"

CLI Examples for Running Tests

# Run unit tests with pytest
pytest -q tests/

# Execute Molecule scenario (requires Docker or podman)
molecule test --all

# Run Ansible playbook in check mode to see diff without applying
ansible-playbook -i inventory.yml site.yml --check --diff

# Execute OPA test suite
opa test . -v

Bounded Execution for Trust

Definition of Bounded Execution

Bounded execution is the practice of limiting the scope, duration, and side‑effects of an automated operation so that any failure remains contained within a predefined blast radius. Bounds can be expressed as:

• Device count – max number of devices touched per run.
• Time window – execution must complete within a configured timeout.
• Change type – only read‑only or specific command categories allowed.
• Rollback window – a guaranteed ability to revert to a known good state within a bounded time.

Importance of Bounded Execution in Ensuring Trust

Even with perfect artifacts and tests, automation can encounter unknown edge cases (e.g., a bug in the device OS). Bounded execution ensures that:

• A faulty run cannot cascade across the entire fabric.
• Operators can observe the impact in a limited set of devices and intervene quickly.
• Rollback or compensatory actions are feasible because the state space explored is small.

Implementing Bounded Execution in Automated Systems

1. Semaphore‑based concurrency limiter (example in Python using asyncio.Semaphore):

import asyncio
from typing import List

SEM_LIMIT = 10  # max concurrent devices

async def configure_device(sem: asyncio.Semaphore, device: str):
    async with sem:
        # Placeholder for actual NETCONF/SSH call
        print(f"Configuring {device}")
        await asyncio.sleep(1)  # simulate work

async def run_batch(devices: List[str]):
    sem = asyncio.Semaphore(SEM_LIMIT)
    await asyncio.gather(*[configure_device(sem, d) for d in devices])

# Usage
asyncio.run(run_batch(["rtr01", "rtr02", "...", "rtrN"]))

2. Time‑boxed execution with a watchdog (bash wrapper):

#!/usr/bin/env bash
set -euo pipefail

TIMEOUT=300   # seconds
MAX_DEVICES=20

# Get list of targets from inventory (example: NetBox API)
TARGETS=$(curl -s http://netbox/api/dcim/devices/?limit=1000 | jq -r '.results[].primary_ip4.address' | cut -d/ -f1)

# Trim to max devices
TARGETS=$(echo "$TARGETS" | head -n $MAX_DEVICES)

# Execute with timeout
timeout "$TIMEOUT" ansible-playbook -i <(echo "$TARGETS") site.yml

if [ $? -eq 124 ]; then
    echo "Execution exceeded $TIMEOUT seconds – aborting"
    exit 1
fi

3. Declarative rollback boundaries using Terraform’s -target and state snapshots:

# Take a snapshot before apply
terraform state pull > pre-apply.tfstate

# Apply only a subset of resources (bounded by tag)
terraform apply -target='aws_security_group.[*]"tag:Environment"="staging"' -auto-approve

# On failure, restore snapshot
terraform state push pre-apply.tfstate

Troubleshooting Common Issues in Bounded Execution

Scaling Limitations of Trust in Automated Systems

Horizontal Scaling Limitations

• Artifact storage contention – Git repositories or object stores can become bottlenecks when thousands of concurrent pushes/pulls occur. Mitigation: use sharded backends (e.g., GitLab Geo, S3 multipart upload with prefix partitioning) and employ pull‑through caches.
• Test execution fan‑out – Running a full test suite per device leads to O(N×M) complexity (N devices, M tests). Mitigation: adopt test selection based on changed files (e.g., using git diff --name-only) and maintain a test‑impact analysis matrix.
• Bounded execution coordination – Centralized semaphores or watchdogs can become a single point of failure. Mitigation: distribute limiters via a token bucket service (e.g., Redis-backed rate limiter) or use Kubernetes Job parallelism with a podDisruptionBudget.

Vertical Scaling Limitations

• Device API rate limits – NETCONF/SSH or RESTCONF interfaces on hardware have finite session limits