Design an Operator Workbench where an LLM Suggests Object Mappings, Relationship Fixes, and Taxonomy Cleanup for NetBox/Nautobot while Deterministic Validators and Human Approval Remain the Commit Boundary

Author: Sophia Lin – Cloud & Automation Architect

Introduction

NetBox/Nautobot Overview

NetBox and its fork Nautobot are open‑source IPAM/DCIB systems that model network topology as objects (Devices, Interfaces, IPAddresses, Prefixes, VLANs, etc.) and relationships (e.g., a Device contains an Interface, an Interface connected to another Interface).
Both expose a REST API (and, in Nautobot, a GraphQL endpoint) for external tools to read, create, update, and delete objects.

Typical operational workflows:

1. Discovery – pull data from SNMP, LLDP, or CMDBs.
2. Normalization – map discovered attributes to the canonical model.
3. Validation – enforce referential integrity (e.g., an Interface must belong to an existing Device).
4. Commit – write changes back to the source of truth.

When inventories reach thousands of devices, manual mapping and relationship correction become error‑prone and time‑consuming.

Role of LLMs in the Operator Workbench

An LLM acts as a suggestion engine that consumes unstructured or semi‑structured input (e.g., a spreadsheet of legacy asset tags, a ticket description, or a diff of two NetBox exports) and emits structured proposals for:

• Object mapping – how a legacy identifier should map to a NetBox object type and its fields.
• Relationship fixes – missing or incorrect links between objects (e.g., an Interface lacking a connected_endpoint).
• Taxonomy cleanup – normalizing inconsistent values (e.g., device role names, site prefixes) into a controlled vocabulary.

The LLM does not write to NetBox/Nautobot. Its output is fed to deterministic validators that check schema, referential integrity, and business rules. Only after a human operator explicitly approves the validated patch set is the change committed via the NetBox/Nautobot API. This preserves the commit boundary as a human‑in‑the‑loop gate.

Architecture

Components

Data Flow (ASCII diagram)

+----------------+    +----------------+    +-------------------+
| Ingestion      | -->| Context Store  | -->| LLM Suggestion    |
| Adapter        |    | (snapshot)     |    | Engine            |
+----------------+    +----------------+    +-------------------+
                                                    |
                                                    v
+----------------+    +----------------+    +-------------------+
| Deterministic  | <--| Approval       | <--| Commit Agent      |
| Validator      |    | Gateway (UI)   |    | (NetBox API)      |
+----------------+    +----------------+    +-------------------+
                                                    |
                                                    v
                                            +----------------+
                                            | Observability  |
                                            +----------------+

Integration with NetBox/Nautobot

• Read‑only snapshot – The Context Store is refreshed every N seconds (or via webhook) using a read‑only API token. This ensures the LLM works with a consistent view without risking accidental writes.
• Write path – The Commit Agent uses a separate write‑enabled API token with the minimal scope required (e.g., dcim:device:write, ipam:prefix:write). The token is stored in a vault (HashiCorp Vault, AWS Secrets Manager) and rotated regularly.
• Webhooks – NetBox/Nautobot can emit object_created, object_updated, object_deleted events; the Operator Workbench subscribes to these to keep the Context Store fresh and to trigger re‑validation of dependent suggestions.

LLM Suggestion Engine

The engine receives a prompt template that includes:

1. Task description (e.g., “Map the following legacy asset tags to NetBox device objects”).
2. Current schema – a trimmed JSON schema of the relevant NetBox models (to keep token usage low).
3. Batch of input records – each record contains the raw fields to be mapped.

The LLM is instructed to output a list of JSON Patch operations (add, replace, remove) targeting the NetBox REST endpoints. The engine then:

• Parses the LLM output with a tolerant JSON parser (e.g., demjson3).
• Normalizes patch paths to the canonical NetBox API format (/api/dcim/devices/{id}/).
• Wraps the patch set in a Proposal object:

{
  "proposal_id": "uuid",
  "timestamp": "2025-09-26T14:32:00Z",
  "source": "legacy_asset_csv",
  "object_type": "dcim.device",
  "patches": [ { "op": "add", "path": "/name", "value": "router01" }, … ],
  "metadata": { "confidence": 0.87, "token_usage": { "prompt": 420, "completion": 78 } }
}

The confidence field is derived from the LLM’s internal log‑probabilities (if available) or from a heuristic (e.g., proportion of fields that matched known enumerations).

Object Mapping Suggestions

LLM‑Driven Object Mapping

Object mapping translates a legacy identifier (e.g., an asset tag, a hostname pattern) into a NetBox object with appropriate field values. The LLM receives:

• Legacy fields – asset_tag, hostname, model_string, location_freeform.
• Canonical enumerations – list of valid device_role, device_type, site, tenant values (pulled from the Context Store).

The LLM must decide:

• Which NetBox object type (usually dcim.device).
• Which site and tenant the device belongs to.
• Which device_role and device_type match the model string.
• What values to assign to custom fields (if any).

Example Code

# suggestion_engine/object_mapping.py
import json
import os
import requests
from typing import List, Dict, Any

LLM_ENDPOINT = os.getenv("LLM_ENDPOINT", "http://localhost:8080/v1/completions")
LLM_MODEL = os.getenv("LLM_MODEL", "llama-2-7b-chat")
NETBOX_URL = os.getenv("NETBOX_URL", "https://netbox.example.com")
NETBOX_TOKEN = os.getenv("NETBOX_TOKEN")  # read‑only token for context

def fetch_context() -> Dict[str, Any]:
    """Pull reference data needed for the prompt."""
    headers = {"Authorization": f"Token {NETBOX_TOKEN}"}
    roles = requests.get(f"{NETBOX_URL}/api/dcim/device-roles/", headers=headers).json()["results"]
    types = requests.get(f"{NETBOX_URL}/api/dcim/device-types/", headers=headers).json()["results"]
    sites = requests.get(f"{NETBOX_URL}/api/dcim/sites/", headers=headers).json()["results"]
    return {
        "device_roles": [r["name"] for r in roles],
        "device_types": [f"{t['manufacturer']['name']} {t['model']}" for t in types],
        "sites": [s["name"] for s in sites],
    }

def build_prompt(records: List[Dict[str, str]], context: Dict[str, Any]) -> str:
    """Create a few‑shot prompt that asks the LLM to output JSON patches."""
    schema_hint = """
    You are a network automation assistant. Given a legacy asset record,
    output a JSON Patch (RFC 6902) that creates or updates a NetBox device.
    Only include the fields you are confident about.
    Valid device_role: %s
    Valid device_type (manufacturer model): %s
    Valid site: %s
    """ % (
        ", ".join(context["device_roles"]),
        ", ".join(context["device_types"]),
        ", ".join(context["sites"]),
    )
    examples = """
    Example input:
    {"asset_tag": "AT-1001", "hostname": "rtr01.nyc.example.com", "model_string": "Cisco ISR 4321", "location_freeform": "New York"}
    Example output:
    [
      { "op": "add", "path": "/name", "value": "rtr01" },
      { "op": "add", "path": "/device_role", "value": "router" },
      { "op": "add", "path": "/device_type", "value": "Cisco ISR 4321" },
      { "op": "add", "path": "/site", "value": "New York" },
      { "op": "add", "path": "/asset_tag", "value": "AT-1001" }
    ]
    """
    prompt = schema_hint + "\n\n" + examples + "\n\nNow process the following records:\n"
    for i, rec in enumerate(records, 1):
        prompt += f"{i}. {json.dumps(rec)}\n"
    prompt += "\nRespond with a JSON array of patch objects, one per record. Do not add any extra text."
    return prompt

def call_llm(prompt: str) -> List[Dict[str, Any]]:
    payload = {
        "model": LLM_MODEL,
        "prompt": prompt,
        "max_tokens": 1024,
        "temperature": 0.1,
        "logprobs": 1,
        "echo": False,
    }
    resp = requests.post(LLM_ENDPOINT, json=payload, timeout=60)
    resp.raise_for_status()
    text = resp.json()["choices"][0]["text"]
    # Attempt to extract JSON array; fallback to empty list on failure
    try:
        patches = json.loads(text)
        if not isinstance(patches, list):
            raise ValueError("LLM output not a list")
        return patches
    except Exception as e:
        # In production we would log and possibly retry with a stricter prompt
        print(f"[WARN] Failed to parse LLM output: {e}")
        return []

def suggest_object_mappings(records: List[Dict[str, str]]) -> List[Dict[str, Any]]:
    context = fetch_context()
    prompt = build_prompt(records, context)
    patches = call_llm(prompt)
    # Attach metadata for downstream consumption
    proposals = []
    for i, patch in enumerate(patches):
        proposals.append({
            "proposal_id": f"objmap-{i}",
            "timestamp": pd.Timestamp.utcnow().isoformat(),
            "source": "object_mapping",
            "object_type": "dcim.device",
            "patches": patch,
            "metadata": {"confidence": 0.8}  # placeholder; could be derived from logprobs
        })
    return proposals

CLI Commands for Object Mapping

# Install the suggestion engine (if not already installed)
pip install -e .

# Run object mapping on a CSV file of legacy assets
python -m suggestion_engine.object_mapping \
    --input assets.csv \
    --output proposals.json \
    --netbox-url https://netbox.example.com \
    --netbox-token $NETBOX_READONLY_TOKEN \
    --llm-endpoint http://localhost:8080/v1/completions \
    --llm-model llama-2-7b-chat

# Validate the generated proposals (deterministic layer)
python -m validation_layer.validate \
    --proposals proposals.json \
    --netbox-url https://netbox.example.com \
    --netbox-token $NETBOX_READONLY_TOKEN

# Approve proposals via the CLI gateway (example: interactive prompt)
python -m approval_gateway.cli \
    --proposals proposals.json \
    --output approved.json

# Commit approved changes to NetBox/Nautobot
python -m commit_agent.apply \
    --approved approved.json \
    --netbox-url https://netbox.example.com \
    --netbox-token $NETBOX_WRITE_TOKEN