Config archaeology with grep, Git, and local models

Introduction to Legacy Knob Discovery

A legacy knob is a configuration parameter, feature flag, or runtime switch that persists long after its original purpose has been superseded, deprecated, or rendered obsolete. Examples include obsolete sysctl tunables (e.g., net.ipv4.tcp_fack), deprecated CLI commands on routers (e.g., no service pad), or stale feature flags in controller software.

Discovering why such a knob still exists is critical for:

Risk reduction – unused knobs can be toggled accidentally, causing instability or security exposure.
Technical debt clearance – removing dead code simplifies maintenance and improves auditability.
Compliance – many standards require justification for every configurable item.
Operational clarity – engineers waste time investigating knobs that have no effect.

Three complementary approaches answer the question “Why does this knob still exist?”:

Approach	Core Idea	Typical Tools	Strengths
Deterministic Text Search	Literal pattern matching against source, documentation, and configuration files.	`grep`, `rg` (ripgrep), `awk`, `sed`, IDE search, `git grep`	Exact, reproducible, zero false‑negatives for the searched token.
Scripted History Mining	Analyzing version‑control history to trace when a knob was added, modified, or last touched, and extracting associated rationale (commit messages, ticket IDs, review comments).	`git log`, `git blame`, `git show`, `git diff`, custom Python/Shell scripts, `git-filter-repo` for large repos	Provides temporal context and human‑authored explanations; works even if the knob is no longer referenced in current code.
AI‑Assisted Explanation	Using LLMs or retrieval‑augmented generation (RAG) to synthesize a natural‑language answer from correlated artifacts (code, commits, docs, issue trackers).	Local LLMs (e.g., Llama‑2‑7B, Mistral), hosted APIs (OpenAI GPT‑4, Anthropic Claude), vector stores (FAISS, Chroma), LangChain/LlamaIndex pipelines	Can surface implicit rationale, summarize noisy histories, and answer “why” questions that are not explicitly recorded.

The following sections detail each method, illustrate usage with examples, and discuss limitations.

Deterministic Text Search

Methodology and Tools

Treat the legacy knob as a needle and the repository (or a set of artifacts) as a haystack. Locate every occurrence of the knob’s identifier and inspect surrounding context for intent clues.

Typical workflow

Define the search token – exact string, case‑sensitive/insensitive, possibly with word boundaries (\b).
Select the search scope – source tree (src/), documentation (docs/), configuration templates, or binary strings (strings + grep).
Run the search – using a tool that supports regex, line numbers, and context (-A, -B, -C).
Post‑process results – filter false positives (e.g., mere mentions) and aggregate unique file paths.
Inspect surrounding lines – look for associated documentation, TODO, FIXME, or conditional compilation flags (#ifdef LEGACY_KNOB).

Tool choices

grep -Rn – POSIX baseline, universally available.
rg (ripgrep) – faster, respects .gitignore, supports Unicode, can output JSON (--json).
git grep – limits search to tracked files, useful inside a repo.
IDE search (VS Code, IntelliJ) – click‑to‑navigate and refactoring preview.

Example Use Case

Scenario – A network OS still exposes the CLI knob service tcp-small-servers. Determine why it remains.

# Clone the repo (if needed)
git clone https://github.com/example/netops.git
cd netops

# Use ripgrep to find all references with two lines of context
rg -n --context 2 '\bservice\s+tcp-small-servers\b' src/ docs/ templates/

Sample output (truncated)

src/cli/commands/service.c:45:    if (strcmp(argv[1], "tcp-small-servers") == 0) {
src/cli/commands/service.c:46-47:        /* Legacy knob retained for backward compatibility */
src/cli/commands/service.c:48:        enable_tcp_small_servers(true);
--
docs/reference/cli-guide.adoc:112:   * `service tcp-small-servers` – Enables the legacy TCP small services (echo, discard, chargen, daytime). 
docs/reference/cli-guide.adoc:113:     *Deprecated since 7.0; kept for compatibility with older scripts.*

The comment indicates the knob is retained for backward compatibility, and the docs note a deprecation notice.

Alternative: searching for a compile‑time macro

rg -n --context 3 '\bENABLE_TCP_SMALL_SERVERS\b' src/

Might reveal:

src/include/features.h:27:#define ENABLE_TCP_SMALL_SERVERS 1   /* kept for old embeddings */

Limitations and Challenges

Issue	Why it matters	Mitigation
Lexical variance – knob appears under different names/abbreviations.	Misses relevant occurrences.	Build a synonym list (e.g., from alias tables) and run multiple searches.
Comments/documentation drift – knob mentioned only in outdated comments.	Gives false sense of relevance.	Cross‑check with blame/history to see if comment is stale.
Binary artifacts – knob strings embedded in firmware or compiled binaries.	Search yields no hits despite runtime presence.	Use `strings` on binaries or examine firmware manifests.
Scale – very large monorepos (>10 M lines) make naïve grep slow.	Increases iteration time.	Use `rg` with `--files` to limit to relevant file types, or employ indexed search (e.g., Sourcegraph, Hound).
Context loss – single line match may not reveal rationale spread across files.	Requires manual navigation.	Combine with scripted history mining to see commit‑level rationale.

Deterministic search excels at presence/absence checks and locating explicit documentation, but it cannot infer why a knob was kept if the rationale lives only in commit messages or ticket discussions.

Scripted History Mining

Methodology and Tools

Treat the version‑control system (VCS) as a temporal log of intent. By walking the commit graph, extracting blame information, and correlating with external ticket systems, we reconstruct the decision‑making process that introduced and retained a knob.

Core steps

Identify the object – file path, function name, or config key associated with the knob.
Run git blame (or git annotate) to see which commit last touched each line.
Extract commit hashes – unique set of commits that have ever modified the knob’s definition or usage.
Fetch commit metadata – author, date, subject, body, and any trailers (e.g., Signed-off-by, Reviewed-by).
Link to issue tracker – parse commit messages for ticket IDs (JIRA, GitHub Issues, etc.) and retrieve the associated discussion.
Analyze temporal pattern – look for periods of inactivity, re‑versions, or “re‑add” commits that suggest deliberate retention.
Summarize – generate a markdown report or JSON blob with timeline, rationale snippets, and stakeholder list.

Tooling

Native Git commands (git log, git blame, git show, git diff).
Shell/Python wrappers to automate extraction (e.g., git log --pretty=format:%H%x09%s%x09%b).
git-filter-repo or git rev-list --objects for large histories.
Integration with APIs: jira, github-cli (gh), gitlab-cli.
Optional: pygit2 or GitPython for programmatic access.

Example CLI Commands and Scripts

Scenario – Same knob service tcp-small-servers. Determine when it was added, who added it, and why it has never been removed.

# 1. Locate the file that defines the CLI handler (from deterministic search)
FILE="src/cli/commands/service.c"

# 2. Show blame for the lines that reference the knob
git blame -L '/\bservice\s+tcp-small-servers\b/,+5' "$FILE"

Sample blame output

a1b2c3d4 (Jane Doe 2018-03-12 14:05:02 +0000 45)    if (strcmp(argv[1], "tcp-small-servers") == 0) {
a1b2c3d4 (Jane Doe 2018-03-12 14:05:02 +0000 46)        /* Legacy knob retained for backward compatibility */
a1b2c3d4 (Jane Doe 2018-03-12 14:05:02 +0000 47)        enable_tcp_small_servers(true);

The commit a1b2c3d4 is the origin. Inspect it:

git show a1b2c3d4 --stat
git show a1b2c3d4

Example commit message

commit a1b2c3d4
Author: Jane Doe <jane@example.com>
Date:   Mon Mar 12 14:05:02 2018 +0000

    Add CLI knob for legacy TCP small services

    This knob enables the deprecated TCP small service daemons (echo, discard,
    chargen, daytime) for customers that still rely on legacy monitoring
    scripts. The feature is marked as deprecated in the CLI guide and will be
    removed in a future major release.

    Signed-off-by: Jane Doe <jane@example.com>
    Reviewed-by: John Smith <john@example.com>
    Related-to: JIRA-1234

Retrieve the JIRA ticket for more context:

gh api /repos/example/netops/issues/JIRA-1234 --jq '.body'

If the ticket states:

“Customer X requires the echo service for legacy NAT keep‑alive. We will provide a knob to enable it temporarily, with a deprecation warning.”

We now have a clear rationale: customer demand + temporary deprecation plan.

Script to automate extraction of all commits touching a knob and generate a timeline

#!/usr/bin/env bash
set -euo pipefail

KNOB_REGEX="\bservice\s+tcp-small-servers\b"
FILE="src/cli/commands/service.c"
OUT="tcp-small-servers-history.md"

echo "# History of $KNOB_REGEX" > "$OUT"
echo "" >> "$OUT"

# Get all commits that ever touched the lines matching the regex
git log -S "$KNOB_REGEX" --pretty=format:"%H%x09%an%x09%ad%x09%s" --date=short "$FILE" |
while IFS=$'\t' read -r hash author date subject; do
    echo "## Commit $hash ($date) by $author" >> "$OUT"
    echo "" >> "$OUT"
    echo "**Subject:** $subject" >> "$OUT"
    echo "" >> "$OUT"
    # Show the diff for this commit limited to the file
    git show "$hash" -- "$FILE" | sed -n '/^\+/,/^-[^-]/p' >> "$OUT"
    echo "" >> "$OUT"
    echo "---" >> "$OUT"
done

echo "History written to $OUT"

Running the script yields a markdown file with each commit, the exact diff, and the commit message—providing a deterministic audit trail.

Troubleshooting Common Issues

Problem	Symptom	Cause	Fix
Renamed/moved files	`git blame` shows “not found” or empty output.	The knob’s definition moved after initial addition.	Use `git log --follow -- <file>` or `git log -S <pattern> --all -- <file>` to track across renames.
Squashed commits	Blame points to a large squash commit lacking detail.	History was rewritten (e.g., via `git merge --squash`).	Examine the original pull request (GitHub/GitLab) for detailed discussion; retrieve via API.
Missing ticket IDs	Commit messages contain no external reference.	Team didn’t enforce ticket linking.	Search the commit body for keywords, email threads, or search the issue tracker via text search.
Large repositories	`git log -S` takes minutes.	Repository size > 5 GB, many binary blobs.	Limit search to relevant paths (`-- src/`) and/or use shallow clones (`--depth`) for recent history, then fallback to full history only if needed.
False positives from string literals	Blame shows changes in unrelated strings that coincidentally match the knob regex.	Regex too permissive.	Anchor regex with word boundaries and/or require specific surrounding syntax (e.g., `if (strcmp\(argv\[1\], \"tcp-small-servers\"\)`).

Scripted history mining excels at provenance and rationale extraction when commit messages are informative. It fails when the rationale lives only in tribal knowledge, wikis, or informal chat logs.

AI‑Assisted Explanation

Methodology and Tools

Treat the discovery problem as a question‑answering task over a multimodal corpus (source code, commit messages, documentation, issue tracker, wikis, chat logs). A typical pipeline consists of:

Corpus ingestion – collect raw artifacts into a searchable store.
Chunking & embedding – split text into overlapping chunks (e.g., 256‑token windows) and compute dense vectors using a sentence‑transformer model (e.g., all-MiniLM-L6-v2).
Vector index – store embeddings in a FAISS, Annoy, or Chroma index for fast similarity search.
Retrieval – given a query (the knob name + “why does it exist?”), retrieve the top‑k most relevant chunks.
Generation – feed the retrieved chunks as context to an LLM (local or API) with a prompt that instructs it to synthesize a concise, cited answer.
Post‑processing – extract citations, verify factual consistency (e.g., ensure the answer does not contradict commit timestamps), and format output.

Tool choices

Embedding models – HuggingFace sentence-transformers/all-MiniLM-L6-v2 (≈ 384 dim, CPU‑friendly) or BAAI/bge-large-en for higher quality.
Vector stores – FAISS (CPU/GPU), Chroma (persistent SQLite backend), or Pinecone (managed).
LLMs – Local inference via llama.cpp (Llama‑2‑7B‑Chat, Mistral‑7B‑Instruct) or hosted APIs (OpenAI GPT‑4‑turbo, Anthropic Claude‑3).
Orchestration – LangChain, LlamaIndex, or a custom Python script using transformers + datasets.

Example Workflow

# 1. Ingest artifacts (code, commits, docs, issues) into a directory
mkdir -p corpus
cp -r src/ docs/ .git/logs/ corpus/
# Export issue tracker data (example using GitHub CLI)
gh issue list --state all --json number,title,body -R example/netops > corpus/issues.json

# 2. Build embeddings and index (Python pseudo‑code)
python - <<'PY'
from sentence_transformers import SentenceTransformer
import faiss, json, os, textwrap

model = SentenceTransformer('all-MiniLM