The problem with agentic scanning

A plausible, well-evidenced, wrong finding is worse than no finding.

It burns the credibility of every other finding in the report. LLM agents are excellent at locating candidate issues across a large codebase and unreliable at adjudicating them — they will rate a public court record as a leaked secret, confidently, with a quote and a HIGH. So whoever has to act on the report ends up re-deriving it themselves, and the assessment has cost more than it saved.

Exhibit A — as the agent reported it lens 07 · secrets
High severity
Confidential personal data committed to the repository
test/fixtures/court-records/2019-cv-01184.json:118
"ssn": "078-05-1120",
"dob": "1935-04-02"
Agent-reported · model confidence 0.94 · unverified
Adjudicated — retracted

A public court record, filed in a docket anyone can read, used as a test fixture. Not a secret. Never was. The retraction stays in the report, where the reader can see that someone checked.

Without the gate, this ships as a HIGH — and every real finding beside it inherits the doubt.
01 — NOISE

Volume is not coverage

A scanner emits hundreds of candidates and ranks them by pattern confidence, not by whether the path is actually reachable. Nobody triages all of it, so nobody trusts any of it.

02 — UNPROVABLE

You cannot tell which findings were checked

A report that mixes verified issues with unconfirmed agent output, and labels neither, is not a verdict. It is a worklist wearing a verdict's clothes.

03 — NO HISTORY

"What's wrong" without "since when"

A list of defects tells you nothing about whether the risk arrived in one bad fortnight or accreted over three years — which is the thing that decides what you do on Monday.

Assay is built around that exact failure Use agents for reach. Adjudicate every finding yourself. Label where the confidence came from.

Codebase security & supportability assessment

Agents find. You adjudicate.

Assay gives one engineer the reach of nine parallel security lenses across an unfamiliar codebase — then puts every finding through a human verification gate before it earns a place in the report. Evidence-labelled. Reproducible. And your source never has to leave your infrastructure.

Zero runtime dependencies · Node ≥ 24 · bring your own model · ships an /audit skill · self-verifying Word, PDF, HTML & SARIF output

Any languageNode, Python, .NET and Go built in; anything else via complexity tooling and your own reports
Any modelThe kit's own code calls no LLM and locks to no provider — bring your own
Zero egress--offline runs air-gapped and makes provably zero network calls
Zero dependenciesNo runtime deps; deterministic, reproducible measurement you can re-run

How it works

A method, not just a scanner.

Five stages. A depth flag scales the scope — --light, --thorough or --full — but the gate in the middle is never skipped. That gate is what makes the output worth reading.

00

Recon

Map the stack, domain, data sensitivity and which gates already exist — into one recon map every lens reads first.

01

Fan out

Nine lenses run concurrently, each narrowed to the surfaces recon actually found. Quote file:line, trace to sink, refute first.

the gate
02

Verify

Every high-severity finding re-checked by a human in the primary source. Non-negotiable at every depth.

03

Synthesize

An answer-first executive summary plus a labelled technical appendix — every finding carrying its evidence tier.

04

Publish

A self-verifying .docx, PDF, HTML or SARIF, plus the interactive history dashboard.

🎯
Live-exploitedmeasured
🔍
Source-verifiedconfirmed by a human
⚠️
Agent-reportedpending, and marked so
⚖️
Judgedthe rolled-up verdict
The nine lenses: authz & isolation · injection & uploads · LLM surface · backend correctness · frontend · infrastructure & CI/CD · leaked secrets & confidential data · OWASP crypto, design & integrity · maintainability & SOLID.

Two tools, one pipeline

One tells you what is wrong. The other tells you how long it has been wrong.

They share a single report format and stay in sync — but neither depends on the other to run.

codebase-audit-kit

The assessment engine

A repeatable method plus a zero-dependency CLI that reads a codebase and its full git history, adjudicates every finding, and publishes the deliverable.
  • Recon → nine specialist lenses → a non-negotiable verification gate
  • Every finding labelled by evidence tier; retractions stay in the report
  • Deterministic supportability metrics measured from git and your toolchain
  • Branded .docx with a live TOC, index and page numbers — plus PDF, HTML and SARIF
audit-dashboard

The history dashboard

Joins a finished assessment to the repository's git history and renders one self-contained, themeable HTML page — no server, no build step, nothing to install.
  • Shows when each finding's line was first written, by whom, in which commit
  • Timeline, accumulation plot, file treemap and lens icicle — all cross-filtered
  • Pins to the exact commit the report names; refuses to guess a stale location
  • Re-derives the verification ratio and flags a report that disagrees with itself

assessment.md  →  audit-dash  →  a single HTML file you can open anywhere

The differentiator

A defensible verdict, not raw scanner output.

Scanners and agents are excellent at locating candidates across a large codebase. Assay uses them for exactly that, then makes a human confirm each high-severity finding in the source before it ships.

Every finding carries its evidence tier, and retractions stay in the report — because a report with no retractions either got lucky or never verified.

The count is published too, so the discipline is legible to whoever reads the report rather than something you have to take on faith.

audit-kit attest
# the discipline, made legible Verification attestation critical/high coverage 15 / 15 🎯 live-exploited 3 🔍 source-verified 12 ⚠️ agent-reported 0 retractions kept 2 # counts do not prove correctness. # compare against the source.

Capabilities

Everything needed to measure, verify and ship a verdict.

🎯

Adjudicated findings

Every finding source-verified before it ships and labelled with its evidence tier — retractions and all.

📐

Deterministic metrics

Repo age, velocity, application LOC, bus factor, churn hotspots, cyclomatic complexity, coverage, mutation, vulnerable dependencies and an AI-authorship signal.

🧭

Language agnostic

Node, Python, .NET and Go out of the box; any language via lizard complexity. Anything it can't run, you pass in — LCOV, SARIF, audit JSON.

📄

Word, PDF & HTML

A .docx with cover page, live clickable TOC, index and page numbers — no pandoc, no LaTeX. Same layout to PDF, or standalone print-ready HTML anywhere.

🎨

Corporate branding

Brand at the shell: pour the report into your own .docx template so its theme, cover, header and footer genuinely drive the document.

🛡️

Scanner orchestration

secscan normalises Semgrep, gitleaks, Trivy and OWASP ZAP into one severity-sorted worklist — every row pending until a human adjudicates it.

📊

SARIF & readiness

Findings as SARIF 2.1.0 for GitHub code scanning or Defender, evidence tiers preserved. A weighted readiness score rolls the verdict into one number.

⏱️

Continuous monitoring

A scheduled CI workflow re-runs the deterministic checks weekly and gates on only two objective signals — an end-of-life runtime and critical/high vulnerable deps.

Self-verifying build

The build checks every heading against the generated TOC and throws if one is missing. The kit holds itself to the standard its reports demand.

The finding-history dashboard

The report says what is wrong. This says how long it has been wrong.

Every finding lands on the commit that first wrote its line — so you can see whether your risk arrived in one bad fortnight or accreted over three years. One self-contained HTML file that computes every chart in the browser. No server, no build, no CDN.

📌

The commit is not optional

Line numbers mean something only at the commit the report names. The dashboard pins to it and stops if the checkout doesn't contain it — rather than blaming HEAD and inventing a plausible, wrong answer.

⚠️

Stale locations reported, not guessed

A location git can't resolve gets git's own reason shown in the row and counted in a banner. That's a finding about the report, and more useful than a blank.

🔎

Filters scope every view

Filter by severity, evidence, lens or churn hotspot and the KPIs, both plots, the table, treemap, icicle and churn chart all re-render together — so the numbers always agree.

🧮

Derives, then compares

It recomputes the verification ratio from the findings table and raises a banner when the report's prose disagrees with its own numbers.

👁️

Colourblind-safe by design

Severity rides position, not colour, and every plotted value is repeated in the table. A report whose whole purpose is not to mislead cannot ship a chart that does.

📈

Churn, overlaid

Commits touching each file up to the assessed commit — high churn × high complexity made visible as the next likely defect site.

No data egress

Bring your own model. Your source never leaves the building.

The kit's own code calls no LLM. The mechanical half — scan, secscan, readiness, attest, build, the dashboard — is pure deterministic Node. The intelligent half is done by an agent driving a model you choose, and it is deliberately not locked to one provider.

TierYou run it withWhat leaves your network
TurnkeyThe /audit skillPrompts + code excerpts to your model provider
BYO keyA thin runner on an agent SDKPrompts + code excerpts to your provider
BYO any modelAny agent runtime, or by handNothing — on a self-hosted model
The deterministic half's only network calls, ever, are dependency-metadata lookups to public registries — package names and versions, never your code. And you can prove the boundary rather than trust it.
air-gapped
$ audit-kit scan --offline # runs and makes ZERO network calls. # run it air-gapped and check. git history · application LOC complexity · bundled runtime-EOL registry / advisory tools skipped executes no target code # the one property a per-seat SaaS # competitor cannot match.

Deliverables

.docx Editable Word, live TOC & index .pdf Immutable, bookmarked .html Print-ready, self-contained .sarif GitHub code scanning / Defender dashboard One themeable HTML file CI Weekly monitoring summary

Contact

Tell us about the codebase.

Whether it's due diligence on an acquisition, an inherited system nobody owns any more, or a security review you need to be able to defend line by line — start here.

anthony.terra@assaycenter.com Prefer email? That address reaches the same inbox as the form.
We reply to everything, usually within two business days.
No newsletter, no sharing. Just a reply.