Skip to content

feat(arsenal): structured parsers for source / supply-chain scanners (semgrep, trivy, grype, gitleaks)#43

Open
Pazificateur69 wants to merge 2 commits into
elder-plinius:mainfrom
Pazificateur69:feat/supply-chain-parsers
Open

feat(arsenal): structured parsers for source / supply-chain scanners (semgrep, trivy, grype, gitleaks)#43
Pazificateur69 wants to merge 2 commits into
elder-plinius:mainfrom
Pazificateur69:feat/supply-chain-parsers

Conversation

@Pazificateur69

Copy link
Copy Markdown

Stacked on #42 (the structured-parser layer) and pairs with #40 (the arg templates that make these tools emit JSON). Until those land, this PR's diff includes #42's commit; the net-new commit here is c20c24c — please review after #42. Parsers degrade to [] when a tool isn't emitting JSON yet, so nothing misbehaves in the meantime.

What

Extends the structured-parser layer (#42) to the source / supply-chain scanners, so their output becomes gate-verifiable Findings instead of raw text (they were parserStatus: 'planned'):

tool command (from #40) mapping
semgrep scan --config auto --json . one finding/rule match · ERROR/WARNING/INFO → severity · CWE token from metadata ("CWE-78: …"CWE-78) · refs → remediation
trivy fs --format json . one finding/package vuln · CVE + CweIDs · upgrade <pkg> to <fixed> remediation
grype dir:. -o json one finding/match · CVE + package coordinates
gitleaks detect --report-format json --redact one high finding/secret · see redaction note

Honesty / safety

  • Same contract as feat(arsenal): parse JSON scanner output into structured, gate-verifiable findings #42: parsers are pure, dependency-free, and never throw — empty / garbled / non-JSON input yields [], never a fabricated finding. Raw stdout is always kept as the evidence of record.
  • Secret redaction (defence in depth): the gitleaks adapter already runs with --redact, but the parser also routes finding details through redactString() (src/redact.ts), so a raw secret can never reach a Finding or the evidence vault. A test pins this: an AWS key present in the raw report is [redacted] in the finding.
  • Broadens the severity map to fold scanner vocabularies (ERROR/WARNING, CRITICAL…NEGLIGIBLE, moderate) onto the five canonical levels.

Tests

Extends parsers.test.ts + fixtures: per-scanner field mapping (CVE/CWE/severity/remediation), the gitleaks redaction guarantee, and the end-to-end assertion that every new scanner's parsed finding passes gateLiveFinding with provenance='tool'.

Scope

SARIF-style output and secret scanners beyond gitleaks (trufflehog etc.) remain a clean follow-up.

Review standard (all green locally)

typecheck · test · doctor · verify-claims · test:no-fitting · test:no-self-fitting · test:gate · prompt:audit · smoke

…able findings

The generic adapter factory returned raw stdout only (`adapter-tools.ts` success
path), so 35 catalogued scanners with `parserStatus: 'planned'` never populated the
structured `ToolResult.findings` channel — their output was dropped on the floor and
the agent could neither verify nor chain on it. The whole downstream pipe already
exists and is honest-by-construction: the agent loop stamps `provenance:'tool'` + the
raw output, the operator materialises each finding with output-evidence, and the live
gate (`evidence/gate.ts`) marks it verified.

Add `src/arsenal/parsers.ts` — `parseToolOutput(toolId, rawStdout) → ToolFinding[]`
for the five scanners that already emit machine-readable JSON via their existing arg
templates (zero template changes): nuclei (-jsonl), httpx (-json), dalfox (--format
json), ffuf (-of json), katana (-jsonl). Parsers are pure, dependency-free, and NEVER
throw: empty/garbled/unknown input yields `[]`, never a fabricated finding (same
discipline as stub-honesty / no-phantom-tools). The factory attaches parsed findings
while ALWAYS keeping the raw stdout as `output` — the evidence of record the gate checks.

- nuclei/dalfox emit CWE, which the tool channel dropped: add optional `cwe?: string[]`
  to `ToolFinding` and thread it through the operator's finding materialisation.
- flip `parserStatus: 'planned' → 'structured'` for httpx, dalfox, katana (nuclei/ffuf
  were already structured).

Adds `src/__tests__/parsers.test.ts` + fixtures: per-scanner field mapping, the
honesty contract (empty/garbled/adversarial → [], no throw), a factory-wiring check
that a minted adapter returns findings while preserving raw stdout, and an end-to-end
assertion that every parsed finding PASSES `gateLiveFinding` with provenance='tool'.

The minted-adapter path is gated behind T3MP3ST_FULL_ARSENAL, so the benchmarked
default arsenal is untouched. Supply-chain JSON tools (semgrep/gitleaks/trivy/grype)
and secret scanners (need redaction) are a clean follow-up.
Follow-up to the JSON-scanner parser layer: wire semgrep, gitleaks, trivy and
grype into parseToolOutput so their findings become structured, gate-verifiable
evidence instead of raw text (they were parserStatus:'planned').

- semgrep  (scan --json)          → one finding per rule match; ERROR/WARNING/INFO
                                     mapped to severity; CWE token pulled from
                                     metadata (e.g. "CWE-78: …" → CWE-78); refs → remediation.
- trivy    (fs --format json)     → one finding per package vuln; CVE + CweIDs +
                                     "upgrade <pkg> to <fixed>" remediation.
- grype    (dir:. -o json)        → one finding per match; CVE + package coordinates.
- gitleaks (detect --report json --redact) → one high finding per secret. Defence in
                                     depth: the details are run through redactString()
                                     so a raw secret can NEVER reach a finding / the vault,
                                     even though the adapter already passes --redact.

Broadens the severity map to cover scanner vocabularies (ERROR/WARNING, CRITICAL…
NEGLIGIBLE, moderate) onto the five canonical levels. Flips parserStatus to
'structured' for the four tools. Extends parsers.test.ts + fixtures: field mapping,
the gitleaks redaction guarantee, and the end-to-end gate pass for every new scanner.

Stacked on the parser layer (elder-plinius#42); pairs with the scanner-invocation fix (elder-plinius#40) that
gives these tools the JSON-emitting arg templates. Until elder-plinius#40 lands the parsers simply
degrade to [] (no JSON to parse) — no misbehaviour. Secret scanners beyond gitleaks
and SARIF-style tools remain a follow-up.
@jmagly

jmagly commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Reviewed — sound, and correctly stacked on #42 (net-new commit c20c24c). The added semgrep/trivy/grype/gitleaks parsers map the real schemas, severity aliases check out (semgrep ERROR→high / WARNING→medium; trivy/grype HIGH→high, asserted in tests), and the gitleaks redactString scrub genuinely works — the fixture's AKIA… key is replaced with [redacted] and the test asserts the raw key is absent. One real-world note you already call out: some scanners (ffuf, gitleaks, trivy) don't emit JSON to stdout without an output-path flag, so those parsers receive [] until the arg-template work lands — degrades safely to empty, no crash. Merge after #42.

@jmagly

jmagly commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Please rebase this branch onto current main and remove the unrelated deleted files.

The parser work should be reviewed on its own, but the branch currently carries stale-base deletions of benchmark corpora, benchmark scripts, provenance docs, and tests. That makes the merge unsafe.

Suggested agent repair path:

  1. Recreate/rebase from current main.
  2. Keep only source/supply-chain parser code, fixtures, and parser tests.
  3. Restore benchmark corpora/scripts, provenance docs, and unrelated tests.
  4. Add focused malformed-output tests for each structured parser path.
  5. Verify with git diff --name-status upstream/main...HEAD that the only deleted files are intentionally part of the parser change, ideally none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants