Skip to content

feat(sarif): add a pentest profile that appends covered exposures and boundary claims - #51

Merged
Animesh-Sri-bugb merged 2 commits into
mainfrom
fm/fx-guardlink-pentest-profile
Oct 1, 2026
Merged

Animesh-Sri-bugb merged 2 commits into
mainfrom
fm/fx-guardlink-pentest-profile

Conversation

@Animesh-Sri-bugb

Copy link
Copy Markdown
Contributor

What

guardlink sarif --profile <github|pentest> (and profile on the MCP guardlink_sarif tool).

  • github (default) is the export as it was, byte for byte.
  • pentest keeps the github export unchanged and appends results after it. No existing result index moves.
    • one guardlink/mitigated-exposure result per covered @exposes. It is built by the same code as an uncovered one, so it has the same message, properties, identity and declared context. It is at level: "note", and its suppressions[] name each covering @mitigates / @accepts and where it is.
    • one guardlink/boundary-claim result per @boundary, with kind: "review" and level: "none". It carries:
      • the boundary's claim key, in the fingerprint and in properties.claimKey;
      • boundary {id, a, b, asset_a, asset_b, basis, outer?, inner?};
      • crossings, the @flows hops across it;
      • inner_routes, the route-channel @flows into its inner side.
    • every exposure, confirmed and boundary-claim result gets properties["guardlink/hypothesis"]: the claim's state in .guardlink/hypotheses.json, with its evidence and whether it expired.
    • --baseline <file> sets baselineState (new / unchanged / updated) against an earlier export.
      • Results match by claim key first, then by threat id, message and file.
      • A baseline result that matches nothing is counted in run.properties.baseline.absent and is not exported. Readers treat every result as something to test, so an absent result would be probed.

SPEC §6.8 describes the profile. §6.1 and §6.7 now point to it, and the "no suppressed result" invariant is scoped to the github profile.

Why

The github export leaves out exactly what a test run needs:

  • exposures whose control is only declared, so nothing tests the control;
  • the declared trust boundaries a probe should try to cross;
  • what has already been tested.

Adding those as results to the default export would move every result index and open a GitHub alert per annotation. So they live in a separate profile, appended after the existing results.

Boundary direction. @boundary is undirected. The outer side is stated only when exactly one side is an endpoint no @asset declares; this is the guardlink paths convention, and run.graphs[0] already uses it. Otherwise basis is "unknown" and no side is written.

A mitigated exposure stays a failure. It is a failure suppressed in source (kind absent) rather than a pass, because a declared control is a claim until something tests it.

How it was verified

  • tests/sarif-pentest.test.ts (35 tests):
    • github is unchanged. The default and --profile github exports of both fixtures are compared whole against tests/fixtures/sarif-baseline/*.github.sarif. Those files were cut from main before this change. Via the CLI, --profile github stdout equals the no-flag stdout.
    • pentest only appends. The first N results and rules equal the github export's. The envelope's existing members are unchanged.
    • Schema. Every pentest export validates against the OASIS SARIF 2.1.0 schema: both fixtures, this repository, and one with a baseline. Every non-fail result has level: "none", as SARIF requires.
    • mitigated-exposure, boundary-claim, ledger states and baselineState are covered. That includes a removed mitigation (updated by claim key, although the result moved), a reworded claim (new plus absent: 1), a baseline with no claim keys, a corrupt ledger, an @accepts suppression and --min-severity.
    • Front ends. The CLI refuses --baseline without the pentest profile, and refuses an unknown profile. The MCP pentest export equals the CLI's.
  • tests/actor-entitlement.test.ts: the pentest profile is byte-identical with and without @entitles / @actor, including its suppressions and boundary claims.
  • npm test: 115 files, 2079 passed, 1 skipped. eslint is clean on the changed files. guardlink validate . --artifacts passes. guardlink diff HEAD~2 shows three added @flows and no change in risk.

Fixtures for SARIF readers

tests/fixtures/sarif-pentest/{sarif-shop,expense-api}.sarif are the pentest exports of the two fixtures. Between them they cover:

  • all seven rules;
  • a boundary claim with an inferred side and one with basis: "unknown";
  • the ledger states untested, confirmed, refuted, supported and unverified, from a new tests/fixtures/sarif-shop/.guardlink/hypotheses.json.

The test regenerates and compares both files. Their README says what a reader can rely on and how to regenerate them.

Review notes

  • CoverageIndex gains acceptancesFor, the acceptance counterpart of mitigationsFor; isAccepted now uses it.
  • sarif-context.ts exposes each boundary's sides, crossings and claim key (boundaries()). Boundary-claim results and run.graphs[0] therefore come from one computation.
  • The code graph's measured access level is not included in inner_routes. The exporter reads only the declared model.
  • The second commit only regenerates the synced agent-instruction and threat-model artifacts.

… boundary claims

`guardlink sarif --profile <github|pentest>`, and `profile` on the MCP
`guardlink_sarif` tool. `github` is the default and is the export as it
was, byte for byte: tests/fixtures/sarif-baseline/*.github.sarif are full
exports cut before this change, and the test compares the default and
`--profile github` against them whole.

`pentest` appends, after every github result so no result index moves:

- one `guardlink/mitigated-exposure` result per covered @exposes, built by
  the same code as an uncovered one (same message, properties, identity
  and declared context) at level note, with `suppressions[]` naming each
  covering @mitigates or @accepts and its location;
- one `guardlink/boundary-claim` result per @boundary (kind review, level
  none), keyed by the boundary's claim key, with its sides, the outer side
  only when exactly one side is undeclared (`basis: "unknown"` and no side
  otherwise), the @flows that cross it and the routes into its inner side.

Every exposure, confirmed and boundary-claim result carries the hypothesis
ledger's state as `properties["guardlink/hypothesis"]`; a corrupt ledger
stamps nothing and the envelope says so. `--baseline <file>` sets
`baselineState` against an earlier export, matching by claim key and then
by threat id, message and file; an unmatched baseline result is counted in
`properties.baseline.absent` rather than exported, since every reader of
this export treats a result as something to test.

SPEC §6.8 describes the profile. tests/fixtures/sarif-pentest/ holds the
pentest export of sarif-shop (with a new hypothesis ledger) and expense-api
for SARIF readers to test against; the suite regenerates and compares them
and validates every pentest export against the SARIF 2.1.0 schema. The
coverage index gains `acceptancesFor`, the acceptance counterpart of
`mitigationsFor`.
@Animesh-Sri-bugb
Animesh-Sri-bugb merged commit 9e59e35 into main Oct 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant