Repository navigation
feat(sarif): add a pentest profile that appends covered exposures and boundary claims - #51
Merged
Merged
Conversation
… boundary claims `guardlink sarif --profile <github|pentest>`, and `profile` on the MCP `guardlink_sarif` tool. `github` is the default and is the export as it was, byte for byte: tests/fixtures/sarif-baseline/*.github.sarif are full exports cut before this change, and the test compares the default and `--profile github` against them whole. `pentest` appends, after every github result so no result index moves: - one `guardlink/mitigated-exposure` result per covered @exposes, built by the same code as an uncovered one (same message, properties, identity and declared context) at level note, with `suppressions[]` naming each covering @mitigates or @accepts and its location; - one `guardlink/boundary-claim` result per @boundary (kind review, level none), keyed by the boundary's claim key, with its sides, the outer side only when exactly one side is undeclared (`basis: "unknown"` and no side otherwise), the @flows that cross it and the routes into its inner side. Every exposure, confirmed and boundary-claim result carries the hypothesis ledger's state as `properties["guardlink/hypothesis"]`; a corrupt ledger stamps nothing and the envelope says so. `--baseline <file>` sets `baselineState` against an earlier export, matching by claim key and then by threat id, message and file; an unmatched baseline result is counted in `properties.baseline.absent` rather than exported, since every reader of this export treats a result as something to test. SPEC §6.8 describes the profile. tests/fixtures/sarif-pentest/ holds the pentest export of sarif-shop (with a new hypothesis ledger) and expense-api for SARIF readers to test against; the suite regenerates and compares them and validates every pentest export against the SARIF 2.1.0 schema. The coverage index gains `acceptancesFor`, the acceptance counterpart of `mitigationsFor`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
guardlink sarif --profile <github|pentest>(andprofileon the MCPguardlink_sariftool).github(default) is the export as it was, byte for byte.pentestkeeps the github export unchanged and appends results after it. No existing result index moves.guardlink/mitigated-exposureresult per covered@exposes. It is built by the same code as an uncovered one, so it has the same message, properties, identity and declared context. It is atlevel: "note", and itssuppressions[]name each covering@mitigates/@acceptsand where it is.guardlink/boundary-claimresult per@boundary, withkind: "review"andlevel: "none". It carries:properties.claimKey;boundary {id, a, b, asset_a, asset_b, basis, outer?, inner?};crossings, the@flowshops across it;inner_routes, the route-channel@flowsinto its inner side.properties["guardlink/hypothesis"]: the claim's state in.guardlink/hypotheses.json, with its evidence and whether it expired.--baseline <file>setsbaselineState(new/unchanged/updated) against an earlier export.run.properties.baseline.absentand is not exported. Readers treat every result as something to test, so anabsentresult would be probed.SPEC §6.8 describes the profile. §6.1 and §6.7 now point to it, and the "no suppressed result" invariant is scoped to the github profile.
Why
The github export leaves out exactly what a test run needs:
Adding those as results to the default export would move every result index and open a GitHub alert per annotation. So they live in a separate profile, appended after the existing results.
Boundary direction.
@boundaryis undirected. The outer side is stated only when exactly one side is an endpoint no@assetdeclares; this is theguardlink pathsconvention, andrun.graphs[0]already uses it. Otherwisebasisis"unknown"and no side is written.A mitigated exposure stays a failure. It is a failure suppressed in source (
kindabsent) rather than apass, because a declared control is a claim until something tests it.How it was verified
tests/sarif-pentest.test.ts(35 tests):--profile githubexports of both fixtures are compared whole againsttests/fixtures/sarif-baseline/*.github.sarif. Those files were cut frommainbefore this change. Via the CLI,--profile githubstdout equals the no-flag stdout.failresult haslevel: "none", as SARIF requires.baselineStateare covered. That includes a removed mitigation (updatedby claim key, although the result moved), a reworded claim (newplusabsent: 1), a baseline with no claim keys, a corrupt ledger, an@acceptssuppression and--min-severity.--baselinewithout the pentest profile, and refuses an unknown profile. The MCP pentest export equals the CLI's.tests/actor-entitlement.test.ts: the pentest profile is byte-identical with and without@entitles/@actor, including its suppressions and boundary claims.npm test: 115 files, 2079 passed, 1 skipped.eslintis clean on the changed files.guardlink validate . --artifactspasses.guardlink diff HEAD~2shows three added@flowsand no change in risk.Fixtures for SARIF readers
tests/fixtures/sarif-pentest/{sarif-shop,expense-api}.sarifare the pentest exports of the two fixtures. Between them they cover:basis: "unknown";untested,confirmed,refuted,supportedandunverified, from a newtests/fixtures/sarif-shop/.guardlink/hypotheses.json.The test regenerates and compares both files. Their README says what a reader can rely on and how to regenerate them.
Review notes
CoverageIndexgainsacceptancesFor, the acceptance counterpart ofmitigationsFor;isAcceptednow uses it.sarif-context.tsexposes each boundary's sides, crossings and claim key (boundaries()). Boundary-claim results andrun.graphs[0]therefore come from one computation.inner_routes. The exporter reads only the declared model.