Skip to content

Commit 107591f

Browse files
hyperpolymathclaude
andcommitted
feat(ci): add fail_on_severity so the scan gate can actually fail
`panic-attack assail` has no non-zero exit path of its own — no process::exit, no ExitCode, no --fail-on anywhere in its 3,132-line src/main.rs. It always exits 0. Consumers of this reusable therefore ran a scanner whose only value was the cross-repo report; when the dispatch is skipped (no VERISIMDB_PAT) the job scans, discards the result, and reports success regardless of findings. Adds a `fail_on_severity` input (none | low | medium | high | critical) and a gate step that fails the job when an unsuppressed finding sits at or above the threshold. Design notes: * **Default is 'none'**, so this is a no-op for every existing caller until it opts in. verisimdb, echidna and ambientops all stay green on merge. * **Counts only unsuppressed findings.** `suppressed` is set by the repo's own classification registry and by test-context detection, and is *omitted* rather than set false — hence `select(.suppressed != true)` rather than `== false`. * **An unrecognised threshold aborts.** A typo like 'hgih' must not degrade to 'threshold 0' and pass everything; that silent-pass behaviour is exactly what this change exists to remove. * The failure message points at the classification registry, not at this workflow — suppression belongs next to the evidence and in review, not buried in CI config. Verified against a real scan (verisimdb origin/main, 113 findings): critical -> 0 offending, high -> 1, medium -> 39; the one High was a genuine finding live only because a classification key had gone stale after a rename. YAML parses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent d032185 commit 107591f

1 file changed

Lines changed: 60 additions & 0 deletions

File tree

.github/workflows/scan-and-report.yml

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,18 @@ on:
99
description: 'Path to scan (default: .)'
1010
default: '.'
1111
type: string
12+
fail_on_severity:
13+
description: >-
14+
Fail the job when a non-suppressed finding at or above this severity
15+
is present. One of: none | low | medium | high | critical.
16+
17+
Defaults to 'none', which preserves the historical behaviour exactly:
18+
`panic-attack assail` has no non-zero exit path of its own (no
19+
process::exit, no ExitCode, no --fail-on anywhere in src/main.rs), so
20+
without this input a Security Scan reports success regardless of what
21+
it finds. Existing callers are unaffected until they opt in.
22+
default: 'none'
23+
type: string
1224
secrets:
1325
VERISIMDB_PAT:
1426
description: 'PAT with repo scope for cross-repo dispatch to verisimdb-data'
@@ -41,6 +53,54 @@ jobs:
4153
panic-attack assail ${{ inputs.repo_path }} --output scan-result.json
4254
echo "scan_complete=true" >> $GITHUB_OUTPUT
4355
56+
- name: Enforce severity gate
57+
if: steps.scan.outputs.scan_complete == 'true' && inputs.fail_on_severity != 'none'
58+
env:
59+
FAIL_ON: ${{ inputs.fail_on_severity }}
60+
run: |
61+
set -euo pipefail
62+
63+
# Normalise and validate up front. An unrecognised value must abort,
64+
# never silently degrade to "threshold 0" -- a typo'd threshold that
65+
# quietly passes everything is precisely the failure this gate exists
66+
# to remove.
67+
case "$(printf '%s' "$FAIL_ON" | tr '[:upper:]' '[:lower:]')" in
68+
low) THRESHOLD=1 ;;
69+
medium) THRESHOLD=2 ;;
70+
high) THRESHOLD=3 ;;
71+
critical) THRESHOLD=4 ;;
72+
*)
73+
echo "::error::fail_on_severity must be one of none|low|medium|high|critical (got '${FAIL_ON}')"
74+
exit 1
75+
;;
76+
esac
77+
78+
# Count only findings the scanner did NOT suppress. `suppressed` is
79+
# set by the user-classification registry (audits/*.a2ml) and by
80+
# test-context detection, and is omitted rather than set false, so
81+
# test for `!= true`.
82+
OFFENDING=$(jq --argjson t "$THRESHOLD" '
83+
def rank: {"Low":1,"Medium":2,"High":3,"Critical":4}[.] // 0;
84+
[ .weak_points[]?
85+
| select(.suppressed != true)
86+
| select((.severity | rank) >= $t) ]
87+
' scan-result.json)
88+
89+
COUNT=$(printf '%s' "$OFFENDING" | jq 'length')
90+
91+
if [ "$COUNT" -gt 0 ]; then
92+
echo "::error::${COUNT} unsuppressed finding(s) at or above '${FAIL_ON}'"
93+
printf '%s' "$OFFENDING" \
94+
| jq -r '.[] | " [\(.severity)] \(.category) \(.file // .location // "?")"'
95+
echo
96+
echo "To accept one of these, add a (file, category) entry with an audit"
97+
echo "rationale to the repo's assail classification registry -- not to this"
98+
echo "workflow. Suppression belongs next to the evidence, in review."
99+
exit 1
100+
fi
101+
102+
echo "OK: no unsuppressed findings at or above '${FAIL_ON}'"
103+
44104
- name: Send to verisimdb-data
45105
if: steps.scan.outputs.scan_complete == 'true'
46106
env:

0 commit comments

Comments
 (0)