fix(baseline): allow Scorecard probe names in type - #589
Conversation
No Scorecard finding could be baselined by ANY repository in the estate.
The schema required `type` to match ^([a-z][a-z0-9_]*|[A-Z]{2,3}[0-9]{3})$ —
snake_case rule names, or SD007-style structural-drift codes. Hypatia emits
Scorecard probe names verbatim, in CamelCase (DependencyPinning,
BranchProtection, ...), which matches neither.
The failure mode is worse than a rejected entry: apply-baseline.sh rejects the
WHOLE FILE with exit 2 and the message "baseline violates
hypatia-baseline.schema.json". A maintainer reads that as "my baseline is
malformed" and starts hunting for their own typo, when the truth is that the
finding cannot be expressed at all. Found 2026-08-06 in
metadatastician/stapeln, where the only way forward was to drop the finding.
Adds a third alternative for CamelCase probe names. Verified: a Scorecard entry
is now accepted, a `type` with spaces is still rejected with exit 2, and the
existing apply-baseline test suite still passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Jonathan D.A. Jewell <6759885+hyperpolymath@users.noreply.github.com>
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
| # Found 2026-08-06 in metadatastician/stapeln. | ||
| (if ($e.type|type) == "string" | ||
| and (($e.type|test("^([a-z][a-z0-9_]*|[A-Z]{2,3}[0-9]{3})$"))|not) | ||
| and (($e.type|test("^([a-z][a-z0-9_]*|[A-Z]{2,3}[0-9]{3}|[A-Z][A-Za-z0-9]+)$"))|not) |
There was a problem hiding this comment.
⚠️ Bug: type pattern fix not applied to authoritative schema
The inline jq validator in apply-baseline.sh:109 now accepts CamelCase Scorecard probe names, but the canonical schema .machine_readable/hypatia-baseline.schema.json:33 (titled the "Authoritative schema" for baselines) still has the old pattern ^([a-z][a-z0-9_]*|[A-Z]{2,3}[0-9]{3})$. The script's own rejection message (line 136) even names this schema. Any other consumer that validates a baseline against the real JSON Schema (editors, CI, other tools) will still reject Scorecard findings, so the fix is incomplete and the two validators now disagree. Update the schema pattern to ^([a-z][a-z0-9_]*|[A-Z]{2,3}[0-9]{3}|[A-Z][A-Za-z0-9]+)$ and extend the type description (line 31) to mention CamelCase Scorecard probe names.
Mirror the new alternative in the authoritative schema and update the neighbouring description to document CamelCase Scorecard probe names.:
"pattern": "^([a-z][a-z0-9_]*|[A-Z]{2,3}[0-9]{3}|[A-Z][A-Za-z0-9]+)$"
Was this helpful? React with 👍 / 👎
|
Note Automatic reviews are paused because your trial's included automatic processing has been used for this period. Upgrade now, or comment "Gitar review" to run a review anytime. Code Review
|
| Compact |
|
Important
Your trial ends in 4 days — upgrade now to keep code review, CI analysis, auto-apply, custom automations, and more.
Was this helpful? React with 👍 / 👎 | Gitar
There was a problem hiding this comment.
Configure merge blocking · Maintainers can dismiss this review.
… too Found by gitar-bot review on this pull request. The previous commit widened the `type` pattern in apply-baseline.sh's jq validator, but not in .machine_readable/hypatia-baseline.schema.json. That script's own header says it MIRRORS the schema file — so the two would have disagreed, with the schema still rejecting Scorecard probe names while the validator accepted them. A divergence here is worse than either state alone: the schema is what a human reads to learn what is allowed, and any other tool validating against it would reject a baseline the estate's own gate had just passed. Verified the two now carry byte-identical patterns, and the existing apply-baseline test suite still passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Jonathan D.A. Jewell <6759885+hyperpolymath@users.noreply.github.com>
…esentable entry Three findings from gitar-bot review on this pull request. 1. install_just_verified HARDCODED x86_64 LINUX. detect_platform() already computes $OS and $ARCH, and this script supports macOS and FreeBSD, so the previous version would have downloaded an x86-64 Linux binary onto Apple Silicon and every ARM machine. The target is now resolved from OS/ARCH with a pinned digest per target (linux x86_64/aarch64, macOS x86_64/arm64, computed from the artifacts on 2026-08-06). ⚠ An unrecognised platform RETURNS FAILURE rather than guessing, and the caller falls back to the package manager. Downloading a plausible-looking artefact for the wrong target is worse than not downloading one: it fails later and further away from the cause. 2. BASELINE NOTES MISLABELLED NON-RESCRIPT FINDINGS. Eight entries said "ReScript source" when six were TypeScript test files and two were pattern entries. TypeScript is banned separately — the governance reusable carries its own TypeScript check, distinct from the ReScript rule — so the note was naming the wrong policy. An exemption whose stated reason is wrong is worse than one with no reason: it looks reviewed. 3. THE DependencyPinning ENTRY WAS STILL PRESENT. The previous commit's message said it had been dropped as unrepresentable; the file still contained it, so the message and the file disagreed. Now actually removed, and the file validates. It becomes representable once hyperpolymath/standards#589 lands, which widens the schema's `type` pattern to accept Scorecard probe names. Verified: apply-baseline.sh accepts the file, every entry carries a note, and no note misnames its language. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Jonathan D.A. Jewell <6759885+hyperpolymath@users.noreply.github.com>
|
1 participant
No Scorecard finding could be baselined by any repository in the estate.
The schema required
typeto match^([a-z][a-z0-9_]*|[A-Z]{2,3}[0-9]{3})$— snake_case rule names, orSD007-style structural-drift codes. Hypatia emits Scorecard probe names verbatim, in CamelCase (DependencyPinning,BranchProtection, …), which matches neither.The failure mode is worse than a rejected entry.
apply-baseline.shrejects the whole file with exit 2 and the message "baseline violates hypatia-baseline.schema.json". A maintainer reads that as "my baseline is malformed" and goes hunting for their own typo — when the truth is the finding cannot be expressed at all.Found 2026-08-06 in
metadatastician/stapeln, where the only way forward was to drop the finding.Verified: a Scorecard entry is now accepted (exit 0), a
typecontaining spaces is still rejected (exit 2), and the existingapply-baseline-test.shsuite still passes.🤖 Generated with Claude Code