ci: mechanical quality gates — golden-bot hole, CODEOWNERS, ratchet guard, supply chain#734
Open
joshuakrueger-dfx wants to merge 3 commits into
Open
ci: mechanical quality gates — golden-bot hole, CODEOWNERS, ratchet guard, supply chain#734joshuakrueger-dfx wants to merge 3 commits into
joshuakrueger-dfx wants to merge 3 commits into
Conversation
… ratchet guard, supply-chain monitoring
Every rule below existed as documentation or convention; this PR makes
each one a machine gate (or an automatic process), because a guideline
without enforcement silently rots:
- golden-regenerate: push baselines via TAG_DEPLOY_KEY instead of
GITHUB_TOKEN. Token pushes do not trigger workflows, so the bot-pushed
head SHA had ZERO checks while GitHub showed the PR as mergeable —
required checks never ran against regenerated baselines unless a human
remembered the empty-commit re-arm (documented gotcha in
docs/visual-regression-tests.md). Deploy-key pushes trigger the full
PR gate automatically.
- CODEOWNERS: signing/hardware-wallet paths, pubspec, the floor files
and .github/ itself now route to maintainers. Today one approval from
anyone with write access suffices, even for ceremony code.
- floor-ratchet-guard: lowering .coverage-floor-{lines,functions}
without the coverage:lower-floor label now fails CI. The label was a
documented convention no workflow checked — a quiet floor edit inside
a big diff could de-arm the Coverage Floor Gate permanently.
- pr-policy: conventional PR titles + 1500-line size cap
(size:override label for goldens/codegen churn).
- dependabot (pub + github-actions, 7-day cooldown) and
dependency-review on PR diffs: the repo currently has zero automated
dependency monitoring; the cooldown window would have blocked every
major 2025 supply-chain incident.
- PR template encoding the existing testing/handbook/API-authority
rules as a checklist.
Collaborator
Author
|
Heads-up for admins (added to the checklist): 🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this PR
Every rule in here already exists in this repo — as documentation, convention, or a comment. None of it is enforced by a machine. This PR closes that gap for the five cheapest, highest-impact cases. The guiding principle: a guideline without a gate is documentation, and documentation rots silently. (Same overhaul we just shipped for
DFXswiss/dfx-walletin #186; the ideas for the tier model, the floor files and the behavioral fakes came from THIS repo — these five points are the return delivery.)1. Golden-regenerate bot push now triggers CI (
golden-regenerate.yaml)Why it's needed: pushes made with
GITHUB_TOKENdeliberately do not trigger workflows. After the bot commits regenerated baselines, the PR head SHA has zero status checks — and GitHub renders such a PR as mergeable (required checks only block when they exist and fail).docs/visual-regression-tests.mddocuments the manual workaround (push an empty commit to re-arm checks), which means the safety of every golden-regen PR depends on someone remembering a footnote. A visual regression — or anything else on that SHA — can merge unvalidated.Fix: the bot now pushes via the existing
TAG_DEPLOY_KEY(same ssh-agent pattern asauto-tag.yaml, no new secret needed). Deploy-key pushes trigger workflows, so the full PR gate re-runs on the regenerated head automatically. The manual re-arm note in the docs can be deleted once this lands.2. CODEOWNERS for signing-critical paths
Why it's needed: the rulesets require 1 approval from anyone with write access — including for
lib/packages/wallet/,lib/packages/hardware_wallet/and.github/itself. For an app that drives BitBox signing ceremonies, a single non-specialist approval on ceremony code is too thin, and the CI gates can currently be edited by any PR with any single approval (a gate nobody guards is not a gate).Fix: path-based CODEOWNERS routing those paths to @davidleomay @TaprootFreak @konstantinullrich. Admin action needed to make it blocking: enable Require review from Code Owners in the develop/staging/main rulesets. Until then it still auto-requests the right reviewers.
3. Coverage Floor Ratchet Guard (new required-check candidate)
Why it's needed:
docs/testing.mddefines the ratchet protocol — lowering.coverage-floor-lines/-functionsrequires reviewer sign-off, made visible via thecoverage:lower-floorlabel. No workflow checks this. A quiet one-line floor edit inside a big diff permanently de-arms the Coverage Floor Gate for every future PR, and review history shows big diffs are exactly where attention is scarcest.Fix: new
floor-ratchet-guardjob — floor decrease without the label fails the PR; raising floors passes without ceremony. Admin action: add it to the required checks alongsideCoverage Floor Gate.4. Supply-chain monitoring (dependabot + dependency-review)
Why it's needed: this repo has no automated dependency monitoring at all — no Dependabot, no advisory gate. A known-vulnerable transitive pub package or a compromised GitHub Action would sit unnoticed until someone happens to read an advisory. The 2025 npm supply-chain worms (Shai-Hulud, 25k+ repos) made the pattern concrete; pub is a smaller target but the Actions we run carry repo credentials either way.
Fix: weekly Dependabot for
pub+github-actionstargetingstaging, with a 7-day cooldown — freshly published versions are the riskiest (compromised releases are typically yanked within days; a one-week wait would have blocked every major 2025 incident at zero cost). Plusdependency-reviewon every PR diff, so a newly added dependency with a known high-severity advisory cannot enter the repo in the first place (GitHub's dependency graph covers pub).5. PR template + PR policy job
Why it's needed: the repo's strong conventions (tier model, golden↔handbook pairing, API-authority rules,
@no-integration-testannotations) live in three different docs; nothing puts them in front of the author at PR time. And review quality degrades sharply with diff size — for a wallet app an under-reviewed PR is a security risk, not a style problem.Fix: a PR template encoding the existing rules as a checklist, plus a
pr-policyjob: conventional-commit title (already the de-facto convention) and a 1500-line size cap with asize:overridelabel for legitimate golden/codegen/lockfile churn (label created).What this deliberately does NOT touch
tier3:fulllabel, escapable path filter for the BitBox simulator) — worth doing, but needs a design conversation about runner budget on dfx01; follow-up proposal.strict_required_status_checkson the rulesets (stale-base green merges) — admin-only setting, listed here for visibility.@v4tags are mutable), mechanical, but touches all 13 workflows; cleaner as its own diff. New/touched files in this PR are already SHA-pinned where added.pubspec.lockfully governs app builds, so exact pins inpubspec.yamladd ceremony without closing a real gap.Verification
build, coverage-floor, golden-tests, bitbox-audit, floor-ratchet-guard, pr-policy.Admin checklist after merge
Coverage Floor Ratchet Guard+PR Policy+Dependency Reviewto required checks.TAG_DEPLOY_KEYis authorized for pushes to feature branches (it is a repo deploy key with write — it is).strict_required_status_checksto true.🤖 Generated with Claude Code