feat(ci): detect lockfile version drift — the estate's recurring CI killer - #638
Merged
Conversation
…iller
THE FAULT. Dependabot bumps an action version in a workflow file. Nobody
regenerates .github/workflows/actions.lock. The workflow then requests a
version the lockfile does not record, and GitHub rejects the run at
startup_failure with ZERO jobs -- no log, no annotation, nothing in REST
or GraphQL.
It is SELF-REINFLICTING: any repo with dependabot AND a lockfile AND no
regeneration step re-acquires the fault on every bump. Measured
2026-08-25: 60 of 60 sampled repos carry both. That is why repos appear
to "go bad again" after being fixed -- nobody broke them, the clock did.
PROVEN on hypatia#723. Three drifted entries (docker/setup-buildx-action,
github/codeql-action, taiki-e/install-action) discriminated PERFECTLY:
every workflow using one was dead, every workflow using none was alive.
Regeneration took all four dead workflows from startup_failure to
running, and CodeQL to green.
Ships:
* scripts/check-lockfile-drift.sh -- per-repo checker, TSV output
* .github/workflows/lockfile-drift-detect.yml -- weekly estate sweep,
Tuesday 07:20 UTC (after dependabot's window, so a bump and its drift
are caught the same week)
REPORT-ONLY by design. It opens/updates one tracking issue and never
mutates another repository, per the estate guardrail that unattended
cross-repo mutation is a human decision. It follows the one findings->
issue model in the estate that actually works (farm/drift-detect.yml).
Verified both ways, because a check that cannot fail is not a check:
* against hypatia POST-fix -> reports only a genuine residual SHA/tag
mismatch in tests.yml (confirmed real: workflow pins 34e11487,
lockfile records v7.0.1 -> 3d3c42e5; different commits)
* against hypatia PRE-fix -> finds all three known drifted actions
across the correct six workflows
* against a lockfile-free repo -> exits 0, out of scope (absence of a
lockfile is failure mode 1, not drift)
* dogfooded against standards itself -> clean, 41 workflows
Calibrated honestly: a hit is a genuine inconsistency but NOT proof the
workflow is dead -- tests.yml starts despite its residual mismatch. The
reliable direction is the converse: workflows that were dead had drifted
entries.
Two self-inflicted defects caught in review and fixed here, both worth
noting because the estate has been bitten by each before:
* the issue-body heredoc sat at column 0, terminating the enclosing
`run: |` block scalar -- the exact defect that made
.git-private-farm/inbox-steward-propagate.yml unparseable and blocked
gh actions-lock from running at all
* `[ test ] && cmd` under `set -e` exits the run when the test is false,
and false was the DEFAULT path
Lockfile entry added for the new workflow; without it, it would die at 0s
-- the very fault it exists to detect.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



The fault this catches
Dependabot bumps an action version in a workflow.
actions.lockis not regenerated. The workflow then requests a version the lockfile doesn't record, and GitHub rejects the run atstartup_failurewith zero jobs — no log, no annotation, nothing in REST or GraphQL.It is self-reinflicting. Measured 2026-08-25: 60 of 60 sampled repos carry both a lockfile and a Dependabot config, so the estate re-acquires this on every bump. That is why repos appear to "go bad again" after being fixed — nobody broke them, the clock did.
Proven, not theorised
On hypatia#723, three drifted entries discriminated perfectly:
codeql.ymlrelease.yml,tests.ymlci.ymlrust.yml,governance,mirror,qualityWhat ships
scripts/check-lockfile-drift.sh— per-repo checker, TSV output.github/workflows/lockfile-drift-detect.yml— weekly estate sweep, Tuesday 07:20 UTC (after Dependabot's window, so a bump and its drift land in the same week)Report-only by design. It opens/updates one tracking issue and never mutates another repository, per the estate guardrail that unattended cross-repo mutation is a human decision. It copies the one findings→issue model in the estate that actually works,
farm/drift-detect.yml.Verified both ways
A check that cannot fail is not a check, so:
tests.ymlpins34e11487, lockfile recordsv7.0.1→3d3c42e5. Different commits; confirmed real, not a false positivestandardsCalibrated honestly
A hit is a genuine inconsistency but not proof the workflow is dead —
tests.ymlstarts despite its residual mismatch. The reliable direction is the converse: workflows that were dead had drifted entries. The tool says so in its own header so nobody over-reads its output.Two self-inflicted defects caught in review
Both are faults this estate has already been bitten by, which is why they're called out rather than quietly fixed:
run: |block scalar — the exact defect that made.git-private-farm/inbox-steward-propagate.ymlunparseable and blockedgh actions-lockfrom running at all.[ test ] && cmdunderset -eexits the run when the test is false — and false was the default path.A lockfile entry was added for the new workflow. Without it, it would die at 0s — the very fault it exists to detect.
🤖 Generated with Claude Code