Skip to content

feat(ci): detect lockfile version drift — the estate's recurring CI killer - #638

Merged
hyperpolymath merged 1 commit into
mainfrom
feat/lockfile-drift-detector
Aug 25, 2026
Merged

feat(ci): detect lockfile version drift — the estate's recurring CI killer#638
hyperpolymath merged 1 commit into
mainfrom
feat/lockfile-drift-detector

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

The fault this catches

Dependabot bumps an action version in a workflow. actions.lock is not regenerated. The workflow then requests a version the lockfile doesn't record, and GitHub rejects the run at startup_failure with zero jobs — no log, no annotation, nothing in REST or GraphQL.

It is self-reinflicting. Measured 2026-08-25: 60 of 60 sampled repos carry both a lockfile and a Dependabot config, so the estate re-acquires this on every bump. That is why repos appear to "go bad again" after being fixed — nobody broke them, the clock did.

Proven, not theorised

On hypatia#723, three drifted entries discriminated perfectly:

Workflow uses a drifted action before after
codeql.yml codeql-action v4.37.7→8 ❌ startup_failure success
release.yml, tests.yml buildx v4.2.0→4.3.0 ❌ startup_failure ▶ runs
ci.yml taiki-e v2.86.1→2.86.5 ❌ startup_failure ▶ runs
rust.yml, governance, mirror, quality none ✅ alive ✅ alive

What ships

  • scripts/check-lockfile-drift.sh — per-repo checker, TSV output
  • .github/workflows/lockfile-drift-detect.yml — weekly estate sweep, Tuesday 07:20 UTC (after Dependabot's window, so a bump and its drift land in the same week)

Report-only by design. It opens/updates one tracking issue and never mutates another repository, per the estate guardrail that unattended cross-repo mutation is a human decision. It copies the one findings→issue model in the estate that actually works, farm/drift-detect.yml.

Verified both ways

A check that cannot fail is not a check, so:

Test Result
hypatia post-fix reports only a genuine residual mismatch — tests.yml pins 34e11487, lockfile records v7.0.13d3c42e5. Different commits; confirmed real, not a false positive
hypatia pre-fix finds all three known drifted actions across the correct six workflows
lockfile-free repo exits 0 — out of scope (that's failure mode 1, not drift)
dogfood on standards clean, 41 workflows

Calibrated honestly

A hit is a genuine inconsistency but not proof the workflow is deadtests.yml starts despite its residual mismatch. The reliable direction is the converse: workflows that were dead had drifted entries. The tool says so in its own header so nobody over-reads its output.

Two self-inflicted defects caught in review

Both are faults this estate has already been bitten by, which is why they're called out rather than quietly fixed:

  1. The issue-body heredoc sat at column 0, terminating the enclosing run: | block scalar — the exact defect that made .git-private-farm/inbox-steward-propagate.yml unparseable and blocked gh actions-lock from running at all.
  2. [ test ] && cmd under set -e exits the run when the test is false — and false was the default path.

A lockfile entry was added for the new workflow. Without it, it would die at 0s — the very fault it exists to detect.

🤖 Generated with Claude Code

…iller

THE FAULT. Dependabot bumps an action version in a workflow file. Nobody
regenerates .github/workflows/actions.lock. The workflow then requests a
version the lockfile does not record, and GitHub rejects the run at
startup_failure with ZERO jobs -- no log, no annotation, nothing in REST
or GraphQL.

It is SELF-REINFLICTING: any repo with dependabot AND a lockfile AND no
regeneration step re-acquires the fault on every bump. Measured
2026-08-25: 60 of 60 sampled repos carry both. That is why repos appear
to "go bad again" after being fixed -- nobody broke them, the clock did.

PROVEN on hypatia#723. Three drifted entries (docker/setup-buildx-action,
github/codeql-action, taiki-e/install-action) discriminated PERFECTLY:
every workflow using one was dead, every workflow using none was alive.
Regeneration took all four dead workflows from startup_failure to
running, and CodeQL to green.

Ships:
  * scripts/check-lockfile-drift.sh -- per-repo checker, TSV output
  * .github/workflows/lockfile-drift-detect.yml -- weekly estate sweep,
    Tuesday 07:20 UTC (after dependabot's window, so a bump and its drift
    are caught the same week)

REPORT-ONLY by design. It opens/updates one tracking issue and never
mutates another repository, per the estate guardrail that unattended
cross-repo mutation is a human decision. It follows the one findings->
issue model in the estate that actually works (farm/drift-detect.yml).

Verified both ways, because a check that cannot fail is not a check:
  * against hypatia POST-fix  -> reports only a genuine residual SHA/tag
    mismatch in tests.yml (confirmed real: workflow pins 34e11487,
    lockfile records v7.0.1 -> 3d3c42e5; different commits)
  * against hypatia PRE-fix   -> finds all three known drifted actions
    across the correct six workflows
  * against a lockfile-free repo -> exits 0, out of scope (absence of a
    lockfile is failure mode 1, not drift)
  * dogfooded against standards itself -> clean, 41 workflows

Calibrated honestly: a hit is a genuine inconsistency but NOT proof the
workflow is dead -- tests.yml starts despite its residual mismatch. The
reliable direction is the converse: workflows that were dead had drifted
entries.

Two self-inflicted defects caught in review and fixed here, both worth
noting because the estate has been bitten by each before:
  * the issue-body heredoc sat at column 0, terminating the enclosing
    `run: |` block scalar -- the exact defect that made
    .git-private-farm/inbox-steward-propagate.yml unparseable and blocked
    gh actions-lock from running at all
  * `[ test ] && cmd` under `set -e` exits the run when the test is false,
    and false was the DEFAULT path

Lockfile entry added for the new workflow; without it, it would die at 0s
-- the very fault it exists to detect.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@hyperpolymath
hyperpolymath merged commit e23bb02 into main Aug 25, 2026
13 of 14 checks passed
@hyperpolymath
hyperpolymath deleted the feat/lockfile-drift-detector branch August 25, 2026 20:01
@sonarqubecloud

Copy link
Copy Markdown

@gitar-bot

gitar-bot Bot commented Aug 25, 2026

Copy link
Copy Markdown

Important

You are using the Gitar free plan. Upgrade to unlock code review, CI analysis, auto-apply, custom automations, and more.

Gitar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant