ci(gate): bump the hybrid-gate pin for the checkout blob filter - #6739
Merged
Conversation
`check-trailer` clones full history under a 5-minute budget and reads only commit trailers and changed path names — never file content. On this repo that clone is the whole job, and when it overruns the job reports `cancelled`, which the gate cannot distinguish from a supersede. The PR goes red with nothing wrong with it, and `full-gate-build` is skipped because it needs `check-trailer`. Measured here today: runs cancelled at exactly 5m04s and 5m02s, both inside that checkout step, on unrelated PRs. forkwright/.github#33 adds `filter: blob:none` to that checkout and to ai-attribution's. Full history and trees are kept, blobs are not fetched, and git lazily fetches any blob a step turns out to need.
|
This was referenced Aug 13, 2026
forkwright
added a commit
that referenced
this pull request
Aug 13, 2026
## Finding `detect CodeQL scope` failed at **5m05s** against `timeout-minutes: 5` on PR #6740. The dependent `codeql-pr` then failed four seconds later, having done nothing. This is the same defect #6739 fixed for `check-trailer`, in a workflow that fix could not reach. The job checks out with `fetch-depth: 0` — every commit, every tree, **every blob** — and then does exactly this: ``` git diff --name-only "origin/${BASE_REF}...HEAD" ``` Path names. It never opens a file. The blob download is the entire cost of the job. ## Why it is worse than a slow job When the clone overruns the budget, the job reports **`cancelled`**, not `failed`. GitHub uses that same status for a superseded run, so the two are indistinguishable in the checks list — a real timeout reads as "an older run got replaced." That is what made the `check-trailer` version of this survive so long, and it is why `codeql-pr` looks like a CodeQL problem when nothing about CodeQL is involved. ## Why #6739 did not cover it #6739 bumped the pin to `forkwright/.github`'s fixed reusable workflow, which carries the filter for `check-trailer`. aletheia's **own** workflows were never touched, and an audit shows **none of its 30 `actions/checkout` steps across 20 workflows carried `filter: blob:none`**. ## What changed Two jobs qualify — both clone everything and read only git metadata: | workflow | what it actually reads | |---|---| | `codeql-pr.yml` (`detect CodeQL scope`) | path names, via `git diff --name-only` | | `no-ai-attribution.yml` | commit messages, via `git log --format` | Both get `filter: blob:none`, keeping the commits and trees they do need. Each carries a `WHY(filter)` comment so a later reader does not "tidy" it away as a stray option. ## Deliberately not changed The filter is not free for every job, so this is not a sweep: - **`codeql-pr.yml`'s second checkout** feeds CodeQL's analyzer, which reads sources. It needs blobs. - **`gate-attestation.yml`** reads commit trailers *and* builds, across three separate checkouts. Those need judging one at a time, not as a class. - Every build/test job legitimately needs the full tree — a lazy blob fetch during a compile is slower, not faster. Filed as **#6757** so the remaining 28 get triaged deliberately rather than blanket-patched. ## Verification Both files parse as valid YAML (`yaml.safe_load`). The real proof is the next run of `detect CodeQL scope`: it should drop from ~5 minutes to seconds, the way `check-trailer` went from a 5m02s timeout-cancel to **7 seconds** after the equivalent fix. --- ## Third instance, caught live `attestation-verify` was **cancelled at 5m04s** on PR #6749 while this PR was open. The gate's own evaluation says it outright: ``` ATTESTATION_VERIFY_RESULT: cancelled CANCELLED: attestation-verify did not run to a verdict (result: cancelled). This is NOT a check failure: re-run the job. ``` So a PR with nothing wrong with it reads as gate-failed, and the only remedy is knowing to re-run. That job reads a commit message, one `kanon.toml`, and a tree hash. **It never builds.** Its entire 5-minute budget was going to blobs it does not open. I had deferred all of `gate-attestation.yml` from this PR on the grounds that the workflow "reads trailers *and* builds". That was too coarse a judgment: the file has three checkouts and they are not alike. `attestation-verify` is metadata-only and is now filtered. The other two are untouched — `gate-coverage-scripts` runs `cargo metadata` and scripts across the tree, and genuinely wants blobs. Three demonstrated instances now — `check-trailer`, `detect CodeQL scope`, `attestation-verify` — each found only by reading a job log after something else looked broken. That is the argument for #6757 doing the remaining triage systematically rather than waiting for each one to bite. --------- Co-authored-by: Cody Kickertz <admin@forkwright>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
check-trailerin the shared hybrid gate clones full history undertimeout-minutes: 5, but reads only commit trailers and changed path names — never file content. On a repo this size that clone is the job.When it overruns, the job reports
cancelled. The gate cannot distinguish that from a supersede, so the PR goes red with nothing wrong with it — and becausefull-gate-buildhasneeds: check-trailer, the build is skipped rather than run, which presents as a build failure.Evidence from this repo, today
31621077227check-traileractions/checkout31632599095check-traileractions/checkoutBoth land on the timeout to within seconds, both inside the checkout step, with sibling jobs in the same run succeeding — in the second,
ai-attributionfinished in 21 seconds whilecheck-trailersat in checkout for five minutes.This kept reappearing as four different-looking failures across unrelated PRs (#6721, #6723, #6731, #6737), which is what made it read as four independent breaks rather than one infrastructure timeout.
It is a different root cause from #6701's concurrency keying: that cancelled whole runs via the concurrency group; this cancels a single job while its siblings succeed.
What the bump brings
forkwright/.github#33 adds
filter: blob:nonetocheck-trailer's andai-attribution's checkouts. Full history and all trees are kept — sogit log, trailer reads andgit diff --name-onlyare unaffected — while blobs go unfetched.It is safe against future edits: a partial clone lazily fetches any blob a step turns out to need, so it cannot break a step that reads content.
full-gate-buildkeeps its unfiltered clone, since it compiles the tree.aed4ca519→963532686f7853fa3555aab434455693969b63a1.