ci: emit the gate status context the org ruleset requires - #18
Merged
Merged
Conversation
The paiml org ruleset "Green Main" (id 13878864) requires exactly one status
context:
$ gh api orgs/paiml/rulesets/13878864 \
--jq '.rules[]|select(.type=="required_status_checks")
|.parameters.required_status_checks[].context'
gate
This repo had one workflow, .github/workflows/pr-gate.yml, and it emits a job
named `authorize`, not `gate`. Nothing here has ever produced a check named
`gate`. So every pull request reports mergeStateStatus: BLOCKED with every
visible check green, and no contributor can fix it -- the missing check is one
the repo cannot produce. PR #17 (dependabot, nokogiri 1.9.1 -> 1.14.3) has sat
BLOCKED with an empty statusCheckRollup for exactly this reason. Eight repos in
the org are in this state.
A ruleset naming a context no workflow emits does not raise the bar, it closes
the repo. The fix is to actually emit it, the same way
paiml/ds500-course-processing was unblocked.
What the gate measures
This repo is a Jupyter Book / Jekyll course with no test suite to aggregate.
pr-gate.yml cannot be aggregated: it is a reusable org workflow on
pull_request_target, opened/reopened only, so it does not run on pushes to a
PR. So the gate checks what the repo actually is, via
scripts/ci_gate_check.py:
* notebooks -- 60 tracked .ipynb parse as JSON and have nbformat + cells,
every cell having cell_type and source
* yaml -- 6 tracked .yml parse
* json -- 4 tracked .json parse
* python -- 4 tracked .py compile
* toc -- 20 internal _data/toc.yml urls each resolve to a real
content/<url>.{md,ipynb}; this catches a renamed or deleted
lesson that would otherwise ship as a dead sidebar entry
* lessons -- all 14 top-level Colab notebooks are published into
content/lessons/, the invariant `make book` maintains
Every check prints its denominator ("checked N ...") and FAILS when N is zero,
so a check that silently inspects nothing cannot read as a pass. If PyYAML is
not importable the script exits 2 rather than skipping: an absent verifier is a
NO-GO, not a pass.
`ci_gate_check.py selftest` runs first and proves each checker can fail: it
builds deliberately broken fixtures in a temp dir and asserts every checker
rejects them, and that each reports denominator 0 on an empty tree. Verified by
mutation -- neutering the JSON checker's error append makes selftest exit 1
with "SELFTEST FAIL: json accepted a broken fixture". A guard that has never
been shown to fail is not evidence of anything.
Two naming/runner constraints, deliberately
* The job id and `name:` are both the literal string `gate`. The ruleset
matches the context, not a display name; a friendlier name silently
re-blocks every PR here.
* runs-on is ubuntu-latest, not the self-hosted fleet. This repo is public
and a `pull_request` job runs the head ref's own code, so routing it to the
clean-room runners would execute fork-authored content on the fleet. Bare
`self-hosted` would also match the aarch64 gx10 training node
(paiml/infra#342).
Adding this in a PR bootstraps itself: a pull_request workflow added in a PR
runs on that PR, so this PR emits its own `gate` and can merge normally. No
admin override is needed or used.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect, measured
The paiml org ruleset "Green Main" (id 13878864) requires exactly one status context:
This repo has one workflow —
.github/workflows/pr-gate.yml— and it emits a job namedauthorize, notgate. Nothing in this repository has ever produced a check namedgate.So every pull request here reports
mergeStateStatus: BLOCKEDwhile every visible check is green, and there is nothing a contributor can fix — the missing check is one the repo cannot produce. PR #17 (dependabot, nokogiri 1.9.1 → 1.14.3) has sat blocked with a literally emptystatusCheckRollupfor exactly this reason. Eight repos in the org are in this state right now.The only way through would be an admin override, which the org rules forbid, correctly. A ruleset naming a context no workflow emits does not raise the bar — it closes the repo.
paiml/ds500-course-processinghad exactly this and was unblocked by adding agatejob; this is the same fix.What the gate actually measures
This repo is a Jupyter Book / Jekyll course with no test suite to aggregate.
pr-gate.ymlcannot be aggregated withneeds:— it is a reusable org workflow onpull_request_target,opened/reopenedonly, so it does not run on pushes to a PR at all.So the gate checks what this repo actually is, via
scripts/ci_gate_check.py:notebooks.ipynbparses as JSON and hasnbformat+cells, each cell havingcell_typeandsourceyaml.ymlparsesjson.jsonparsespython.pycompilestoc_data/toc.ymlurl resolves to a realcontent/<url>.{md,ipynb}— catches a renamed or deleted lesson that would otherwise ship as a dead sidebar entrylessonscontent/lessons/, the invariantmake bookmaintainsEvery step prints its denominator (
checked N ...) and FAILS when N is zero, so a check that silently inspects nothing cannot read as a pass. If PyYAML is not importable the script exits 2 rather than skipping — an absent verifier is a NO-GO, not a pass.The gate proves it can fail
The first step is
ci_gate_check.py selftest, which builds deliberately broken fixtures in a temp dir and asserts every checker rejects them, plus that each reports denominator 0 on an empty tree:Verified by mutation: neutering the JSON checker's error append makes selftest exit 1 with
SELFTEST FAIL: json accepted a broken fixture. A guard that has never been shown to fail is not evidence of anything, and agatethat is effectivelyexit 0would be worse than the block it replaces.Two constraints, deliberately
name:are both the literal stringgate. The ruleset matches the context, not a display name — a friendlier name would silently re-block every PR in this repo.runs-on: ubuntu-latest, not the self-hosted fleet. This repo is public and apull_requestjob runs the head ref's own code, so routing it to the clean-room runners would execute fork-authored content on the fleet. Bareself-hostedis also wrong regardless: it matches the aarch64 gx10 training node (paiml/infra#342).Bootstrapping
A separate PR adding
gatewould itself be blocked for not emittinggate— chicken and egg. It resolves itself: apull_requestworkflow added in a PR runs on that PR, so this PR emits its owngateand can merge normally. Nothing is bypassed.Once this merges, PR #17 and every future PR here become mergeable on their own merits.
🤖 Generated with Claude Code