Skip to content

ci: emit the gate status context the org ruleset requires - #18

Merged
noahgift merged 1 commit into
masterfrom
ci/emit-gate-context
Aug 28, 2026
Merged

noahgift merged 1 commit into
masterfrom
ci/emit-gate-context

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

The defect, measured

The paiml org ruleset "Green Main" (id 13878864) requires exactly one status context:

$ gh api orgs/paiml/rulesets/13878864 \
    --jq '.rules[]|select(.type=="required_status_checks")|.parameters.required_status_checks[].context'
gate

This repo has one workflow — .github/workflows/pr-gate.yml — and it emits a job named authorize, not gate. Nothing in this repository has ever produced a check named gate.

So every pull request here reports mergeStateStatus: BLOCKED while every visible check is green, and there is nothing a contributor can fix — the missing check is one the repo cannot produce. PR #17 (dependabot, nokogiri 1.9.1 → 1.14.3) has sat blocked with a literally empty statusCheckRollup for exactly this reason. Eight repos in the org are in this state right now.

The only way through would be an admin override, which the org rules forbid, correctly. A ruleset naming a context no workflow emits does not raise the bar — it closes the repo. paiml/ds500-course-processing had exactly this and was unblocked by adding a gate job; this is the same fix.

What the gate actually measures

This repo is a Jupyter Book / Jekyll course with no test suite to aggregate. pr-gate.yml cannot be aggregated with needs: — it is a reusable org workflow on pull_request_target, opened/reopened only, so it does not run on pushes to a PR at all.

So the gate checks what this repo actually is, via scripts/ci_gate_check.py:

step what it proves denominator today
notebooks every tracked .ipynb parses as JSON and has nbformat + cells, each cell having cell_type and source 60
yaml every tracked .yml parses 6
json every tracked .json parses 4
python every tracked .py compiles 4
toc every internal _data/toc.yml url resolves to a real content/<url>.{md,ipynb} — catches a renamed or deleted lesson that would otherwise ship as a dead sidebar entry 20
lessons every top-level Colab notebook is published into content/lessons/, the invariant make book maintains 14

Every step prints its denominator (checked N ...) and FAILS when N is zero, so a check that silently inspects nothing cannot read as a pass. If PyYAML is not importable the script exits 2 rather than skipping — an absent verifier is a NO-GO, not a pass.

The gate proves it can fail

The first step is ci_gate_check.py selftest, which builds deliberately broken fixtures in a temp dir and asserts every checker rejects them, plus that each reports denominator 0 on an empty tree:

selftest ok: notebooks rejects broken input (n=1)
selftest ok: notebooks reports denominator 0 on empty input
selftest ok: notebooks-shape rejects broken input (n=1)
selftest ok: yaml rejects broken input (n=1)
... 13 assertions ...
checked 7 checker(s) for falsifiability
OK: every checker demonstrably fails on broken input.

Verified by mutation: neutering the JSON checker's error append makes selftest exit 1 with SELFTEST FAIL: json accepted a broken fixture. A guard that has never been shown to fail is not evidence of anything, and a gate that is effectively exit 0 would be worse than the block it replaces.

Two constraints, deliberately

  • The job id and name: are both the literal string gate. The ruleset matches the context, not a display name — a friendlier name would silently re-block every PR in this repo.
  • runs-on: ubuntu-latest, not the self-hosted fleet. This repo is public and a pull_request job runs the head ref's own code, so routing it to the clean-room runners would execute fork-authored content on the fleet. Bare self-hosted is also wrong regardless: it matches the aarch64 gx10 training node (paiml/infra#342).

Bootstrapping

A separate PR adding gate would itself be blocked for not emitting gate — chicken and egg. It resolves itself: a pull_request workflow added in a PR runs on that PR, so this PR emits its own gate and can merge normally. Nothing is bypassed.

Once this merges, PR #17 and every future PR here become mergeable on their own merits.

🤖 Generated with Claude Code

The paiml org ruleset "Green Main" (id 13878864) requires exactly one status
context:

    $ gh api orgs/paiml/rulesets/13878864 \
        --jq '.rules[]|select(.type=="required_status_checks")
              |.parameters.required_status_checks[].context'
    gate

This repo had one workflow, .github/workflows/pr-gate.yml, and it emits a job
named `authorize`, not `gate`. Nothing here has ever produced a check named
`gate`. So every pull request reports mergeStateStatus: BLOCKED with every
visible check green, and no contributor can fix it -- the missing check is one
the repo cannot produce. PR #17 (dependabot, nokogiri 1.9.1 -> 1.14.3) has sat
BLOCKED with an empty statusCheckRollup for exactly this reason. Eight repos in
the org are in this state.

A ruleset naming a context no workflow emits does not raise the bar, it closes
the repo. The fix is to actually emit it, the same way
paiml/ds500-course-processing was unblocked.

What the gate measures

This repo is a Jupyter Book / Jekyll course with no test suite to aggregate.
pr-gate.yml cannot be aggregated: it is a reusable org workflow on
pull_request_target, opened/reopened only, so it does not run on pushes to a
PR. So the gate checks what the repo actually is, via
scripts/ci_gate_check.py:

  * notebooks  -- 60 tracked .ipynb parse as JSON and have nbformat + cells,
                  every cell having cell_type and source
  * yaml       -- 6 tracked .yml parse
  * json       -- 4 tracked .json parse
  * python     -- 4 tracked .py compile
  * toc        -- 20 internal _data/toc.yml urls each resolve to a real
                  content/<url>.{md,ipynb}; this catches a renamed or deleted
                  lesson that would otherwise ship as a dead sidebar entry
  * lessons    -- all 14 top-level Colab notebooks are published into
                  content/lessons/, the invariant `make book` maintains

Every check prints its denominator ("checked N ...") and FAILS when N is zero,
so a check that silently inspects nothing cannot read as a pass. If PyYAML is
not importable the script exits 2 rather than skipping: an absent verifier is a
NO-GO, not a pass.

`ci_gate_check.py selftest` runs first and proves each checker can fail: it
builds deliberately broken fixtures in a temp dir and asserts every checker
rejects them, and that each reports denominator 0 on an empty tree. Verified by
mutation -- neutering the JSON checker's error append makes selftest exit 1
with "SELFTEST FAIL: json accepted a broken fixture". A guard that has never
been shown to fail is not evidence of anything.

Two naming/runner constraints, deliberately

  * The job id and `name:` are both the literal string `gate`. The ruleset
    matches the context, not a display name; a friendlier name silently
    re-blocks every PR here.
  * runs-on is ubuntu-latest, not the self-hosted fleet. This repo is public
    and a `pull_request` job runs the head ref's own code, so routing it to the
    clean-room runners would execute fork-authored content on the fleet. Bare
    `self-hosted` would also match the aarch64 gx10 training node
    (paiml/infra#342).

Adding this in a PR bootstraps itself: a pull_request workflow added in a PR
runs on that PR, so this PR emits its own `gate` and can merge normally. No
admin override is needed or used.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift
noahgift merged commit 7251cc4 into master Aug 28, 2026
2 checks passed
@noahgift
noahgift deleted the ci/emit-gate-context branch August 28, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant