Skip to content

Run the gate on the self-hosted runner, and gate the hosted jobs - #453

Merged
KenTaniguchi-R merged 4 commits into
mainfrom
ci/self-hosted-runner
Aug 17, 2026
Merged

Run the gate on the self-hosted runner, and gate the hosted jobs#453
KenTaniguchi-R merged 4 commits into
mainfrom
ci/self-hosted-runner

Conversation

@KenTaniguchi-R

Copy link
Copy Markdown
Owner

Refs #452.

Hosted Actions minutes are unavailable — jobs fail with zero steps and the annotation "recent account payments have failed". That blocked the 0.5.1 publish, which then went out manually and therefore with attestations: none.

What moves

Job Runner Trigger
ci.yml verify [self-hosted, macOS, ARM64] push to main, pull_request, dispatch
invariants.yml check [self-hosted, macOS, ARM64] push to main, pull_request, dispatch
release.yml build / publish / attach [self-hosted, macOS, ARM64] release published

release.yml moving matters most: OIDC trusted publishing works on a self-hosted runner, because GitHub issues the token rather than the runner host. 0.5.2 gets its provenance back.

What deliberately does not move

packed-cli-consumer and windows-compat stay hosted, gated with:

if: github.event_name == 'workflow_dispatch'

Neither can run on one macOS ARM64 box — the first matrixes over ubuntu-latest and has no checkout by design, the second needs Windows (#251, #252). Without the gate they would fire on every push and fail on billing, leaving CI permanently red for reasons unrelated to the change. Drop both if: lines when hosted billing returns.

The test that had to change

release-workflow.test.ts asserted blanket manual-only for both workflows. That premise held while every job was billable; it now would keep the gate switched off for jobs that cost nothing.

Replaced with the narrower rule that still has to hold: no automatic trigger may reach a job on a hosted runner — checked per job rather than per workflow, so a third hosted job added later is caught too. Plus an assertion that verify and check really are on the runner, so the move cannot silently revert.

I could not plant an end-to-end violation, because planting one means editing a guarded path. The detection logic was verified in isolation against synthetic workflows: gated → passes, ungated hosted job on an automatic trigger → flags it by name, manual-only workflow → correctly skipped. To confirm against the real file: delete one if: line, run the test, restore it.

Not changed

No --repo on the attach step. GH_REPO: ${{ github.repository }} already landed between v0.5.0 and v0.5.1 and covers it — it has simply never run, because build died before attach on the last release.

Verification

pnpm verify passes — six steps, ten invariants, no warnings, first try, and the pre-push hook passed without --no-verify.

This PR is its own test. If the checks below run on macmini-agentcall rather than queueing against unavailable hosted minutes, the change works.

🤖 Generated with Claude Code

Hosted Actions minutes are unavailable — a release job fails with zero steps
and the annotation "recent account payments have failed". That blocked the
0.5.1 publish, which then went out manually and therefore without provenance.

`verify` and the invariants check now run on macmini-agentcall. Both cost
nothing, so the workflows can fire on push and pull_request again rather than
sitting behind workflow_dispatch, and `release.yml` moves too: OIDC trusted
publishing works on a self-hosted runner because GitHub issues the token, not
the runner host, so 0.5.2 gets its attestation back.

`packed-cli-consumer` and `windows-compat` stay hosted and are gated to manual
dispatch. Neither can run on one macOS ARM64 box — the first matrixes over
ubuntu-latest and deliberately has no checkout, the second needs Windows — and
without the gate they would fire on every push and fail on billing, leaving CI
permanently red for reasons unrelated to the change. Drop both `if:` lines when
hosted billing returns.

The test that enforced blanket manual-only asserted a premise that has stopped
being true: it made sense while every job was billable, and would now keep the
gate switched off for free jobs. Replaced with the narrower rule that still
has to hold — no automatic trigger may reach a job on a hosted runner, checked
per job rather than per workflow, so a third hosted job added later is caught
too — plus an assertion that `verify` and `check` really are on the runner.

Refs #452. Deliberately not adding `--repo` to the attach step; GH_REPO already
covers it and has simply never run.
`YAML.safe_load_file` needs Psych 3.3, which means Ruby 3.0. The hosted ubuntu
image had that; the self-hosted runner ships an older stock Ruby, so the
invariants job died with `undefined method 'safe_load_file' for Psych:Module`.

The failure reads like a pinning violation — the job is named "Third-party
actions are immutable" and it exits 1 — when in fact the check never ran.

Reading the file ourselves works on every Ruby that takes keyword arguments to
`safe_load`, which is 2.6 onward. Verified behaviour-preserving: both forms
parse all five workflow files to identical structures.

Fixed here rather than by pinning a Ruby in the workflow, because the script
should not require a specific toolchain to run by hand — and because
.github/workflows/ is a hard deny for automated edits.
The pattern was written with literal backslashes:

    grep -F 'app.use(\"/v1/*\", requireIdentity)'

Inside single quotes those are not escapes, so the search string contained
backslashes the source never has. apps/relay/src/index.ts:37 reads
`app.use("/v1/*", requireIdentity);`, so the check could not match on any
machine and reported "Missing the fail-closed /v1 identity middleware" whenever
it ran.

It has not run since Actions billing broke, which is why nobody saw it.
Re-enabling the workflow surfaced it on the first execution.
scripts/ci-local.sh:146 carries the correct unescaped form, so the local gate
was quietly stricter than the remote one this whole time.

This is the only escaped-quote grep in the file.
@KenTaniguchi-R

Copy link
Copy Markdown
Owner Author

Verified live — both workflows green on the runner

CI:         success
  verify:               success  [macmini-agentcall]
  packed-cli-consumer:  skipped  (hosted, gated to workflow_dispatch)
  windows-compat:       skipped  (hosted, gated to workflow_dispatch)

Invariants: success
  check:                success  [macmini-agentcall]

First automatic CI on this repository since Actions billing broke. The if: gate behaves as intended: hosted jobs skip rather than queueing into a billing failure.

Three blockers surfaced on the way, none of them the YAML

1. Both workflows were disabled_manually. Not a trigger problem — CI and Invariants were switched off at the GitHub level, in addition to being reduced to workflow_dispatch. No amount of trigger editing would have started them. Re-enabled with gh workflow enable.

2. Psych.safe_load_file is not available on the runner. It needs Psych 3.3 (Ruby 3.0); the Mac mini ships an older stock Ruby, and the hosted ubuntu image had hidden this. The job died inside the step named "Third-party actions are immutable", which reads like a pinning violation when in fact the check never ran.

Fixed in scripts/verify-action-pins.rb rather than by pinning a Ruby in the workflow — the script should not need a specific toolchain to run by hand. Verified behaviour-preserving: both read forms parse all five workflow files to identical structures.

3. The identity-middleware grep has never worked. invariants.yml:101 searched for a pattern containing literal backslashes:

grep -F 'app.use(\"/v1/*\", requireIdentity)'

Inside single quotes those are not escapes, so the search string carried backslashes the source never has. apps/relay/src/index.ts:37 is app.use("/v1/*", requireIdentity);, so the check failed on every machine and always has — it simply had not executed since billing broke. scripts/ci-local.sh:146 has the correct unescaped form.

What this implies about the gate

The local mirror was stricter than the remote one, which is the safe direction — but nothing would have caught the reverse.

inv_gate_mirrors_ci only classifies shell steps in ci.yml's verify job, so invariants.yml can drift from scripts/ci-local.sh without any check noticing. That is a real blind spot in the mechanism CLAUDE.md relies on, and it is how a check sat broken long enough to be forgotten. Worth extending the invariant to cover invariants.yml too — filing separately rather than widening this PR.

@KenTaniguchi-R
KenTaniguchi-R merged commit 6db5f7f into main Aug 17, 2026
6 checks passed
@KenTaniguchi-R
KenTaniguchi-R deleted the ci/self-hosted-runner branch August 17, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant