Skip to content

ci(node-runtime-guard): evaluate with the runner's pinned mip, not the stale shim - #761

Merged
bryan-minimal merged 3 commits into
mainfrom
ci-guard-current-evaluator
Oct 11, 2026
Merged

bryan-minimal merged 3 commits into
mainfrom
ci-guard-current-evaluator

Conversation

@bryan-minimal

@bryan-minimal bryan-minimal commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Why every PR's guard is red

Since #741 merged (2026-09-23 23:58), node-runtime-guard fails in Dump catalog on every PR (go-1.27.1, node-26, cadical, binaryen, ghidra, …):

error: contract broken by a value
       failed to validate source_provenance
  packages/tree/build.ncl:61   category = 'Gitlab, host = "gitlab.com", …

tree's recipe is valid. pkgs CI installs minimal through the public shim, and the shim's CLI pointer (gs://minimal-shim/config/cli-*-linux.json) has said 422f245 ("this is 0.4.1", 2026-05-28) since May. That build predates 'Gitlab provenance, which minimal added on 07-14 (d06a77db). So CI evaluated recipes with a four-month-old minimal, and the guard correctly failed closed on a catalog it couldn't read. The build-bot and the pkgsec runner use current evaluators, which is why only GitHub Actions broke.

Fix

A new composite action, .github/actions/setup-mip:

  • Fetches the public minimalone-25d38aaa archive and sha256-verifies it.
  • Extracts mip-linux-amd64 and sha256-verifies the binary. It is byte-identical to the pkgsec runner's pinned evaluator (MIP_SHA256 2c5e01cd…a1ed), so CI and production now evaluate the catalog with the same binary.
  • Caches it. The binary hash is re-checked on every run, whether or not the cache hit.

The guard's dump step now calls mip dump. This PR changes the guard workflow, so the guard runs on this PR: that run is the end-to-end test.

Follow-ups (not in this PR)

  • The other minimal dump workflows (version-age, stable-closure-check, …) are report-only and still use the shim; they should move to setup-mip too.
  • The root cause is the stale shim pointer. It's the minimal team's channel, so advancing it fixes this for every consumer of install.sh.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores
    • Updated automated catalog and compatibility checks to use a version-pinned MIP evaluator across both supported architectures. The evaluator is downloaded only after its archive is verified, and its binary is checked before use, including when retrieved from cache. Workflows also record the evaluator’s version when available and identify it consistently in validation reports. Dump failures continue to follow the existing handling behavior.

…e stale shim

Every PR's guard has failed since #741 merged (2026-09-23 23:58) in "Dump
catalog":
    error: contract broken by a value / failed to validate source_provenance
    packages/tree/build.ncl:61  category = 'Gitlab
The recipe is valid. The public shim's cli pointer
(gs://minimal-shim/config/cli-*-linux.json) has said 422f245 (2026-05-28,
"this is 0.4.1") since May, and that predates 'Gitlab provenance (minimal
d06a77db, 07-14). So CI evaluated with a four-month-old minimal, and the guard
correctly failed closed on a catalog it could not read.

New composite action .github/actions/setup-mip: fetch the public
minimalone-25d38aaa archive, sha256-verify the archive and then the extracted
mip-linux-amd64, cache it, and put it on PATH. That mip is byte-identical to
the pkgsec runner's pinned evaluator (MIP_SHA256 2c5e01cd...a1ed), so the guard
and production now evaluate the catalog with the same binary. The binary hash
is re-checked on every run, cache hit or not. The guard's dump step now uses
`mip dump`.

The other report-only workflows still use the shim and should move to
setup-mip too (follow-up).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@bryan-minimal
bryan-minimal requested a review from a team as a code owner September 24, 2026 16:00
@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: gominimal/pkgs/.coderabbit.yaml

Review profile: CHILL

Plan: Essentials

Run ID: 5d33915f-fd60-4567-aced-53fba673e037

📥 Commits

Reviewing files that changed from the base of the PR and between d8692b0 and 43f0438.

📒 Files selected for processing (4)
  • .github/actions/setup-minimal/action.yml
  • .github/scripts/version_age_report.py
  • .github/workflows/stable-closure-check.yml
  • .github/workflows/version-age.yml
💤 Files with no reviewable changes (1)
  • .github/actions/setup-minimal/action.yml

Included review availability: 0 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour.


📝 Walkthrough

Walkthrough

The pull request adds a composite action that installs and verifies a pinned mip binary. Runtime and version-age workflows switch from Minimal to mip. The version-age report also updates its dump error labels.

Changes

Pinned mip evaluator migration

Layer / File(s) Summary
Pinned mip setup
.github/actions/setup-mip/action.yml, .github/actions/setup-minimal/action.yml
The new action caches mip by version and binary hash. On cache misses, it downloads and verifies the archive, extracts the binary, and makes it executable. It verifies the binary on every run before adding its directory to GITHUB_PATH. The previous Setup Minimal action is removed.
Workflow and report migration
.github/workflows/node-runtime-guard.yml, .github/workflows/stable-closure-check.yml, .github/workflows/version-age.yml, .github/scripts/version_age_report.py
The workflows use setup-mip and mip for catalog dumps, stable closure checks, and version-age dumps. The version-age report identifies mip dump in its error and unexpected-shape messages. Existing dump failure handling remains in place.

Sequence Diagram(s)

sequenceDiagram
  participant RuntimeWorkflows
  participant SetupMip
  participant GitHubCache
  participant ReleaseArchive
  participant Mip
  RuntimeWorkflows->>SetupMip: Set up pinned mip
  SetupMip->>GitHubCache: Look up binary by version and hash
  SetupMip->>ReleaseArchive: Download archive on cache miss
  ReleaseArchive-->>SetupMip: Return archive
  SetupMip->>SetupMip: Verify archive and binary hashes
  SetupMip-->>RuntimeWorkflows: Add verified binary directory to GITHUB_PATH
  RuntimeWorkflows->>Mip: Run catalog, closure, or version-age dump commands
Loading

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 43f04

No actionable issue was established that would block merging. The first CI run should still confirm the pinned evaluator’s output.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (2 skipped: 2 … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the primary change: node-runtime-guard now evaluates with the runner's pinned mip binary instead of the stale shim. It is concise and specific.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

bryan-minimal and others added 2 commits September 24, 2026 13:56
Both report-only jobs used setup-minimal, i.e. the public shim's stale
minimal (422f245), whose `minimal dump` fails on current stdlib features.
They stayed green by design, so the failure was invisible: version-age
warned and no-op'd on an empty dump, stable-closure-check printed
UNVERIFIED under --no-fail. Switch both to setup-mip (the runner's
sha256-verified 25d38aaa) with identical dump arguments; report-only
semantics are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…on-age's summary

setup-minimal resolved to the public shim's stale 422f245 build. No workflow
uses it after this PR, and leaving it around invites copying it into a new
one. version-age's step summary still said "minimal dump" after the job
moved to mip, so point triage at the right tool.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@bryan-minimal

Copy link
Copy Markdown
Member Author

Scope extended: every CI job that evaluates the catalog now uses the pinned mip.

  • version-age.yml: setup-mip, then mip dump with the same args. Still warn-and-continue.
  • stable-closure-check.yml: setup-mip, plus --minimal-bin mip (the script already took the flag). Still continue-on-error + --no-fail.
  • .github/actions/setup-minimal is removed. No workflow used it any more, and it resolved to the stale 422f245 build.
  • version-age's step summary now names mip dump, not minimal dump.

Not changed on purpose: ci.yml / minimal-check uses gominimal/check@v1 (sandbox builds and checks), which is a different evaluator and out of scope. blast-radius, dwell, trivial-update and smoke-test-coverage only mention minimal dump in comments; they're pure git/text jobs.

Checked before this push:

  • stable-closure-check.py --minimal-bin mip gives PASS. Negative control: a nonexistent binary gives UNVERIFIED.
  • version-age parses a mip dump.
  • All workflow YAML parses.

Not verified yet: these checks used a local mip build, not the pinned 25d38aaa. On this PR's first CI run, version-age should log "dumped N packages" with N > 0, and stable-closure-check should give PASS or NOT CLOSED, not UNVERIFIED.

@bryan-minimal
bryan-minimal added this pull request to the merge queue Sep 29, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Sep 29, 2026
@bryan-minimal
bryan-minimal added this pull request to the merge queue Oct 11, 2026
Merged via the queue into main with commit 549a572 Oct 11, 2026
11 checks passed
@bryan-minimal
bryan-minimal deleted the ci-guard-current-evaluator branch October 11, 2026 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants