[MaxGhenis]

Lets any team member or agent session build, sign and assemble a Microcosm UK national cut, and open its policyengine.py certification, without the one machine that held the key and some of the inputs. Every input is fetched by the pins the build already enforces. Draft until the independent review agrees.

What's in it

1. Signing-key registry (microcosm.data.gate_signing_keys).
   • The gap. UK gate reports are HMAC-SHA256 signed, and every verifier trusted whichever key was in MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY.
   • Verification. The publisher's schema-4 checks (battery report, release-cut certification, dense report) now accept only a registered active or retired key.
   • Signing. Every UK signer requires an active one: the gate battery, the certifier, the scoped re-signer and the local preflight.
   • The national role checks its key before loading any input; a missing key used to fail only after the solve (runbook F8).
   • The registry holds one key per builder. HMAC verification needs the signer's secret, so a cut is signed, certified and published by one key holder.
   • Continuity. It starts with the key that signed every cut on record, 66bf0e29…. The team key dd31e325… is registered separately in Register a team UK gate signing key (hold: needs María's agreement) #1350, held for María's agreement.
   • CLI. python -m microcosm.data.gate_signing_keys {check,list,generate}.
2. tools/run_uk_cut.py: one resumable command. fetch → preflight → spine → national (dry run first) → score (only if the in-build evaluation did not pass) → certify → stage-1 (diagnostic) → assemble → publish preflight → publish (opt-in).
   • Done-checks read the exact fields the tools write (build record, score receipt, certification, assembly summary), so re-running resumes.
   • Never destructive. It never rebuilds over a spine, overwrites a calibration run (it moves one aside only with --retry-national) or re-assembles a release.
   • Inspection only. --publish is always --no-latest --tag-only, so main's files and latest-*.json never move (runbook Q12, F14).
   • From main. It builds only from origin/main, unless --allow-off-main "<who agreed>".
3. tools/uk_cut_inputs.json says only where each input lives. Digests come from source_stages.json, frs_release.json, chronicle_feed.json, gates.json and the mass-reference pin.
   • Verified here on 10/22:
     • FRS, SPI, WAS and ETB come byte-exact from the gated zips in policyengine-uk-data-private (all 18 tabs match their pins).
     • The HMRC ODS comes from gov.uk.
     • The incumbent eFRS comes from HF.
     • The input-mass reference comes from the new private policyengine/microcosm-uk-build-inputs. It was reproduced byte-for-byte to fd41cb5f… at microcosm 0eac9bbbd with policyengine-uk 2.89.0; today's engine emits 2 more columns, and the shared 131 totals are identical.
   • Pending:
     • NTS 2002-2024 and LCFS 2024-25 need a one-time upload by a UKDS licence holder (exact commands in the doc).
     • The Chronicle UK artifact is being rebuilt on Modal.
4. tools/modal_rebuild_uk_chronicle_artifact.py rebuilds the pinned Chronicle UK artifact from the public repo, which needs no R2 credentials. It hosts the artifact privately only if both digests reproduce; otherwise it uploads diagnostics.
5. tools/certify_uk_cut_in_policyengine_py.py: the policyengine.py certification handoff, as Net-new UK stages: UC deduction attributes, bus fares, WAS debt #685 did it by hand.
   • Checks first: one writer per cut, and a regional-route pre-check.
   • Then: certify-data, the lock, TROs, policyengine.py's own constant rewrite, snapshots, the strict check, the UC floor and the tests.
   • PR body. It drafts the PR with every judgment call as a checklist.
   • Accepted mismatch. For an accepted regional mismatch (Allow a UK local-area release on an accepted model mismatch (hold: needs María's routing call) policyengine.py#699) it measures what area runs lose and refuses dropped columns.
6. docs/uk-cut-anyone.md documents all of this, plus key rotation and revocation and the regional routes, with a recommendation.

Since the first review (2026-10-23)

• UK hub rehearsal folded in. The hub ran tools/certify_uk_cut_in_policyengine_py.py end to end on María's cut a133f37a (pe-uk 2.147.2), in 97 min and mechanically clean. Its 13 commits are here under their own authorship:
  • --base-ref and --pr-base, to stack on policyengine.py#704;
  • resource guards (now operator flags --max-load1 and --min-free-gb) and per-step timings;
  • streamed logs;
  • exact-core model probes;
  • scoped country imports;
  • a measured UC summary;
  • pytest exit-code authority;
  • -vv, durations and stall tracebacks;
  • the TMPDIR-outside-Git preflight;
  • --resume-worktree with allowlisted staging.
• GPT-6.1 Sol review round 1 (REQUEST_CHANGES), all fixed:
  • a retried spine never reruns over an attempt (--retry-spine moves it aside, keeping a gate-blocked attempt's signed report);
  • retried score, certification and stage-1 outputs keep the earlier file as .superseded-<time>;
  • --work-dir is made absolute;
  • the stage-1 done-check binds both inputs by sha256;
  • the verification cache keys on inode, size, mtime and ctime;
  • the regional loss is measured on the regional H5's own column inventory, fetched at its manifest pin and sha-checked, reading no values. Under 2.147.2 it finds nothing dropped and 16 inputs defaulted, matching the hub's independent probe.
• Run the UK test suite and a smoke-rung build in CI #699 now belongs to the UK hub. Its Saturday plan is national-only on UK rebuild: sequence calibration after the amounts imputations; staging seam has no CGT target parity evidence #704, so it is not on the critical path.

Regions (runbook Q13, María's call)

• The blocker. The only dense cut is on 2.132.6, so policyengine.py refuses to keep area routing for a national cut on a newer policyengine-uk.
• Measured cost of keeping it anyway (2.132.6 dense, its own columns, under 2.147.2):
  • no column is dropped on load;
  • PIP categories still load (they became formulas in 2.144.0, and stored values win);
  • 16 inputs default in area runs, including salary_sacrifice_returned_to_income, local_bus_uncapped_single_fare and directly_held_shares;
  • national figures are unaffected.
• Recommendation:
  • an accepted, recorded mismatch (policyengine.py#699) for the Sat/Mon re-cuts;
  • a dense cut on the final version for the Budget pair, if it can be started by Monday; otherwise the accepted mismatch.

Invariants (tested)

• Signing key:
  • An unregistered key never signs or verifies.
  • A registered key verifies only itself.
  • Active signs and verifies, retired only verifies, revoked does neither (Hypothesis).
  • A report signed under an active key passes the publisher's dense check, still passes once retired, and fails once revoked or unregistered (round trip).
  • The key behind every cut on record stays registered.
• Fetch:
  • A file whose bytes miss its pin is never installed, and no partial file is left behind (Hypothesis).
  • A second fetch downloads nothing.
  • A corrupted installed file is replaced, never trusted.
  • Archive digests are checked when pinned.
• Runner:
  • Each step runs once, and a second all runs nothing.
  • A failed step stops the run, and a rerun resumes at it without repeating finished steps.
  • A half-built spine is never rebuilt over.
  • A retried national run keeps the failed attempt.
  • A candidate that loses the incumbent score is never certified.
  • Every argv is accepted by its tool's own parser, or uses only flags its --help declares.
  • Publication is inspection-only for any tag.
• Pins. The runner's pins are the build's pins (lockstep with chronicle_feed.json, frs_release.json, gates.json and source_stages.json).
• Handoff:
  • Only cut tags are certified, never @main.
  • The constant rewrite touches quoted literals only, and is invertible.
  • The input-surface change is exact set algebra (Hypothesis).

Local results

• New tests: registry 17 (microcosm-data) + 15 (microcosm-build), runner 27, handoff 13 (+3 properties), all passing.
• Full engine-free suite (4 workers): 20,243 passed, 6 failed.
  • 1 was mine: the retired-package reference guard. It is fixed by allowlisting the inputs manifest and the doc, both of which name the gated input repo, as frs_release.json does.
  • 5 are environmental. 4 Orrery parser tests need the Orrery contract's node modules, which this checkout doesn't have installed, and 1 asserts the UK engine is absent, but this venv has the uk extra.

axiom: n/a: build infrastructure (no policy encoding)

🤖 Generated with Claude Code

[juaristi22]

NTS and LCFS are on the Hub. I've checked the licensing, and both inputs are now in policyengine/policyengine-uk-data-private:

• nts_2002_24.zip: the UKDS TAB download of SN 5340 (19th edition), unchanged; its sha256 is the one UKDS puts in the download's file name. Revision 559b69e94f0e03e696c5b7a55feffc54b7423717, sha256 bc9955021d12ca82230c75626d393574dc6a44d9cbd13f412af76c5af93bf136.
• lcfs_2024_25.zip: SN 9603 (2nd edition), TAB. I no longer had the original download zip, so this one repacks the extracted files: all seven tabs and the two mrdoc workbooks, under lcfs_2024_25/. Revision 1105105276991902d28e5fa3973d654ab09327bc, sha256 66b5993b2b791874ab712bf03dca4688044857f24a357fc6d2d308037e3f2896.

tools/run_uk_cut.py fetch --only nts_2002_2024 --only lcfs_2024_25 installs and verifies all seven pinned tabs from the Hub. I ran it on this branch merged with main at 4e31e42ef.

Could you pin both here rather than in a follow-up PR? In tools/uk_cut_inputs.json:

• nts_2002_2024: "revision": "559b69e94f0e03e696c5b7a55feffc54b7423717", "archive_sha256": "bc9955021d12ca82230c75626d393574dc6a44d9cbd13f412af76c5af93bf136"
• lcfs_2024_25: "revision": "1105105276991902d28e5fa3973d654ab09327bc", "archive_sha256": "66b5993b2b791874ab712bf03dca4688044857f24a357fc6d2d308037e3f2896"

The doc's inputs table can then mark both as fetched, and its LCFS row can say the zip is a repack.

Regions (Q13). Yes, a new national and dense pair on policyengine-uk 2.151.0 will be in Microcosm soon. In the meantime, we can accept the mismatch (PolicyEngine/policyengine.py#699) if we need it to test the machinery.

CI. The last run on 300ce55 was cancelled partway through, so I've re-run it. Locally, on the merge with main at 4e31e42ef, 870 tests pass: this PR's new test files plus main's signing, assembler and calibration-run tests.

[juaristi22]

Review at 300ce55. The key registry and the verifier changes look right: every UK signer now needs an active registered key, and every verifier a registered one. The CI re-run on this head has passed engine-uk, integration-uk, lint and wheels; engine-free and engine-us were still running when I posted this.

Findings, most important first:

1. The accepted-mismatch route can't certify (tools/certify_uk_cut_in_policyengine_py.py:1011). It passes only --accept-regional-model-mismatch <reason>, but PolicyEngine/policyengine.py#699 also requires --regional-input-surface-evidence, so certify-data refuses after the worktree, sync and column audit have run. Following my ruling there: once that PR measures drops against the build model, generate the evidence with its scripts/audit_regional_input_surface.py and pass the file. This tool already downloads the regional H5 and reads its columns, so that replaces input_surface_change rather than adding a step.
2. A resumed cut isn't bound to the key that signed it (tools/run_uk_cut.py:1482 and preflight's signing_key_check). With two active keys, a second key holder resuming all gets an HMAC failure at certify that preflight never warned about, and cut_receipt.json records the resumer's key rather than the signer's. Check the environment key against the fingerprint in the cut's signed reports, and take the receipt's fingerprint from those reports.
3. The one-writer check misses suffixed branches (tools/certify_uk_cut_in_policyengine_py.py:496). It looks only for certify-uk-<id>, while the hub's branch for a133f37a is certify-uk-a133f37a-national. Match any branch that starts with certify-uk-<id>.
4. all hasn't run end to end on a real cut. Spine through assembly and publication has only run against a fake executor; the hub rehearsal ran the policyengine.py handoff on my 23 Oct cut. The done-checks rely on fixture formats, such as taking the assembly summary from the last JSON object in an append-mode log. Before the team relies on it for a Budget re-cut, please run one full cut that doesn't matter.
5. The signing-key variable name is copied (tools/run_uk_cut.py:59, tools/modal_uk_cut.py:45) although this PR exports UK_GATE_SIGNING_KEY_ENV. docs/agent-guide.md asks us to import shared static data "instead of copying it".
6. Defaults specific to one machine: the reservation tool path (tools/run_uk_cut.py:66), the certifier's load limit of 110 (tools/certify_uk_cut_in_policyengine_py.py:173) and the doc's pointer to a runbook on your machine. On other machines, the reservation check is silently skipped and the load guard never trips.
7. Every certification commit credits Claude (tools/certify_uk_cut_in_policyengine_py.py:1180), whoever runs the tool.
8. A UC-floor receipt without passed crashes with a KeyError (tools/certify_uk_cut_in_policyengine_py.py:1126 and uc_floor_summary) instead of the intended "failed operationally" refusal.