Compare UK cuts with the current build of their own line by juaristi22 · Pull Request #1574 · PolicyEngine/microcosm

juaristi22 · 2026-11-10T17:08:46Z

Part of #1095.

Now that the migration in #665 is complete, a UK cut checks and certifies its candidate against the current build of its own line, as releases.json in policyengine/populace-uk-private names it, instead of the enhanced FRS: the national line against the national current build, the dense line against the dense one.

Decisions (María, 2026-11-10)

Resolve the reference live, with four safeguards. A cut resolves its reference once and keeps the record; the dense build fetches its own reference; a redundant declaration is reported, not failed; the fetch refuses a current build that is withdrawn or not certified.
The score is a report with no verdict, as the US head-to-head is. One country-neutral module (microcosm.build.head_to_head) builds both; the US tool's payload is byte-identical on 300 random fixtures.
All four enhanced-FRS checks move: the score, input-mass parity, the export surface and release input coverage.
Input-mass bounds. The tolerance is the largest drift between the last two national builds (5.998, no headroom). Because a symmetric tolerance above one fails no drop short of zero, a column may also lose at most 90% of its reference total: the same pair's largest drop (0.883), rounded up for headroom.
On the review's design points (S4, S5). A cut may pin a listed build as its reference, and each national certification records its change since a fixed baseline, since that stays light (item 9).

What changes

Reference resolution (uk_runtime/cut_reference.py, tools/resolve_uk_reference.py). microcosm-data's loader exposes the build-folder resolution; the fetch checks the national release_certification.json (shippable, names this file, registered key) or the dense gate_summary.json (every gate passed). tools/run_uk_cut.py gains a reference step before the spine, writing the dataset, reference.json, input_mass_reference.json and reference_surface.json under <cut>/reference/; --re-resolve-reference sets it aside.
- A line with no current build can pin a listed one (--reference-build, or --build on resolve_uk_reference.py). The pinned build passes the same checks as the current one, is refused if withdrawn, and is recorded as pinned in the reference record and the certification.
- From the review: the cut refuses to re-resolve to its own candidate, and the certifier refuses a reference that is the candidate. The certify step refuses a reference withdrawn since the fetch. The certifier and the dense build each refuse a reference of the other line, and resolve_uk_reference.py refuses offline mode, as the runner does.
Shared head-to-head module, which the US tool now uses.
Score and certification. The receipt is score_vs_reference.json (schema 2, no verdict). The certification and the release contract move to schema 2 with a reference block; unresolvable measures are registered per reference build.
Input mass compares each column's total with the reference's at the reference's own solved weights (the factor its mass log records). Exclusions are keyed <release>@<build>. A column one or two source households carry on either side is checked, but its figures are withheld.
Export surface and input coverage.
- The reference surface lists the columns the reference exports and the inputs it carries, by the coverage gate's own rule (non-default signal on the reviewed share of effective mass). It holds names and flags only.
- The declared extras shrink from 250 to the 32 the current national build does not export: the dense line's geography and clone columns, plus six columns new since the current builds, from UK November event data, PR2: feed re-pin, trading allowance claim status, Scottish council tax and NSND, young people and PIP rows, UC take-up age fit, PIP points, UC limited capability #1550. An entry the reference carries is reported as redundant.
- The coverage manifest moves to schema 4 and keeps only the reviewed exclusions, the effective-mass policy and the stage-family contracts.
- The certifier and dense builds take --reference-surface; the national role refuses it.
Dense reference files are read with the row-wise reader, and a published file's area-code names map back before the export boundary. Both broke on a dense file.
Disclosure. The coverage gate withholds per-column figures for a column one or two source households carry, since the required set now includes the reference's thin inputs and the report is published with the release. The target and national kernels now hash receipt_counts beside weighted_integrity, as their import walk requires.
Columns UK November event data, PR2: feed re-pin, trading allowance claim status, Scottish council tax and NSND, young people and PIP rows, UC take-up age fit, PIP points, UC limited capability #1550 changed. The six columns UK November event data, PR2: feed re-pin, trading allowance claim status, Scottish council tax and NSND, young people and PIP rows, UC take-up age fit, PIP points, UC limited capability #1550 adds are declared as new since the current builds. person.self_employment_gross_receipts, which UK November event data, PR2: feed re-pin, trading allowance claim status, Scottish council tax and NSND, young people and PIP rows, UC take-up age fit, PIP points, UC limited capability #1550 (UK trading allowance: store the claim status once policyengine-uk#2765 is released #1439) stops writing and both current builds still carry, gets three exclusions María signed on 2026-11-10:
- a reviewed export exclusion, and the release boundary drops the column;
- a coverage exclusion for inputs retired since the reference;
- a signed input-mass approval for each current build, scoped to its release line.
Each goes dormant once a promotion moves the reference past it.
Change since a fixed baseline. Every check compares a cut with the build before it, so drift could add up across cuts unseen. Each national certification now records it in a since_baseline block, which the release contract checks:
- each input column's total as a ratio of the baseline's;
- the inputs the baseline carried;
- which of those this build no longer carries.
It chains, so no baseline dataset is loaded. The reference step keeps the reference's certification and continues its record, or starts one at the reference. The input-mass gate multiplies in the ratios it already measures, and the coverage gate names the baseline's inputs the candidate no longer carries. A column one or two source households carry leaves the chain. The record carries no verdict. The national chain starts at eeb6093e. The dense line's published evidence carries none of these gates, so its chain restarts at each reference.
Docs and changelog: uk-cut-anyone.md, runbooks 623, 762 and 806, uk-staging-operations.md, gate-battery-contract.md, the README, and changelog.d/1095-certify-against-current-build.changed.md.

The battery digests and the release-cut part digests are re-pinned in each commit that moves the spec.

Testing

The 75 test files that touch the changed surfaces (engine-free and engine) and the US head-to-head tests, run in a US environment: 3,745 tests. Two failures are environmental and unrelated:
- an orrery reconstruction test that needs the @axiom-foundation/orrery package;
- test_spawned_workers_score_identically_to_sequential_cli, which needs Python 3.14.7.
  The files touched by the last fixes were run again afterwards with no failures. tools/ci_test_plan.py verify passes, and the coverage manifest regenerates unchanged.
After the review's fixes and S4 and S5, the 75 UK test files that touch the changed surfaces (65 engine-free, 10 engine) ran again at ee8fa0353: 3,587 tests, with the orrery test the only failure. Nine skip because the pinned feeds or artifacts they read are not on this machine. tools/ci_test_plan.py verify passes, and the coverage manifest regenerates unchanged.
Real builds, run locally; the outputs stay on the machine:
- Both lines resolve live.
- Against the national build before the current one, the current national build passes input mass with the drop limit and the column checks of coverage. Its export surface fails only on the columns it added since, as designed.
- The dense pair gives the same result.
- The dense reference step writes both evidence files.
- On the national pair, S5's record covers 259 columns. The coverage gate, chain included, ran in 19 s at a peak of 1.6 GB, counting the dataset load.
On this branch, the spine's declared export surface equals the current builds' except for UK November event data, PR2: feed re-pin, trading allowance claim status, Scottish council tax and NSND, young people and PIP rows, UC take-up age fit, PIP points, UC limited capability #1550's changes, which are declared here.

Before the next cut

CI early warning. The test that caught a new spine column without an extras entry needs the current build's column names, which the repository does not hold, so it now checks only that the release drops every reviewed export exclusion. A names-only snapshot refreshed at each promotion would bring it back; open question.
Run steps. tools/run_uk_cut.py's assemble, publish and promote steps still expect a cut tag the assembler no longer prints; that is a separate fix.

🤖 Generated with Claude Code

A UK cut will check and certify its candidate against the current build of its own release line, as the Hub's releases.json names it (microcosm#1095), instead of the enhanced FRS. This adds the resolution, before any check uses it. - microcosm-data's loader exposes CertifiedRelease, resolve_certified, download_certified and download_build_file: the build-folder resolution it already does for load(), returning the build's record, without enforcing the installed engine's versions, since a comparison reads a build under the engine it runs. - uk_runtime/cut_reference.py resolves the national or dense line's current build, checks its certification evidence by its fields (the fetch runs without the signing key), and keeps a strict record that later steps read instead of resolving again. - tools/resolve_uk_reference.py checks a line's current build read-only, or fetches it with its record for runs outside run_uk_cut. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Both countries score each artifact with microcosm.calibrate's canonical loss, but each wrote its own comparison on top: per target, whose absolute relative error is lower, the counts, the loss gap and the family counts. microcosm.build.head_to_head now holds that comparison once, with no verdict, as microcosm.build.input_mass did for the input-mass gate (#609). The US head-to-head scorer's _comparison_payload calls it and keeps its own payload names, so its scorecard is unchanged: on 300 random fixtures (ties, NaN, signed zeros, empty surfaces) its JSON is byte-identical to main's, and a new test pins the payload. The UK cut's report moves onto the module next (microcosm#1095). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A cut's candidate is now compared with the current build of the national line, which the Hub's releases.json names, instead of the enhanced FRS (microcosm#1095), and the comparison is evidence with no verdict, as the US head-to-head is. - run_uk_cut resolves the reference once per cut, in a new reference step before the spine, into <cut>/reference/ with its record. The national, score and certify steps read that record; none resolves again, so a promotion mid-cut cannot split it. --re-resolve-reference sets it aside. status says when the current build has moved; stage 1 compares with the reference by default. The incumbent_efrs input and its gates.json pin selector are gone. - candidate_score builds on microcosm.build.head_to_head. Its receipt, score_vs_reference.json, schema 2, names the reference and carries the losses, per-target comparisons and family counts, with no verdict. Measures the reference cannot carry are pruned under reference_unresolvable_measures.json, scoped per reference build; the enhanced FRS's eight entries retire with it. - The national role, the build CLI (--reference-h5, --reference-record) and the score command follow; the dense role refuses the national role's reference flags. - The certifier requires a schema-2 receipt for this candidate and the cut's own reference, and the certification (schema 2) names that reference. The release contract moves to schema 2 and binds score_vs_reference.json, which the assembler now publishes. Only new builds are validated against the contract, at the spec they are assembled under, so schema 1 keeps no path. The input-mass, export-surface and input-coverage gates still read the enhanced-FRS references; they move to the current build next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The input-mass gate compared every cut with the enhanced FRS's weighted totals (microcosm#1095). It now compares with the current build of the cut's own line, the build the cut's reference step resolved. - uk/gates.json names the reference kind, current_build, not a file. The report records the build (reference_label, reference_identity) and its totals digest. The tolerance is 5.9984285545209834: the exact maximum drift between the consecutive promoted national cuts a6299504 and eeb6093e over the 259 columns at least three source households carry, with no headroom, as María signed on 2026-11-10. A zeroed or missing column still fails at any tolerance. - The reference step (run_uk_cut, and tools/resolve_uk_reference.py for a dense build) writes input_mass_reference.json beside the reference: schema 2, the build's export totals and distinct source-household carriers at its own solved weights. uk_recorded_solve_year_frame gives a build back its solve by the factor its mass log records, since an older policyengine-uk's projection differs from the installed one. - A column one or two source households carry on either side is still checked, but its figures leave the report (withheld_thin_columns). - Exclusions are signed per reference build (<release>@<build>) and go dormant when a promotion moves past it. The enhanced FRS's three entries, its registry entry, the hosted totals input and their pins retire; microcosm-data's contract checks that a report names its build instead of pinning the eFRS identity and digests. Digests re-pinned. - certify refuses an input-mass reference that is not the cut's reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. The export-surface and release-input-coverage gates compared every UK cut with the frozen enhanced-FRS reference. A cut now compares with the current build of its own line, the build its reference step records. - The reference step writes reference_surface.json beside the reference: the columns the build exports and the loader inputs it carries, names and flags only. An input is carried by the coverage gate's own rule (uk_effective_input_signal: non-default signal on the reviewed share of effective mass), flagged thin when one or two source households carry it. - Export surface: the reference side is that surface. The declared extras shrink from 250 to the 26 the current national build does not export (the dense line's geography and clone columns); an entry the reference carries is reported as redundant, not failed. - Input coverage: the required inputs are the ones the reference carries, less the reviewed exclusions. The manifest moves to schema 4 and keeps the exclusions, the effective-mass policy and the stage-family contracts; the preflight stops checking the frozen enhanced-FRS surface and ledger. The gate refuses a reference measured at another floor, and an empty required set. - The certifier takes --reference-surface and refuses one that names another build than the cut's reference; run_uk_cut passes it. - Dense builds take --reference-surface as a graph source, beside --input-mass-reference, and refuse the two if they name different builds; without it the two gates are evidence_absent. The national role refuses it. - Battery digests re-pinned in contract.py and its test copies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. The operator docs described the enhanced-FRS reference, the incumbent flags and the rule-1 verdict. They now walk through the reference step, the files it writes, the --reference-* flags of the national role, the certifier and the dense build, and the score as a report with no verdict. - uk-cut-anyone.md: the reference step, its directory, and how to check or move what a cut compares with; the eFRS and input-mass input rows go. - Runbooks 806, 623 and 762: resolve the reference, then pass it to the build, the scorer, the certifier and the dense build. - uk-staging-operations.md, gate-battery-contract.md and the README: the score_vs_reference artifact, the reference_surface artifact and the current_build reference rule. - A changelog fragment for the change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. The reference step's evidence (input-mass totals and the reference surface) loaded the reference with the compact national loader and passed it through the release export boundary. Both refuse a published dense file: it carries one clone_index name on every table, and the consumers' area-code names the boundary writes. The national line never showed it, since its files carry neither. The evidence now reads the file with the row-wise reader, which handles both lines, and uk_published_release_frame maps a published file's area-code names back to the ladder's before the boundary applies today's drops and names, so the reference leaves exactly as the candidate does. A test writes a dense-format file and computes both pieces of evidence from it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. The input-coverage gate's report lists, for every required column, its signal rows and effective mass. The report is copied into the published release directory. Since the required inputs now include the ones the reference carries on one or two source households, a column's figures could rest on fewer cases than the UKDS floor of three (CD171). The gate counts the distinct source households behind each column's effective signal (household_source_id) and, for one or two, reports only "<3" and withholds the figures, in the details and in the failure message. The verdict still stands on the real values. A frame without lineage (a hand-made fixture; every built frame carries it) reports as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. The input-mass gate's tolerance is symmetric, and at the 5.998 measured between the last two national builds it failed no drop short of zero: a column could lose 99% of its total and pass. maximum_relative_drop now bounds the share of its reference total a column may lose: 0.9, the same pair's largest drop (0.8831) rounded up for headroom, as María ruled on 2026-11-10. The check sits in the UK wrapper beside the shared gate, which the US line also uses and which keeps its semantics. A reviewed exclusion covers both checks and is stale only if the column passes both; a thin column's drop fails by name only. The policy takes the limit with no default, gates.json declares it, and the release contract requires the declared value and an empty excess_drops on a passing report. Digests re-pinned. On the real pairs the gate passes with the limit: national eeb6093e against a6299504 (largest drop 0.8831), and dense 38605444 against 83802786 (largest drop 0.58). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. #1550 adds six exported columns the current builds (national eeb6093e, dense 38605444) do not export: the PIP points (microcosm#1095 P1), the trading allowance claim status and full-relief carrier (microcosm#1439) and the UC limited capability flags (microcosm#1095 Y1). Measured against those builds, the export-surface gate needs each declared as new since the reference, naming its PR; each leaves once a promotion moves the reference past it. Digests re-pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…uilds Part of #1095. #1550 (microcosm#1439) stops writing self_employment_gross_receipts: the trading allowance stage stores each trader's claim status and leaves gross receipts at the engine's zero. Both current builds (national eeb6093e, dense 38605444) still carry the receipts encoding microcosm#1428 wrote, so a cut measured against them would fail on the column in three gates. María signed the three exclusions on 2026-11-10: - export surface: a reviewed export exclusion (runtime constant and its gates.json mirror), and the release boundary drops the column, as it drops every reviewed export exclusion; - input coverage: a reviewed exclusion the manifest generator declares for inputs retired since the reference (manifest regenerated); - input mass: a signed approval for each current build, scoped to its release line, in force from 2026-11-10 to 2027-05-10. Each goes dormant once a promotion moves the reference past it. Digests re-pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. The target and national kernels hash weighted_integrity for its exclusion clock, so every import it makes must be hashed too. This branch had given it three: the reference evidence's dataset readers (rowwise_dataset, weight_year), which no target value needs, and receipt_counts, which the input-mass gate counts source households with. The reference evidence's input-mass payload moves to cut_reference, beside the writer that calls it, so weighted_integrity reaches no dataset reader. receipt_counts is hashed beside weighted_integrity in the target, problem and national gate kernels; the national gate kernel runs it. The record of unclassified imports stays as it was, since it may only shrink. The input-mass register test now expects the two signed entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vahid-ahmadi · 2026-11-10T17:21:50Z

Review of 8ea0ee1: I traced how the reference is resolved, how the cut resumes, the dense and national roles, and the certifier, and ran the PR's engine-free tests against the stubbed Hub (test_uk_cut_reference, test_resolve_uk_reference, test_run_uk_cut, test_head_to_head, test_uk_reference_surface, the loader, release certification, weighted integrity, the certify command and candidate score: all pass, with 3 skips; no Hub fetches, no builds).

The four safeguards, and where each is enforced:

Safeguard	Enforced at	Holds?
Resolve once, keep the record	`cut_reference.py:457` refuses a directory that already holds a record; `run_uk_cut.py:1529-1590` reuses the record, re-verifies the sha256 at each use (`reference_done`, `reference_arguments`, `score_uk_national_candidate.py:62-63`), and `score_done` / `certify_done` / the certifier all require the receipt and the certification to name the same `release, build, sha256`	Yes, with the gaps in S1 and S2
Dense build fetches its own reference	`tools/resolve_uk_reference.py`; `graph_terminal.py:654` checks that the mass file and the surface name one build	Partly (S3)
Redundant declaration reported, not failed	`terminal_gates.py:845` `redundant_allowed_extra_columns`	Yes
Withdrawn or uncertified current build refused	`releases_record.py:152` refuses a withdrawn `current`; `cut_reference.py:244-263` (national: shippable, names this sha256, registered key); `cut_reference.py:266-284` (dense: every gate passed); `loader.py` checks the dataset and the evidence sha256 against the manifest	Yes at fetch time, but not afterwards (S2)

A releases.json that is missing falls back to the pointer file, and _require_build_folder then refuses (cut_reference.py:346-351). A dataset whose sha256 does not match is refused twice: by the loader against the manifest, then by verify_uk_cut_reference_file on the copy. Resolving the reference only reads from the Hub (hub_download calls only; nothing in cut_reference.py, the resolve tool or the new loader functions writes). The write-access checks in run_uk_cut.py preflight are the existing upload checks, not part of resolving the reference.

Should

--re-resolve-reference can make a cut compare with itself (run_uk_cut.py:1545-1578). Nothing checks that the reference differs from the candidate. Case: a cut is assembled, published and promoted. The operator then reruns all --re-resolve-reference (once the promote step works again, or after promoting by hand), for example to regenerate the certification after an exclusion renewal. The national line's current build is now this candidate, so reference.sha256 == candidate_sha256. The score shows zero difference, input mass shows zero drift, the surface matches itself, and the certification comes out shippable against itself. The same happens on the dense line if resolve_uk_reference.py runs after the candidate's own promotion. Fix: in run_reference, and in compose_uk_release_certification, refuse when reference.sha256 equals the candidate's sha256, or when reference.build is the cut's own build id.
A reference withdrawn mid-run still certifies (release_certification.py, compose_uk_release_certification; run_uk_cut.py:944 certify_done). Withdrawal is checked only when the reference is fetched. A build is usually withdrawn because something is wrong with it, and in that case a cut that resolved it earlier will still certify against it. Case: the cut resolves national build B at 09:00. B is withdrawn at 14:00 and releases.json moves current to null or to C. The certify step at 16:00 reads only the recorded B, and nothing in the certification shows that B had been withdrawn by then. A read-only re-check keeps the "resolve once" rule intact: at certify time, read releases.json and refuse if the recorded build is now marked withdrawn. Do not switch to the new current build; --re-resolve-reference stays the operator's move. status (_current_build_note) could report the same thing.
The dense role does not check that its reference is a dense build (graph_terminal.py:654-670, full_build_cli.py:1476-1486). _require_one_reference only checks that the two files name the same build. Case: an operator runs resolve_uk_reference.py --release microcosm-uk-2024-25 (the national release) and passes its input_mass_reference.json and reference_surface.json to a dense build. The labels agree with each other, so the dense line's export surface and coverage are compared with the national build. The body says this mixing must not happen. The national side is fine, because run_uk_cut.py always resolves RELEASE_ID and rowwise_cli.py:1206 refuses --reference-surface. Neither the surface loader (reference_surface.py:169-172) nor UKCurrentBuildReference.descriptor_for (weighted_integrity.py:282-298) checks more than the label's <release>@<build> shape; descriptor_for also takes its totals_sha256 from the file it is checking, so that check can't fail on a file from the wrong line. The same applies to a stale releases.json: run_uk_cut.py refuses offline mode, but resolve_uk_reference.py has no hub_offline() guard, so with HF_HUB_OFFLINE=1 it resolves whatever current the cached releases.json names. Fix: in the dense terminal phase, refuse unless mass.label starts with UK_DENSE_RELEASE_ID + "@". Make the mirror check in certify_uk_release_cut.py's _input_mass_reference (national only), and refuse offline mode in resolve_uk_reference.py as run_uk_cut.py does.
No first-cut path on a line with no current build (loader.py _resolve_build_folder_release, "has no current build; pass build="; cut_reference.py:70-74). The error message suggests build=, but neither run_uk_cut.py nor resolve_uk_reference.py has a way to pin a build. Case: a new release line, or a line whose only current build has just been withdrawn (current: null). Every cut is refused until someone edits releases.json by hand. _REGISTRY_YEAR = 2025 and the two hard-coded release ids mean the next FRS year also needs a code change. If this is intended, say so in uk-cut-anyone.md. Otherwise add an explicit, recorded --reference-build pin that is still certification-checked, or a declared bootstrap posture.
Design point, not a ruling: no fixed anchor on drift between cuts. Input mass (gates.json uk_input_mass_parity: tolerance 5.998, maximum_relative_drop 0.9), the export surface and input coverage now all compare with the predecessor, and the score has no verdict (release_certification.py, _verify_score_receipt). Under the old rule 1 (US base v2: one CPS+ACS+PUF-detail pool; datasets labeled by exact record count (dense = full pool; exact-k L0 selection) #578), a candidate's loss had to beat a fixed enhanced FRS; nothing in this PR replaces that. Each cut is checked only against the cut before it, so drift can build up. For example, an untargeted column can lose up to 90% of its total in each of two consecutive cuts and pass both, ending at 1% of where it started. It can also grow by up to 7× per cut. An input the reference has stopped carrying also stops being required by coverage, so one cut's drop is accepted from then on. Calibration targets and the per-target loss cap still anchor the targeted measures, but untargeted mass and coverage now have no fixed bound at all. Options: keep one frozen anchor (the last certified build before this PR, or a pinned input_mass_reference digest) as a second, wider report or gate; or record the cumulative ratio against a fixed baseline in each certification so a slow decline is visible. This is your call. I am raising it because the PR does not say whether it was considered.

Nit

Leftover enhanced FRS names. full_build_cli.py:3295 still writes "score_vs_enhanced_frs": None into the build block of the national seam's calibration_diagnostics.json. docs/uk-full-build-graph.md:23 (not in the PR's doc list) still says the dense role refuses --incumbent-h5 and --incumbent-sha256, flags that no longer exist; it should say --reference-h5/--reference-record (rowwise_cli.py:1256). Line 68 of that file still refers to the end-of-build "incumbent evaluation". UKInputMassParityPolicy's docstring (weighted_integrity.py, around line 1003) still says UK values must come from "the pinned eFRS incumbent", and the module docstring's gate description (line 9) still says "a frozen reference". Both now contradict the current_build reference. The incumbent_efrs and input_mass_reference inputs are gone from uk_cut_inputs.json, and test_run_uk_cut.py:1388 checks that, which is good.
The dense line's local score is still against the uk-data incumbent area weights (score_vs_incumbent.json, contract.py:899; runbook 762 line 476 onwards). That looks intentional, since it is a different comparison, but the body says UK cuts now compare with the current build. One line saying the local score is unchanged would prevent a misreading.
After --re-resolve-reference, national_done (run_uk_cut.py:870) does not look at the reference. The national step is skipped, so build_record.json's reference and the in-build evaluation still name the old build, while cut_receipt.json and the certification name the new one. The certification is internally consistent. Either note in the receipt that the build-time evaluation is superseded, or have the receipt carry both.

Checked and fine

sha256 mismatch on the dataset and on the evidence (loader.py download_build_file and _verified_download), and the national certification's candidate.sha256 checked against the dataset.
The resume paths in run_reference. An unusable reference directory is refused until --re-resolve-reference. Missing evidence is rewritten from the same verified file and never re-resolved. The old directory is renamed aside, not deleted.
The certifier refuses an input-mass file or surface whose label or sha256 differs from the reference record (certify_uk_release_cut.py:397-432), and refuses a score receipt that names another reference.
Input-mass exclusions are keyed <release>@<build> and filtered by line. The two self_employment_gross_receipts entries are each scoped to their own line and go dormant after a promotion.
The ad hoc score mode (--reference-sha256/--reference-label) writes no reference record, so the certifier cannot accept its receipt.

CI at 8ea0ee1: 5 checks pass; the engine-free, engine-uk and engine-us jobs (3.13 and 3.14) are still pending.

Verdict: the record-once design holds and the fetch-time checks are sound. S1 to S3 are small guards worth adding before the next cut. S4 and S5 are scope questions for you.

🤖 Generated with Claude Code

Part of #1095. From the review of #1574. - S1: a cut cannot compare with itself. After the cut's own promotion the line's current build is the candidate, so run_uk_cut refuses to re-resolve to it (checked before the recorded reference moves), and the certifier refuses a reference whose sha256 or build is the candidate's. - S2: a reference withdrawn after the fetch stops the certify step. The check reads releases.json and never moves the cut to another build; --re-resolve-reference stays the operator's move. status reports it too. - S3: each line compares with its own. The dense build refuses an input-mass reference or surface that is not a dense build's; the certifier refuses a reference record that is not the national line's; resolve_uk_reference.py refuses offline mode, as run_uk_cut does, since a cached releases.json can name a build that is no longer current. - N6: the diagnostics build block no longer carries score_vs_enhanced_frs; uk-full-build-graph.md and weighted_integrity's docstrings name the current-build reference. - N7: runbook 762 says the dense line's local score is unchanged. - N8: the cut receipt records the national build's own comparison and whether --re-resolve-reference superseded it. - S4: uk-cut-anyone.md says a line with no current build cannot be cut yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. From the review of #1574 (S4), on María's ruling: a line with no current build (a new line, or one whose only current build was withdrawn) could not be cut at all. - resolve_uk_cut_reference, fetch_uk_cut_reference and check_uk_cut_reference take a build= pin. A pinned build passes the same certification checks as the current one, is refused if withdrawn, and is recorded as pinned: the reference record gains selection (current or pinned), and the certification's reference block carries it, which the release contract checks. - tools/run_uk_cut.py takes --reference-build; a cut that already compares with another build keeps it unless --re-resolve-reference is given too. status says when the reference is pinned. resolve_uk_reference.py takes --build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. From the review of #1574 (S5), on María's ruling: every check compares a cut with the build before it, so drift could add up across cuts unseen. Each national certification now records its change since a fixed baseline in a since_baseline block: each input column's total as a ratio of the baseline's, the inputs the baseline carried, and which of them this build no longer carries. It stays light by chaining: - the reference step keeps the reference's own certification beside it and continues its record, or starts one at the reference when it has none, for the columns at least three source households carry; - the input-mass gate multiplies in the ratios it already measures, and the coverage gate names the baseline's inputs the candidate no longer carries, by its own rule; - the certifier copies both into the certification, which the release contract checks. No baseline dataset is ever loaded. On the real national pair the input-mass side is arithmetic over 259 columns, and the coverage side adds one pass over the candidate's columns inside a gate that already loads them. The record carries no verdict. The dense line's published evidence carries none of these gates, so its chain restarts at each reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. The fragment still said 26, from before the six columns microcosm#1550 adds were declared; gates.json declares 32. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

juaristi22 · 2026-11-10T19:16:15Z

Thanks, Vahid. Fixed in 555f76e, with a test for each:

S1. run_uk_cut.py refuses to re-resolve to the cut's own candidate. It checks before the recorded reference moves, so nothing is set aside. The certifier refuses a reference whose sha256 or build is the candidate's.
S2. The certify step re-reads releases.json and refuses a recorded build that has been withdrawn since the fetch. It never moves the cut to another build: --re-resolve-reference stays the operator's move. status reports the withdrawal too.
S3. The dense build refuses an input-mass reference or a surface that is not a dense build's. The certifier refuses a reference record of another line. resolve_uk_reference.py refuses offline mode, as run_uk_cut.py does.
N6. The diagnostics build block no longer carries score_vs_enhanced_frs. uk-full-build-graph.md and the weighted_integrity docstrings now name the current-build reference.
N7. Runbook 762 now says the dense line's local score compares with the uk-data incumbent's area weights, unchanged.
N8. The cut receipt records the national build's own comparison and whether --re-resolve-reference superseded it.

On María's rulings:

S4. A cut can pin a listed build: --reference-build on run_uk_cut.py, --build on resolve_uk_reference.py (ad18f8d). A pinned build passes the same certification checks as the current one and is refused if withdrawn. The record and the certification's reference block mark it pinned. A cut that already compares with another build keeps it unless --re-resolve-reference is given too. The release ids and registry year remain code constants, which uk-cut-anyone.md says.
S5. Each national certification records its change since a fixed baseline, in a since_baseline block (ee8fa03):
- each input column's total as a ratio of the baseline's;
- the inputs the baseline carried;
- which of those the build no longer carries.
It chains: the reference step keeps the reference's certification and continues its record, the input-mass gate multiplies in the ratios it already measures, and the coverage gate names the baseline inputs no longer carried. So no baseline dataset is loaded; on the national pair it adds seconds. The chain starts at eeb6093e. It is a record with no verdict. The dense line's published evidence carries none of these gates, so its chain restarts at each reference.

vahid-ahmadi · 2026-11-11T10:17:09Z

Round 2 at 2d3ab8b

I diffed 8ea0ee1..2d3ab8b and ran the ten affected engine-free test files at the head with the Hub stubbed: 595 pass, none skipped. I then ran the 17 new or extended tests against the 8ea0ee1 source. All 17 fail or error there. The four new test_run_uk_cut tests error at setup on the old code, because their autouse fixture patches a function the old code doesn't have, so for those four the evidence is that they pass at the head.

S1 fixed. run_uk_cut.py:1557-1572 refuses before the recorded reference moves. release_certification.py:491-499 refuses a reference whose sha256 or build is the candidate's (test_re_resolving_refuses_the_cut_s_own_candidate, test_compose_refuses_a_reference_that_is_the_candidate_itself). The dense half of this can't happen in the normal flow: the dense gates run only inside the build (full_build_cli.py:1825-1826), before the candidate can be promoted.
S2 fixed for the certify step. run_uk_cut.py:1704-1711 re-reads releases.json without moving the cut (cut_reference.py:335), and status reports a withdrawal (test_a_reference_withdrawn_since_the_fetch_stops_certification, test_a_build_withdrawn_after_the_fetch_no_longer_stands). There is one remaining gap, in new item 3.
S3 partly fixed: two of the three points are done. The line is now checked: graph_terminal.py:666-675 for the dense role, and certify_uk_release_cut.py:398-413 and release_certification.py:486-490 for the national one. resolve_uk_reference.py:99-109 refuses offline mode. The extended test_the_dense_gates_refuse_references_of_two_builds fails on the old graph_terminal.py. descriptor_for (weighted_integrity.py:285-299) still takes totals_sha256 from the file it checks. Now that the line is checked, I am not pressing that point.
S4 fixed. --reference-build / --build pins a listed build (cut_reference.py:438-501). The pinned build is certification-checked, refused if withdrawn, and recorded as pinned. An existing record moves only with --re-resolve-reference (run_uk_cut.py:1549-1556). uk-cut-anyone.md states that the release ids and the year are constants.
S5 partly fixed. Each national certification now has a cumulative record, since_baseline (release_certification.py:408-448, :598), chained from eeb6093e. It has no bound, and it has two gaps: new items 1 and 2.
N6 fixed. full_build_cli.py:3292, uk-full-build-graph.md:23, and the weighted_integrity.py docstrings at lines 9-12 and 1045-1047.
N7 fixed. uk-dense-release-assembly-runbook-762.md:476.
N8 fixed. run_uk_cut.py:1858-1880 writes build_time_evaluation: {reference, superseded} into the receipt, from the manifest's evaluation.reference (full_build_cli.py:3589).

Should

A column that leaves the drift record leaves no trace. _since_baseline (weighted_integrity.py:1260-1286) drops a column when either side has one or two carriers, when the reference total is zero, or when the column is absent. The certification lists only the ratios that remain. I checked this with a direct call. The baseline ratios were a=0.2, b=0.2 and c=0.5. The cut took a from 100 to 20, b from 100 to 10 on 2 carriers, and c from 0 to 5. The result is {'a': 0.04}. Column a is down 96% over two cuts, and each cut passed the 0.9 drop limit on its own. Column b fell furthest, and it is gone from the record. A column driven to zero under a reviewed exclusion, as the UK November event data, PR2: feed re-pin, trading allowance claim status, Scottish council tax and NSND, young people and PIP rows, UC take-up age fit, PIP points, UC limited capability #1550 gross receipts are, records 0 once. At the next cut its reference total is zero, so it leaves too. The docs mention the thin case (uk-cut-anyone.md:296-298), but the certification doesn't. Fix: carry left_chain: {column: reason, last ratio} forward in since_baseline, so a column that leaves stays listed.
The chain can restart without trace. uk_cut_reference_chain (cut_reference.py:668-714) makes the reference itself the baseline whenever reference_evidence.json has no since_baseline. That happens with a pinned build certified before this PR, or with a cut directory resolved before ee8fa03 whose evidence is rewritten on resume. _keep_evidence runs only at fetch, so a rewrite finds no copy. Nothing compares the new baseline with eeb6093e, so a restart shows only as a different baseline label. Fix: on the national line, refuse a restart, or record restarted_from in the certification.

Nit

The withdrawal check is skipped once the certification exists. In run_certify, the certify_done early return (run_uk_cut.py:1700-1703) comes before the check, and run_publish doesn't check either. If the reference is withdrawn after the cut is certified but before it is published, a resumed all still publishes. Fix: move the check above the early return, or repeat it in run_publish.

Not a finding: the record has no threshold, so it shows drift without bounding it. The dense line has no chain at all (uk-cut-anyone.md says so). Whether either needs a bound, set at the measured edge like the 5.998 tolerance, is for you and Vahid to decide.

CI at 2d3ab8b: all 12 checks pass.

Verdict: S1, S2, S4 and N6 to N8 are fixed, each with a test. S3 is fixed in the parts that matter. S5 now has the record, which should list the columns that leave it and flag a restart before the chain starts at the next promotion.

🤖 Generated with Claude Code

… restart Part of #1095. From round 2 of the review of #1574. - A column that leaves the record of change since the baseline (absent on a side, a zero reference total, or one or two source households on a side) stays listed in input_mass_left, with its reason, its last ratio and the last build that carried it, and the list carries forward from cut to cut. The last ratio is one the reference's certification already recorded, so the list adds no figure. - The chain no longer restarts without a trace. It refuses a missing copy of the reference's evidence instead of reading it as a reference with no chain, and a resumed cut keeps that copy again from the Hub. A national chain whose baseline is not eeb6093e has restarted, and the certification records restarted_from, which the release contract checks. - A reference withdrawn after the cut is certified now stops the cut: the certify step checks before its early return, and the publish step checks too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

juaristi22 · 2026-11-11T10:53:56Z

Thanks, Vahid. Fixed in 8247032, with tests:

1. Columns that leave the record. A column that leaves the chain now stays listed, in a new input_mass_left block of since_baseline. Each entry gives its reason (absent, zero_reference_total or thin), its last ratio and the last build at which the chain recorded it. The list carries forward from cut to cut, so the column stays listed in every later certification. test_a_column_that_leaves_the_chain_stays_listed runs your example over three cuts. b and c are listed after the first. a, carried to zero at the second, is listed after the third. The last ratio is one the reference's certification already recorded, so the list adds no figure.
2. Restarts. The resume case no longer restarts the chain. A missing reference_evidence.json is refused rather than read as a reference with no chain, and run_reference keeps the copy again from the Hub, sha-checked, before it rewrites the evidence. A pinned build certified before the record still starts a chain. But whenever the baseline is not eeb6093e, the certification now records restarted_from with the national origin, and the release contract checks it. I chose recording over refusing so that a pin of an older build still cuts, which is what S4 is for. The origin is a code constant beside the release ids. If another national build is promoted before this merges, the constant moves with it.
3. Withdrawal after certification. run_certify now checks before its early return, and run_publish checks too. A resumed all stops at certify, and a lone publish stops as well.

On the bound: the record still carries no verdict. Whether it needs one is still open between us.

Part of #1095. Main moved the lock to policyengine-uk 2.177.7 (#1572) and signed person.trading_allowance_full_relief as unresolvable on the enhanced FRS in incumbent_unresolvable_measures.json, which this branch retires: the score compares with the current build, whose register is reference_unresolvable_measures.json. The retired register stays deleted, and its test now counts nine entries. Rule 1's probe bound from #1572 (three probes per signed measure, and the clean probe) carries over to prune_reference_unresolvable, with its tests on this branch's names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Part of #1095. María's ruling of 2026-11-11, after merging #1572. Five SPI rows filter on person.trading_allowance_full_relief since #1439. The current national build (eeb6093e) exports neither the claim status nor the carrier, both new since it, and policyengine-uk 2.177.7 does not define the carrier. #1572 signed the measure on the enhanced FRS, in the register this branch retires; the comparison with the current build needs the same entry for that build, or its score refuses those rows. The entry runs from 2026-11-11 to 2027-02-11, three months as #1572's does, and goes dormant once a promotion moves the reference past eeb6093e. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vahid-ahmadi · 2026-11-11T12:08:27Z

Round 3 at 23de0f3

I diffed 2d3ab8b..8247032 and ran the eight changed test files at 8247032 with the Hub stubbed: 600 pass. With those tests on the 2d3ab8b source, all 12 new or extended tests fail. 79ccc4b only merges main after #1572. In the files cited below, the merge changes only one evidence pin in contract.py:827, and 23de0f3 only signs the trading allowance carrier as unresolvable (the measures JSON, its score test and the changelog), so the line numbers hold.

Fixed. weighted_integrity.py:1299-1347 keeps a leaving column in left, with its reason, last ratio and last build, and carries the list forward. contract.py:1072-1081 checks it (test_a_column_that_leaves_the_chain_stays_listed). I re-ran my round 2 direct call: it now gives ratios {a: 0.04} and left {b: thin, 0.2; c: zero_reference_total, 0.5}. Once a's reference total is zero at the following cut, a joins left and b and c stay. b's last ratio is the reference's 0.2, not its fall at this cut, because its figures are withheld.
Fixed. A missing reference_evidence.json is now refused (cut_reference.py:711-716). A resumed cut fetches its reference's evidence again from the Hub before it rewrites it (run_uk_cut.py:1592-1610, test_a_resumed_cut_keeps_its_reference_s_own_evidence_again). A chain with a baseline other than eeb6093e records restarted_from (release_certification.py:451-455), and the contract requires it (contract.py:1092-1104, test_a_chain_that_restarted_says_so). Recording a restart rather than refusing it keeps S4's pins usable, which is a reasonable choice.
Fixed. _refuse_a_withdrawn_reference runs before the certify_done early return (run_uk_cut.py:1737) and again in run_publish (:1868), with test_a_reference_withdrawn_after_certification_stops_the_cut.
S3 remainder: unchanged, not pressed.

Follow-up, not a blocker: I'd set a bound at the measured edge once a cut or two of the record exist. The dense line having no record is fine for now if #1095 tracks it.

CI at 23de0f3: lint, wheels, select-countries, integration-uk and installed-emitter pass; engine-free, engine-uk and engine-us (3.13 and 3.14) were still running after 45 minutes, so I will check them again later.

All three are fixed, each with a test, and I have no new findings. Thanks, María.

🤖 Generated with Claude Code

juaristi22 · 2026-11-11T12:11:02Z

Thanks, Vahid. The two follow-ups, a bound at the measured edge once a cut or two have recorded the change and the dense line's record, are now tracked in #1095: #1095 (comment).

vahid-ahmadi · 2026-11-11T13:21:05Z

CI at 23de0f3 has now finished: all 12 checks pass, including engine-free, engine-uk and engine-us on 3.13 and 3.14. Thanks for tracking the two follow-ups in #1095.

🤖 Generated with Claude Code

juaristi22 and others added 12 commits November 10, 2026 15:41

juaristi22 and others added 4 commits November 10, 2026 17:35

Count #1550's six columns among the declared extras in the changelog

2d3ab8b

Part of #1095. The fragment still said 26, from before the six columns microcosm#1550 adds were declared; gates.json declares 32. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

juaristi22 marked this pull request as ready for review November 11, 2026 10:55

juaristi22 and others added 2 commits November 11, 2026 11:20

juaristi22 mentioned this pull request Nov 11, 2026

UK: every open data item: uk-data parity, engine locks, the local surface, consumption and high-value homes #1095

Open

juaristi22 merged commit 45fcf55 into main Nov 11, 2026
12 checks passed

MaxGhenis mentioned this pull request Nov 11, 2026

Make the UK eFRS parity re-pin one reviewed command; hold every identity mirror in lockstep (companion to #747) #749

Closed

Navigation Menu

Repository navigation

Compare UK cuts with the current build of their own line - #1574

Compare UK cuts with the current build of their own line#1574
juaristi22 merged 19 commits into
mainfrom
uk-certify-current-build-1095

juaristi22 commented Nov 10, 2026 •

edited

Loading

Uh oh!

vahid-ahmadi commented Nov 10, 2026

Uh oh!

juaristi22 commented Nov 10, 2026

Uh oh!

vahid-ahmadi commented Nov 11, 2026

Uh oh!

juaristi22 commented Nov 11, 2026

Uh oh!

vahid-ahmadi commented Nov 11, 2026

Uh oh!

juaristi22 commented Nov 11, 2026

Uh oh!

vahid-ahmadi commented Nov 11, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

juaristi22 commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Decisions (María, 2026-11-10)

What changes

Testing

Before the next cut

Uh oh!

vahid-ahmadi commented Nov 10, 2026

Uh oh!

juaristi22 commented Nov 10, 2026

Uh oh!

vahid-ahmadi commented Nov 11, 2026

Uh oh!

juaristi22 commented Nov 11, 2026

Uh oh!

vahid-ahmadi commented Nov 11, 2026

Uh oh!

juaristi22 commented Nov 11, 2026

Uh oh!

vahid-ahmadi commented Nov 11, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

juaristi22 commented Nov 10, 2026 •

edited

Loading