[juaristi22]

First of two PRs for #1278: the items that are ready before the pre-Budget local build. The November PR (age universe, band H after #1270, support-floor re-sign, representativeness diagnostic) and the follow-ups with outside triggers are listed in the issue. Part of #1278.

What changes

• Council-tax bands stay inside their nation (C1). The SPI housing shell drew council_tax_band from one GB-wide classifier, so SPI-channel households in England and Scotland could hold band I, and Northern Ireland households a band other than the FRS placeholder D. The shell now redraws only a band outside its nation's bands (England A-H, Wales A-I, Scotland A-H, Northern Ireland D), from a softmax of the classifier's raw class scores over those bands, with its own identity-keyed uniform. Valid draws never move. A new release-blocking spine gate, uk_council_tax_band_domain, checks every household, and a terminal instance, uk_council_tax_band_domain_terminal, checks the frame the release ships (after Place high-value homes where the Land Registry sees them (#1240) #1270's band redraw); the shared enum_domain_gate takes optional per-partition domains. The bands live in one module, uk_runtime/council_tax_constants.py.
• Both small-cell rules signed (C2). Approved by juaristi22 on 2026-10-14, expiring 2027-04-01. The local references now defer 16 council-tax band cells of fewer than 50 dwellings and 192 higher rate taxpayer cells HMRC publishes at 5,000 or fewer. Active local references fall from 24,477 on main (with Place high-value homes where the Land Registry sees them (#1240) #1270's high-value homes cells) to 24,269; the scorer's frozen count, the battery pin, the runbook and the local compile-parity receipt move with them. A run with its own ladder now places a small-cell deferral as it places an off-surface cell: a deferral whose English area the ladder cannot place is left out and named in the receipt when the membership binds cells for the area. While the rules were unsigned no cell took this path.
• Local binding adjudications renewed (C3). All six, on 2026-10-14, to 2027-01-14. Four reasons are re-worded where the basis moved. The region band-H rows keep their 2026-11-26 date.
• Unsupported-area flag (C4). support_below_floor is back in the area-support table. The dense assembler writes an area_support block to release_manifest.json, with area codes only, and the data contract checks it.
• Parity key (C5). _local_contract_covers keys the local council-tax rows on the mhclg, welshgov and scotgov prefixes, so they credit uk-data's la_council_tax.
• Local export test (C6). A test that the local export drops the national boundary's columns, incapacity_benefit_reported among them.
• Census sizing of the age-universe change (C7). tools/uk_target_surface_receipt.py --communal-only.
• CGT stage residual script (C8). docs/evidence/uk-local-surface-1278/scripts/cgt_stage_residual.py measures the CGT stage's residual at design weights against the stage's own folded HMRC margins (microcosm#725). A group pools with its neighbour until it rests on 30 or more distinct source households and its weighted gains pass two dominance rules (the largest source household at most half; the p% rule at p = 10). It has been run on main's spine; its results are not in this PR.
• Receipts (C9): experiments/1278-uk-local-surface-receipts.md, with the scripts in docs/evidence/uk-local-surface-1278/.

Ruling needed before the November PR

Under one private-household universe, about 21 constituency cells and 5 local-authority cells would sit 25% or more below uk-data's total-resident age rows by construction, and the incumbent evaluation gates those rows at 25% (C7). The November PR needs a ruling on the basis that evaluation compares.

The census facts the November change needs are requested in PolicyEngine/chronicle#358.

Tests

Local runs on the head (426ac3e, rebased onto main with #1270 and the 735820c feed):

• the tested merge of the branch and main: 1,150 targeted tests passed (gate pins, contract mirrors, small-cell rules, battery bindings, the H2 fixture, the full target graph, release certification); 5 skip without the feed;
• ruff check ., ruff format --check on the changed files, tools/ci_test_plan.py verify and tools/graph_acceptance_burndown.py --verify: clean;
• the full engine-free lane, the engine-uk files that touch the changed modules and the feed-gated compiles on 735820c: running; results follow in the PR thread;
• before the rebases, at the first head: the engine-free lane ran 17,496 tests, and five failed here for this machine's reasons and failed the same way on main; the engine-uk files that touch the changed modules passed 114 of 114;
• C1 on twin licensed spines before the rebases (main 121fd4d and the branch's C1 commits on it): every band that moved was outside its nation on main.

Review round 1

Vahid's review (all seven findings, each in its own commit): the two support-floor exclusion reasons and the generated deferral rationale drop their per-area support figures; the terminal band gate; the dominance rules in the CGT script; the parity receipt names the small-cell rule on its deferred cells; Northern Ireland FRS households take the placeholder band D whatever ctband holds; and a note that the redraw firings are exact counts. The policyengine.py consumer of the support flag is a separate issue.

Disclosure

The receipts give no counts or weighted figures from the licensed spines; census and publisher figures appear as published. Weighted band totals before and after the C1 fix are not reported, because their difference would isolate the moved households.

🤖 Generated with Claude Code

[vahid-ahmadi]

Fine once the two support-floor exclusion reasons drop their support figures (finding 1). The rest are should and nit.

@juaristi22, I reviewed the PR at c9bc232 against main at 121fd4d. C1 is correct as far as I can see:

• a valid draw never moves;
• an invalid draw is redrawn from a softmax over the nation's bands only, using its own identity-keyed salt (:nation);
• so each nation's band ends up distributed as the classifier conditioned on that nation's bands;
• Wales keeps band I, and Northern Ireland can only land on D.

Moving to raw scores to avoid the underflow is the right fix, and the test that compares the softmax with predict_proba covers it.

The −9,779 lines are regenerated register content:

• the 208 newly deferred cells leave local_target_references.json and reappear as deferrals in the membership;
• the parity receipt loses the 207 ledger-only rows those cells had.

Nothing hand-written or still needed is removed.

The renewals are real renewals, not just new dates. Each one has a fresh approved_on, a three-month expiry and a stated reason why its basis still holds, and four reasons are re-worded.

1. must packages/microcosm-build/src/microcosm/build/uk/local_area_support_exclusions.json:6 and :13. The two micro-authority reasons still give per-area effective sample sizes and positive-weight row counts. This text predates the PR and already appears in the uk-dataset-evaluation reports, but the PR rewrites both lines. While they are open, drop those figures and say only which floor each area misses (the 50-row / ESS-50 / 50-source wording is enough).

2. should packages/microcosm-data/src/microcosm/data/contract.py:564, packages/microcosm-build/src/microcosm/build/uk_runtime/calibration_run.py:135. uk_council_tax_band_domain runs only at the transferred boundary. Place high-value homes where the Land Registry sees them (#1240) #1270's uk.full.property.band node redraws council_tax_band for England and Wales owner clones after geography derivation, which is after that boundary, so the new gate will never see those draws. The VOA LSOA mixes should hold only each nation's bands, but nothing checks that. Either add a terminal-phase instance of the same gate, or have Place high-value homes where the Land Registry sees them (#1240) #1270 check its band mix against UK_COUNCIL_TAX_BANDS_BY_NATION when it loads it.

   Separately, on merge order: both PRs regenerate local_target_references.json, the membership, the local parity receipt and uk_data_target_parity.json, and both edit local_binding_adjudications.json and the contract digests. Whichever lands second should regenerate from the pinned feed rather than merge the JSON by hand. The 21,596 and 1,790 pins will move with it.

3. should docs/evidence/uk-local-surface-1278/scripts/cgt_stage_residual.py:37 and :117. The script pools groups until each rests on 30 distinct source households, but it writes weighted gains sums with no dominance check. CGT gains are concentrated enough that a group of 30 can be mostly one household. Before any of its output leaves the licensed environment, add a top-share (or p%) check on each group's gains and pool or withhold a group that fails it.

4. should (follow-up) packages/microcosm-build/src/microcosm/build/uk_runtime/graph_terminal.py:817, tools/assemble_uk_dense_release_dir.py:232. Nothing downstream reads support_below_floor or the release manifest's area_support block yet. I found no use in policyengine.py or the dashboards. py's disclosure layer computes its own per-cell support from source households and does not read this block. As a result, an area that sits below the microcosm area floor could still pass py's cell rule and be published. Whether py should withhold or mark the listed areas is a publication-policy decision, not a code one. It may be worth an issue so the flag has a consumer.

5. nit packages/microcosm-build/src/microcosm/build/uk/ledger_compile_parity_local_incumbent_2025_signed_differences.json:21335. The one newly deferred cell that has an incumbent row (band G, E09000002) carries the generic taxbase / band-H reason. It should name council_tax_band_household_floor.

6. nit packages/microcosm-build/src/microcosm/build/uk_runtime/spi_housing_shell.py:267. The stage firings council_tax_band:redrawn:<nation> are exact counts at spine level, where rows are source households. The Northern Ireland count in particular could be small. These look internal for now, but if stage firings ever go into a published gate report, band them.

7. nit packages/microcosm-build/src/microcosm/build/uk_runtime/frs_spine.py:116. The comment says every Northern Ireland household takes the placeholder band, but the code only maps ctband codes, so it relies on the FRS leaving ctband unset for Northern Ireland. The new gate would catch any case where it does not. Either force D for Northern Ireland or soften the comment.

What I checked

• Read the full PR body, UK local surface, consumption and high-value homes: remaining items (consolidation) #1278, every commit message and the source diff. Compared the overlapping files with Place high-value homes where the Land Registry sees them (#1240) #1270's head.

• Ran all 14 touched test files: 925 passed. Also ran ruff check ., ruff format --check on the changed Python files, tools/ci_test_plan.py verify and tools/graph_acceptance_burndown.py --verify, all clean.

• Reverted each source change to main and re-ran its tests. The tests fail for:

  • the off-surface deferral placement;
  • the rule signatures;
  • the support flag in the final gate kernel;
  • the release-contract area_support check;
  • the C5 parity key (the register currency test).

  For the SPI shell and gates.py, the tests error on import (the new functions are missing) rather than failing on an assertion.

• Disclosure: I checked the receipts, scripts, changelog fragments and the PR body. They give no counts or weighted figures from the licensed spines. The census and publisher numbers are public. compare_twin_bands.py writes only banded counts. The only issue is finding 1.

• Not checked:

  • the engine-uk tests;
  • the licensed twin-spine comparison, including the claim that every moved band was outside its nation on main;
  • the feed-gated compiles on the 942a1fa feed.

🤖 Generated with Claude Code

[juaristi22]

Thanks @vahid-ahmadi. Rebased onto main at 412f916, which brings #1271 (the policyengine-uk 2.130.1 lock) and #1281 (the QRF tail gate removed). Five files conflicted: the imports in frs_spine.py (both kept), and digest pins and generated files. The gate digests, the H2 spine fixture and the coverage manifest are regenerated; the local references, the membership and both parity files regenerate unchanged (21,596 active). Each finding has its own commit.

1. must, fixed in c00f377. Both exclusion reasons now say only which floor each area misses: Isles of Scilly misses the 50-row and ESS-50 floors at every K measured, and City of London clears the 50-row floor at K=15 but misses ESS-50. The generator's deferral rationale for the two areas' own cells (tools/generate_uk_local_target_references.py) carried the same row counts into every such membership entry. It is re-worded the same way and the membership regenerated; no reference moves.

2. should, done in 8361c7b. uk_council_tax_band_domain_terminal is the same release-blocking check at the terminal phase, and a test pins its parameters to the spine entry's. The release-cut scope owns it, so in the full build a failure stops the build rather than counting as a local export exception. Contract mirrors added; five gate digests re-pinned. On merge order: agreed. If this lands first, Place high-value homes where the Land Registry sees them (#1240) #1270 regenerates the four registers from the pinned feed and the 21,596 / 1,790 pins move with it; if Place high-value homes where the Land Registry sees them (#1240) #1270 lands first, I regenerate here.

3. should, done in 44f39ab. Each group that reports gains now also passes two dominance rules on its weighted gains, counted by source household: the largest holds at most half of the group's gains, and the p% rule at p = 10. A group that fails pools with its neighbour, and the script refuses if no grouping passes. Its output stays outside the repository.

4. follow-up. Agreed that the flag needs a consumer. Whether py withholds or marks a listed area is a publication decision, so it goes to a policyengine.py issue rather than this PR.

5. nit, fixed in acd6bda. A fixture-only cell that a signed small-cell rule defers now carries its deferral's signed rationale, which names the rule: Barking and Dagenham band G names council_tax_band_household_floor. A test pins every such row.

6. nit, 4a3b4ed. The firings stay in the spine's stage evidence, which no gate reads. A comment at the source says to band them before any gate detail or report carries them.

7. nit, fixed in a64e9e7. frs_council_tax_band sets D for every Northern Ireland household whatever ctband holds, with a test, and the comment says what the code does. On the licensed spine every Northern Ireland household already held D, so no band moves.

On your not-checked list: the engine-uk tests, the feed-gated compiles on the 942a1fa feed and the twin licensed spines on the new base (control 412f916, head 4a3b4ed) are running now, with the full engine-free lane. I'll add their results here when they finish.

🤖 Generated with Claude Code

[juaristi22]

Rebased again after #1270 merged (main at 5792966). This PR landed second, so it regenerates rather than merging the registers by hand:

• the local references, the membership and the local parity receipt are regenerated from the 735820c feed main now pins (the artifact rebuilt from PolicyEngine/chronicle at 735820c and checked against the pinned row count and facts digest);
• the counts move with Place high-value homes where the Land Registry sees them (#1240) #1270's high-value homes cells: 24,269 active local references (main's 24,477 less the same 208 small-cell cells) and 1,796 reviewed exclusions. The scorer's frozen count, the battery pin and the runbook move with them;
• local_binding_adjudications.json keeps the six renewals and Place high-value homes where the Land Registry sees them (#1240) #1270's high_value_home_stock_derived_estimate entry unchanged;
• the gate digests, the H2 spine fixture and the coverage manifest are regenerated again;
• the head is 426ac3e; its last commit, Reconcile with main after #1270 and the 735820c feed, sets the tree to the tested merge of the branch and main. The review fixes in my reply above now have these ids: 1 is 9b0c674, 2 is 03d9a7f, 3 is 83f405a, 5 is 983a357, 6 is 21426e8 and 7 is 1664839.

Twin spines: not rebuilt. My earlier comment said they were running; I stopped them. The C1 receipt was measured on twin licensed spines at main 121fd4d and the branch's C1 commits on that base. Neither rebase touches the shell's redraw, and #1270's band redraw after geography derivation is covered by uk_council_tax_band_domain_terminal. The receipt now says this.

Tests at 426ac3e: the tested merge of the branch and main passed the targeted suite: 1,150 tests over the gate pins, the contract mirrors, the small-cell rules, the battery bindings, the H2 fixture, the full target graph and the release certification (5 skipped without the feed). ruff check ., ruff format --check on the changed files, tools/ci_test_plan.py verify and tools/graph_acceptance_burndown.py --verify are clean. The full engine-free lane, the engine-uk files that touch the changed modules and the feed-gated compiles on 735820c are running here, and CI is running on the head; I'll add their results to this comment when they finish.

🤖 Generated with Claude Code

[vahid-ahmadi]

Fine once the new C6 test passes on the rebased base (finding 8). All seven round-1 points are dealt with.

@juaristi22, I re-reviewed at 426ac3e. It sits directly on main at 5792966 (the #1270 merge). I read both of your replies. Thanks for the per-finding commits.

Round-1 points

1. Fixed. local_area_support_exclusions.json:6 and :13 now say only which floor each area misses. The generator's deferral rationale (tools/generate_uk_local_target_references.py:584) and all 96 membership entries that carry it say the same, with no figures. I found no per-area ESS or row counts in the added lines of any commit at head, in the commit messages or in the PR body. One thing to know: the diff of commit 6c7818d still carries the old figures on its added side, because it edits the two reason lines before 9b0c674 rewrites them. Those figures are already on main, so this discloses nothing new. If the branch history is kept on merge, though, they appear in it once more. A squash merge would avoid that.

2. Fixed. uk_council_tax_band_domain_terminal (gates.json:1228, calibration_run.py:172) has the same parameters as the spine entry, and a test pins that. I traced the wiring by reading the code:

   • uk_full_gate_manifest drops only local fit gates;
   • the terminal branch of UKFullGateKernel.run evaluates that manifest on the calibration population;
   • that population is based on uk.full.pool, which checkpoints uk.full.expand after uk.full.property.band has rewritten the band.

   Since Place high-value homes where the Land Registry sees them (#1240) #1270 is merged, I probed the case I raised. I ran Place high-value homes where the Land Registry sees them (#1240) #1270's localise_bands with a synthetic English LSOA mix that puts all its mass on band I. The redraw writes I for the English owner, and the terminal entry from uk_full_gate_manifest() then fails with I@ENGLAND. The entry is not in the local export exception, so on a release-candidate run a failure stops the build.

   The merge-order point is handled too. Between main and head, local_target_references.json loses exactly 208 references and gains none:

   • 192 higher rate taxpayer cells;
   • 16 band cells.

   All 208 sit in the head membership as signed deferrals, and the membership's other status counts and its source feed match main's. So nothing from Place high-value homes where the Land Registry sees them (#1240) #1270's register was lost or duplicated. The contract and gate-digest tests pass at head.

3. Fixed. cgt_stage_residual.py:41-89 adds a largest-share rule (at most half) and the p% rule (p = 10). Failing groups are pooled, and the tail is joined backwards until it passes. I checked passes_dominance and pooled_groups on synthetic groups:

   • a dominated group fails;
   • a two-large-holders group fails the p% rule;
   • an even group passes;
   • a dominated group pools with a clean neighbour;
   • a grouping that cannot pass refuses.

4. Open as agreed (follow-up). Fine as a policyengine.py issue. Whether py withholds or marks the listed areas is a publication-policy decision.

5. Fixed. The band G E09000002 row (ledger_compile_parity_local_incumbent_2025_signed_differences.json:21335) now names council_tax_band_household_floor. The dwelling counts in that reason are published figures, so they are fine.

6. Fixed. The comment at spi_housing_shell.py says to band the firings before any gate or report carries them.

7. Fixed. frs_council_tax_band (frs_spine.py:1080-1093) now forces D for Northern Ireland, and the comment matches the code.

New

8. must packages/microcosm-build/tests/engine_free/uk/test_uk_graph_terminal.py:86. The C6 test calls describe_uk_export(frame, bindings={"target_scope": "all"}) without solve_year, which became a required keyword in UK: policyengine-uk 2.130.1 and the fourth batch of #1095 ports, with the release's weight year, identifiers and birth months fixed #1271 (0f0bb86). It fails here with TypeError: describe_uk_export() missing 1 required keyword-only argument: 'solve_year', so the engine-free job should fail too (it was still running when I posted). With solve_year=2025 added, as the neighbouring tests at :194 and :203 do, it passes. Your targeted run of 1,150 tests at 426ac3e does not list test_uk_graph_terminal.py, which may be why this was missed.
9. nit PR body. It still gives the pre-rebase surface: active local references 21,804 to 21,596, the 942a1fa feed, and only the spine gate. The receipt and 426ac3e now say 24,477 to 24,269 on 735820c, and the terminal gate is new. Worth updating so the body matches what merges.

What I checked

• Read both replies, all 17 commit messages, the range against the old head c9bc232, and the PR body.
• Ran the 15 touched test files in a fresh venv: 1,028 passed and 1 failed (finding 8). The failure passes once solve_year is supplied. Also ran ruff check ., ruff format --check on the changed Python files, tools/ci_test_plan.py verify and tools/graph_acceptance_burndown.py --verify, all clean.
• Ran the band-redraw probe, the dominance probe and the register set difference described above. The probes use synthetic data.
• Scanned the added lines of every branch commit, the messages and the body for per-area ESS and row counts.
• Not checked:
  • the engine-uk tests;
  • the feed-gated compiles on 735820c;
  • the licensed twin spines;
  • a real full-graph build. I traced the gate node's inputs by reading the code, not by running a build.

When I posted, CI showed lint, engine-uk, integration-uk and wheels passing. Engine-free and engine-us were still running.

🤖 Generated with Claude Code

[juaristi22]

Thanks @vahid-ahmadi.

8. Fixed in 0e60dee. The C6 test now passes solve_year=2025, as its neighbours do, and test_uk_graph_terminal.py and test_uk_release_identifiers.py pass. UK: policyengine-uk 2.130.1 and the fourth batch of #1095 ports, with the release's weight year, identifiers and birth months fixed #1271 made the keyword required after the test was written, and my targeted runs after the rebases left that file out, which is how it slipped through.

9. Already done. I updated the body at 13:40, a few minutes before your review posted: it gives 24,477 to 24,269 on 735820c and names the terminal gate.

On finding 1's note: agreed. This PR will be squash-merged, so those figures do not enter main's history again.

On the local lanes I said I would report: I stopped them, and CI on 0e60dee runs the full engine-free and engine-us jobs. I'll add CI's result to my earlier comment when it finishes.

🤖 Generated with Claude Code