UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301)

.gitignore

@@ -19,3 +19,8 @@ tools/route_a/route_a.env
 
 # Finder metadata — never payload
 .DS_Store
+
+# Dense sign-off drafts (tools/draft_uk_dense_signoff.py): the private review
+# file holds licensed-data review values and is never committed
+PRIVATE_review.csv
+signoff-draft/

changelog.d/1300-uk-chronicle-repin-abf0921.changed.md

@@ -1,9 +1,9 @@
 The UK Chronicle feed moves to PolicyEngine/chronicle `abf0921` (PolicyEngine/chronicle#359, which answers chronicle#326, #339, #356, #357 and #358; microcosm#1300). Values move only for the UC area rows.
 
-- **UC households by area count paid claims.** Chronicle splits each area-month by Stat-Xplore's Payment Indicator and retires the all-claims cells. `dwp.uc.households_by_area` and its four child bands add `payment_indicator` to their dimensions and bind `Yes` (the three-plus band's summed operands name it too), so the area cells and the GB caseload that controls them both count paid claims. Against the all-claims cells, the GB constituency total falls 8.0% (households with no children 10.5%, one 6.6%, two 4.6%, three or more 3.0%), so the childless share falls from 53.6% to 52.1%. Each cell takes its own area's paid share where it took the GB one: after the bridge, the middle 90% of constituencies move by between -1.7% and +1.3%. The bridge's factor moves from 0.921 to about 1.001, and a new `CrossGrainBridge.max_factor_shift` (2 per cent on this bridge) refuses a factor far from one. Each lower grain is bounded on its own input total, whichever group rescales it first, and a surface with off-surface cells is bounded on the pass that carries them.
+- **UC households by area count paid claims.** Chronicle splits each area-month by Stat-Xplore's Payment Indicator and retires the all-claims cells. `dwp.uc.households_by_area` and its four child bands add `payment_indicator` to their dimensions and bind `Yes` (the three-plus band's summed operands name it too), so the area cells and the GB caseload that controls them both count paid claims. Against the all-claims cells, the GB constituency total falls 8.0% (households with no children 10.5%, one 6.6%, two 4.6%, three or more 3.0%), so the childless share falls from 53.6% to 52.1%. Each cell takes its own area's paid share where it took the GB one: after the bridge, the middle 90% of constituencies move by between -1.7% and +1.3%. The bridge's factor moves from 0.921 to about 1.001, and a new `CrossGrainBridge.max_factor_shift` (2 per cent on this bridge) refuses a factor far from one. The bound applies to each grain's GB total (the constituencies' and the authorities'), read before any group rescales it and, on a surface with off-surface cells, on the pass that carries them; it does not bound each nation or English region within a grain.
 - **OBR table 4.9 lines carry DWP's geography.** Twelve DWP lines are stamped Great Britain or England and Wales (`TARGET_ID_GEOGRAPHY_PINS` in `tools/generate_uk_target_references.py`), and the `obr.state_pension` diagnostic, which `dwp.state_pension.amount` requires, reads its Great Britain fact; no value changes.
 - **DESNZ gas runs on gas years.** Northern Ireland's rows now share Great Britain's concepts, so both sub-national vendor selections pin Great Britain (`geography_vintage: publication_2025_nspl`); the local-authority one accepts `gas_year` and `calendar_year` rows at `domestic_scope: all_domestic`, and its expected row count (697) and the per-fuel reads pair gas with gas years and electricity with calendar years. The energy stage and the local energy appliers name each fuel's period type (`energy_pricing.SUBNATIONAL_PERIOD_TYPES`): gas year Y pairs with calendar year Y, as the rows were read before Chronicle stamped the gas period, so the gas-connected shares and the Great Britain levels do not move.
 - **The Northern Ireland energy interim stays for now.** Great Britain's share of UK households still stands in for its share of the Energy Trends level: the Northern Ireland facts are in the feed on a basis DESNZ calls not directly comparable, and a change stacked on this re-pin derives Great Britain's part from them (María's ruling of 2026-10-16). The adjudication, its census fence and the code say so.
 - **Schema digest.** The v4 consumer-fact schema digest moves to `d953306d…` (the `gas_year` period type); the NZ pin moves at its own next re-pin.
-- **The local compile-parity receipt moves with the pin.** `uk/ledger_compile_parity_local_incumbent_2025_signed_differences.json` is regenerated on `abf0921`: the release-blocking preflight gate `uk_ledger_compile_parity_local_incumbent_2025` compares its signed values with the live compile, and only the 3,508 UC cells' values move.
+- **The local compile-parity receipt moves with the pin.** `uk/ledger_compile_parity_local_incumbent_2025_signed_differences.json` is regenerated on `abf0921`: the release-blocking preflight gate `uk_ledger_compile_parity_local_incumbent_2025` compares its signed values with the live compile, and only the UC cells' signed values move: 3,507 of the 3,508 change, and one, which now equals the incumbent, leaves the receipt.
 - **Follow-up.** The twelve restamped OBR lines move the national register version with no value moved, so the uk-candidate-eval scoring register is re-frozen before the next national solve.

changelog.d/1301-high-value-homes-region-control.added.md

@@ -0,0 +1,7 @@
+The UK local line binds region controls for the high-value homes family (microcosm#1301, the issue's 6a). Three new two-level targets, `microcosm.high_value_homes.over_1m_by_region`, `over_1_5m_by_region` and `over_2m_by_region`, bind the derived Land Registry stock at the nine English regions and Wales, except the North East and Wales rows over £2m, which are signed out below the 400-home floor. They carry the same measurement block and record-set selector as the constituency and local-authority cells, so each row is the cross-grain control for its area cells, and the three `no_higher_control` local routes in `uk/cross_grain_declarations.json` are gone.
+
+**Local line only.** A new contract key, `calibration_lines`, names the lines a target binds on; these three declare `["local"]`. The national spine's `main_residence_value` is the WAS donor's, not a Land Registry placement. The national compiles now merge the derived facts beside the Chronicle feed too, as the local ones already did: both loaders in `full_targets.py`, the release-cut certifier, the compile-parity tool's national receipt, the national reference generator (`--derived-facts` names another resource), the UC matrix diagnosis and the incumbent-surface evaluation. `load_uk_national_target_inputs`, the certifier's calibration reference and the UC matrix diagnosis then drop the local-only rows right after the compile, before reconciliation, the measure exclusions and the frozen-register check. The national registry version is therefore unchanged. `register_completeness.calibration_line` receipts what each line dropped.
+
+**Fan-out.** `tools/generate_uk_target_references.py` now honours a two-level target's `area_scope` by level. The region rows fan out over the English regions and Wales, which is pinned at country level `W92000004`. Scotland and Northern Ireland are not cells.
+
+**Measures.** The bound values `owner_occupied_home_over_gbp_1m`, `_1_5m` and `_2m` are per-household indicators computed in `measure_simulation.py`. Each applies `high_value_home_mask` at the threshold's frame-basis value to the frame's stored `main_residence_value` and `tenure_type`, never an engine-uprated value, exactly as the local cells and the release gates do. A persisted column of any of the three is refused.

changelog.d/1301-l0-search-bracket.fixed.md

@@ -0,0 +1 @@
+The `target_records` L0 budget search no longer clamps a warm start into its bracket, re-probes a penalty it has measured, or bisects a bracket that has collapsed onto a bound (microcosm#1301). `calibrate` takes a `budget_search_bracket` whose default stays [1e-7, 10], so every caller that does not pass one, the US contract included, keeps the same bracket and the same cold path (mid-point first, then bisection). The warm path changes for every caller: a warm start outside the bracket now raises rather than being silently moved to the bound while the receipt recorded the value requested, and a warm search now stops at a bound of the bracket rather than bisecting toward it. The search never spends a probe on a penalty within 1e-6 decades of one it has measured; a probe on the floor or ceiling that asks to go past it stops the search as `search_floor_reached` or `search_ceiling_reached`, and a bracket narrowed onto a measured penalty stops it as `bracket_exhausted`, beside the existing `acceptable_within_tolerance` and `budget_exhausted`. The warm edge probe now follows the first probe's own steer: a warm start on the floor that asked for a smaller penalty was sent half a decade upward. When the edge still asks to go past it, the search probes the bound itself next, so a budget out of reach stops there by name within three probes instead of bisecting toward the bound until every probe is spent; one within reach is interpolated between the bound and the edge. The search receipt and its `budget_search_done` event record the bracket and the stop reason.

changelog.d/1301-staging-read-back-refresh.fixed.md

@@ -0,0 +1 @@
+A failed best-effort upload after a passed staging read-back no longer fails the build (microcosm#1301). Once read-back has given its verdict, the verdict owns the delivery's `last_error_code`: the refresh that follows a passed read-back cannot set `UPLOAD_FAILED`, which the delivery contract refused as a contradiction and which escaped the dense driver before it recorded the staging delivery and the Logbook row. A failed refresh stays visible as an upload attempt without a success, and the remote copy may still show the delivery as it was before read-back.

changelog.d/1301-uk-dense-opt-in-holdout-incumbent.changed.md

@@ -0,0 +1 @@
+The dense UK line's rotated holdout and incumbent-surface comparison are opt-in diagnostics rather than publication requirements (microcosm#1301). `--rotated-holdout` replaces `--skip-holdout`, which is removed with no alias: a dense build skips the holdout unless asked and records the skip, and the national role refuses the new flag. The pre-flight accepts a holdout that was not requested and still requires five folds and a finite mean when it was, in agreement with the request. The scorer reads the active local reference count from the membership file and scores a candidate with no holdout as `holdout_basis: none_requested`. `tools/evaluate_uk_dataset_size.py` runs steps 30 and 40 only with `--incumbent-diagnostic` or an explicit `--steps`. Certification readiness is non-gating: an absent scorecard is `not_requested` and one that misses only its measured limits is `diagnostic_failed`. The release contract always joins the candidate's own evidence to the released bytes; the score, the surface evaluation and the incumbent extraction manifest ship all together or not at all, their identities refuse, and their measured limits are recorded in the manifest rather than enforced. The source coverage carries a `holdout.claim`, an `incumbent.compared` and a `size` discriminator. The assembler takes `--incumbent-manifest` as optional, requires the run's full gate report, prints a `--tag-only` publish command, and writes the limitations the contract checks against the coverage: `no_out_of_sample_claim` or `rotated_holdout` (`claim: report_only`), `no_incumbent_comparison` or `incumbent_surface_diagnostic`, `engine_measurement`, `size_doctrine` for a sized release, and `reviewed_exclusion:<gate>:<name>` for the reviewed exclusions `uk_target_fit`, `uk_local_target_fit` and `uk_input_mass_parity` stood on, each with its register reason copied verbatim. The coverage records those names per gate under `signed_deferrals.reviewed_exclusions`, and the contract requires exactly those limitations. The full gate report does not ship, so the build records the names each of those gates stood on in the candidate manifest's `reviewed_exclusions_stood_on`, and the contract holds the coverage's names to that record; it also checks the `uk_local_target_fit` names against the signed local gate report. A dense manifest must carry its `area_support` block, whose `below_floor` codes are exactly the areas the signed report's area-support exclusions stood on. The coverage's holdout and size records and the `engine_measurement` and `size_doctrine` limitations must restate the shipped candidate manifest, so an edited coverage cannot switch the holdout claim or the size.

changelog.d/1301-uk-dense-signoff-draft.added.md

@@ -0,0 +1 @@
+Add `tools/draft_uk_dense_signoff.py`, which drafts the gate-only sign-off between the two dense UK builds (microcosm#1301). From build 1's run directory it reads the full gate report, the signed local gate report, the target and area-support tables and the Logbook row, checks them against one another and against the checkout's `gates.json`, and drafts `release_lines: ["local"]` entries for every failing national target-fit row, local target-fit cell and area below the support floor. It also drafts the removal of every entry build 2 would fail and the test pins those registers move, as a `git apply` patch with a PR body of counts, gate ids and policy dates. A national target-fit entry that applies on every line is never removed or replaced, since the national line may still stand on it: when build 2 would fail it on the local line it is rescoped to `release_lines: ["national"]`, otherwise unchanged. An expired one also goes to review for the national line to renew or retire, and the draft is unsignable until then if keeping it breaks the national graph test's review invariant; a row build 1 still breaches under such an entry needs a hand sign-off, since a register key holds one entry. Any other release-blocking failure makes the build unsignable, and so does an area exclusion the local reference generator's own deferrals keep, when build 2 would fail it. The receipt and the PR body give the assembly deadline: the earliest expiry among the measure exclusions build 1's solve ran under (from its candidate manifest, or from the target compilation's surface when its gates stopped it before the export), the local line's degenerate-register entries and every gate-only entry build 2's gates hold to its window after the sign-off (those on the local line, every `bound_despite_support_floor` family whatever its line, and the local line's input-mass entries, which the tool reads but never drafts). Build 2's gate-clock range ends there and starts at the latest of build 1's clock and those entries' approvals. A gate clock past the solve-scoped expiries is refused, and an input-mass entry out of its window at the gate clock makes the draft unsignable until it is renewed by hand. Reasons are templated from the key, the gate's bound or the floors the area missed, the build-1 attempt id and the full gate report's sha256. `--approved-by` must be a GitHub login that starts with a letter and holds at most one run of digits; the reviewer's `--class-note`, `--area-name` and `--adjudication` text may carry digits only in `microcosm#N`, `chronicle#N`, `uk-data#N` or `PolicyEngine/<repo>#N` references (optionally anchored `#issuecomment-N`), GitHub URLs of a PolicyEngine issue or pull request and ISO dates of real days in 2000–2099, never in a bare `#N`, and no percent sign in any form; and the drafted entries then get the release contract's disclosure walk, where a finding refuses the draft, though the walk only warns on a release directory; on text the walk catches only an area's support worded with its measured value, and none of these checks reads a figure written in words. Per-row errors, per-area support figures and the local registers' stale-risk and near-bound counts go only to a private review file (the local counts also to the tool's standard output); the PR body and the receipt carry the national register's counts alone. `--out` must lie outside every git work tree, `.gitignore` covers `PRIVATE_review.csv` and `signoff-draft/`, and the support table's maximum weight is never read. `--verify` checks that build 2 stood on exactly the committed entries, that no release-blocking gate failed and that its signed local report passes the dense contract, and, with `--draft-receipt`, that the sign-off committed what was drafted and changed each rescoped entry in its release lines only; it warns when the disclosure walk finds a per-area support figure in that report, fails once the assembly deadline has passed, warns from two days before it, and reports whether build 2 reused build 1's solve. The three fixed-clock target-fit tests in `test_uk_terminal_gates.py` now read the register as the national line does, so local-line sign-offs approved after their clock leave them unchanged.

changelog.d/1301-uk-dense-support-figures-withheld.fixed.md

@@ -0,0 +1 @@
+The files a dense UK release ships no longer carry per-area support figures, an area's rows, effective sample size or distinct source households (microcosm#1301). The `uk_local_area_support` gate's failures and reviewed exclusions name the floors an area misses (`rows < 50`) instead of the measured values, each reviewed exclusion records only those shortfalls and its signed record, and each geography level reports the areas checked and below the floor in place of its minima. The weakest-area rows of the diagnostics and the candidate manifest's support-limited misses carry `support_below_floor` in place of the figures; `UKAreaFit`'s figure fields become optional, so earlier diagnostics still parse. The candidate manifest's `support` block counts the areas checked and below the floor, overall and by level. The figures stay in the build's `.area_support.csv`, which no release ships. The dense release contract walks every shipped JSON document, the signed gate report and the release manifest's strings included. It warns, without refusing the release, on a support figure, as a number or as numeric text, in an area's record (one with an `area_code`, a `geography_id` or an area `code`, or anything beneath an area key), in a code-keyed map under the figure's name, or in a table with an area column (parallel lists or pandas `{row: value}` columns); a minimum, maximum or median of one outside the declared floors; and a shortfall worded with the measured value. It also warns on a shipped `*.json` it cannot parse. Each warning names where the figure is, never its value. `validate_release_dir` now returns the contract's warnings (none on any other line); the dense assembler prints them and lists them in its summary, the release pre-flight prints them without failing, and publication repeats them as Python warnings. Weights and per-cell estimates are unchanged. The full gate node's code identity now also covers `gates` and `terminal_gates`, so an edit to a gate evaluator or a register loader re-keys the gate nodes.

changelog.d/1301-uk-local-gate-only-registers.added.md

@@ -0,0 +1 @@
+The dense UK line's `uk_local_target_fit` gate reads its own reviewed-exclusion register, `uk/local_target_fit_reviewed_exclusions.json`, committed empty, so a failing local cell can be signed after a build by a data-only change (microcosm#1301). Keys are the gate's `family/area_code/metric` cell names. An in-force entry lifts the fence for its cell only and the cell stays bound in the solve; stale, expired and not-yet-in-force entries fail, and an entry scoped by `release_lines` to another line stands aside. The gate now evaluates entries on the build's review date, the clock the other register-reading gates share, rather than the wall clock, and records `evidence_absent` without it. The `uk_local_area_support` exclusions become gate-only: an entry keyed `local_authority/<code>` or `constituency/<code>` excuses its area at the gate and defers none of its cells, and an entry for another line stands aside. The Isles of Scilly and City of London cell deferrals and the families bound despite the floor are now constants of `tools/generate_uk_local_target_references.py` (`SUPPORT_FLOOR_DEFERRED_AREA_IDS`, `SUPPORT_FLOOR_BOUND_FAMILIES`), held in lockstep with the register by tests, so the generator reads nothing from it and the committed local references and membership regenerate byte-identical. The size evaluation's red-row ledger keeps signed cells, which still breach the bound.