diff --git a/changelog.d/1124-uk-local-doctrine.changed.md b/changelog.d/1124-uk-local-doctrine.changed.md new file mode 100644 index 0000000000..2ac50c165b --- /dev/null +++ b/changelog.d/1124-uk-local-doctrine.changed.md @@ -0,0 +1,3 @@ +The UK local doctrine moves to 2,000 epochs and the `nation_grain_family_equal_sqrt_count` target-weight rule and keeps K=15 clones (microcosm#1124; rulings of 2026-10-13), so the dense role's defaults and a release candidate's pinned posture follow, and the release pre-flight checks the ruled values. A sized dense build (`--dataset-households`) defaults to an exact-count draw at π_hi 0.5, a refit baseline floor of 0.5 and a chi-square refit L2 penalty toward uniform weights at λ 0.0216 when the request leaves them unset; explicit values still win. + +The local release can now ship the 100,000-household selection instead of the whole pool. A release candidate may request `--dataset-households 100000` when it carries the size doctrine (`local_doctrine.uk_local_size_doctrine`): those settings, no selection L2 penalty, drawn with the solve seed and selected afresh rather than resumed from a size checkpoint. Any other size or setting is refused, and stays candidate-only. Such a build is releasable on the dense line's requirements, with no matched-size scorecard: every release-blocking gate, the rotated holdout (each fold reruns the size selection, its budget search starting from the main search's `--selection-initial-lambda`), and the incumbent-surface evaluation on one engine over the released file. The pre-flight also checks the size receipt: requested and realized 100,000 households, under the doctrine's settings. diff --git a/changelog.d/1124-uk-local-release-gates.changed.md b/changelog.d/1124-uk-local-release-gates.changed.md new file mode 100644 index 0000000000..e87e9fb80c --- /dev/null +++ b/changelog.d/1124-uk-local-release-gates.changed.md @@ -0,0 +1,7 @@ +Three release-blocking gates that failed on the first K=25 local candidate for reasons of policy, not fit, now have reviewed answers (microcosm#1124; María's rulings of 2026-10-13). + +- **Line-scoped exclusions.** A reviewed exclusion may name `release_lines` (`national`, `local`), the lines whose evidence earned it. A gate context declares its line (`release_line`): the full build declares `local`, and the national gate kernel and the release-cut certifier declare `national`. On any other line, and wherever a context declares no line, a scoped entry stands aside: it neither suppresses a failure nor reads as stale, and the gate details name it. Unscoped entries behave as before everywhere, and unscoped receipts keep their policy payload. The four schema-2 exclusion gates (degenerate surface, target fit, input-mass parity, QRF tail) honour it. +- **`uk_qrf_tail_concentration`.** The `charitable_investment_gifts` exclusion is scoped to the national line, where it was earned. On the K-clone local pool the gate counts clone rows, so the column sits below the threshold and the entry read as stale. +- **`uk_degenerate_release_surface`.** `household.atomic_area_basis` is constant at `assigned` by construction on the local line; it gets a local-line exclusion and stays in the export as assignment provenance (microcosm#932). +- **One engine block per clone.** The local release pre-flight accepts an engine resolved per clone when the build recorded the pool representation exact (identical copies scaled to the pool), as `release_verdict` and the runbook already did. The record now states what exact covers (`exact_scope`): the weight-share scaling, not engine draws that depend on a record's position, which can differ between per-clone and single-block runs. The incumbent-surface evaluation still requires one block, re-measures the released file on it and fails release on any row more than 25% from its target, so the shipped file's fit is gated on the engine a consumer runs. +- The dense runbook passes the national line's licensed `--input-mass-reference`. diff --git a/changelog.d/1124-uk-local-scorer-graph-selection.fixed.md b/changelog.d/1124-uk-local-scorer-graph-selection.fixed.md new file mode 100644 index 0000000000..1950467baf --- /dev/null +++ b/changelog.d/1124-uk-local-scorer-graph-selection.fixed.md @@ -0,0 +1 @@ +`tools/score_uk_local_candidate.py` scores graph-built candidates again. It reads the candidate's target selection (the manifest's `outputs.target_registry`, national and local rows together) and keeps its local rows; a registry artifact still loads as before. Its frozen count is re-pinned to 21,804, the compiled local reference count of the #1123 surface on the 942a1fa feed (it was 20,885 from before #1123). Every current candidate failed both checks, and assembly requires the score receipt. Found by the f1 smoke of the local release path. diff --git a/changelog.d/1124-uk-problem-kernel-identity.fixed.md b/changelog.d/1124-uk-problem-kernel-identity.fixed.md new file mode 100644 index 0000000000..031b8231d9 --- /dev/null +++ b/changelog.d/1124-uk-problem-kernel-identity.fixed.md @@ -0,0 +1 @@ +The UK local solve binds the row-metadata target-weight rules end to end (microcosm#1124): `prepare_uk_full_solve` computes a `grain_family_equal` or `nation_grain_family_equal` (or `*_sqrt_count`) vector from the row carrier of its own targets and keeps the carrier on the prepared solve, the rotated holdout weights its held rows through the same carrier, and the graph problem binds a `target_loss_weight_receipt` (shares by grain, family and nation, and the vector's digest) that the build manifest copies into `solve`. The dense posture, the CLI choices, the holdout diagnostics schema and the candidate scorer admit the new names; the national role refuses them, and release candidates refuse every rule but the doctrine's. The `uk.full.problem` kernel's implementation hash now also covers `local_rowwise`, `national_doctrine` and `target_weights`, where its loss weights are built, so an edit to those modules re-keys the problem instead of replaying a stale vector; the rotated-holdout kernel's hash covers the doctrine and rule modules for the same reason. diff --git a/changelog.d/1124-uk-qrf-tail-surface.changed.md b/changelog.d/1124-uk-qrf-tail-surface.changed.md new file mode 100644 index 0000000000..4ab3d2050f --- /dev/null +++ b/changelog.d/1124-uk-qrf-tail-surface.changed.md @@ -0,0 +1,3 @@ +The UK QRF tail-concentration gate no longer checks the report columns `spi_benefit_coherence` (microcosm#1100) replaces on every SPI-channel row. The stage zeroes the income-tested legacy reports (contributory and income-based JSA, Income Support, Working and Child Tax Credit, income-related ESA, Severe Disablement Allowance) and restores IIDB, AFCS and bereavement support from the FRS twin. After it these columns hold only FRS respondents' own reports, so the donor-tail broadcast the gate looks for (#462) cannot occur in them; their concentration is that of a small population of genuine reporters and their clone copies. The surface reads the superseded columns from the stage's declared operations and lists them in its details as `superseded_by_spi_benefit_coherence`. + +`hmrc_spi_other_income` and `hmrc_spi_taxable_termination_pay` stay on the surface: they are real stage-1 draws. They failed national release-cut certification as genuinely concentrated instruments, rare on the SPI tape, whose top 100 records carry nearly their whole mass whether counted by row or by independent record; the CGT clones and SPI copies also take them past the carrier floor. They are deferred behind national-line reviewed exclusions, time-boxed to 2027-01-14, until the gate is re-armed on independent draws (microcosm#1277). diff --git a/changelog.d/1124-uk-refit-best-iterate.changed.md b/changelog.d/1124-uk-refit-best-iterate.changed.md new file mode 100644 index 0000000000..0de42e6479 --- /dev/null +++ b/changelog.d/1124-uk-refit-best-iterate.changed.md @@ -0,0 +1 @@ +The UK size refit now keeps its best epoch on the objective it minimizes (microcosm#1124). With an L2 penalty the shared solver used to return the closing state, because its best-iterate rule tracked the target loss alone. `calibrate` and `refit_l0_selection` gain `retain_best_objective`: with `l2_lambda > 0` on the plain Adam path (free mass, no L0) the solve returns the epoch with the lowest target loss plus the weighted penalty, recorded as `options["iterate_selection"] == "best_feasible_objective"`. It is off by default, so other solves are unchanged; the UK refit turns it on, since under the 100,000-household release its weights are the released weights. diff --git a/changelog.d/1124-uk-size-stage-l2-wiring.added.md b/changelog.d/1124-uk-size-stage-l2-wiring.added.md new file mode 100644 index 0000000000..7c08e7e8f0 --- /dev/null +++ b/changelog.d/1124-uk-size-stage-l2-wiring.added.md @@ -0,0 +1 @@ +`microcosm-build-uk --dataset-households` runs take the size stages' L2 penalties from the command line (microcosm#1124): `--selection-l2-lambda/--selection-l2-anchor/--selection-l2-basis` for the informed L0 search and `--refit-l2-lambda/--refit-l2-anchor/--refit-l2-basis` for the exact-count refit (chi-square basis; anchors `initial` and `uniform`). Each requires `--dataset-households` and is refused by the national role. A release candidate may carry only the size doctrine's refit penalty and no selection penalty (`rowwise_cli.size_doctrine_drift`), and the pre-flight fails on any other recorded penalty. `UKGraphCalibrationConfig` carries them as `selection_l2`/`refit_l2`: the selection penalty joins the search, draw and refit node parameters, the refit penalty the refit node's alone, and the rotated holdout re-solves each fold under both. An off penalty adds no node parameter, no run parameter and no identity field, so default builds keep their node keys' parameters, manifests and checkpoint identities. diff --git a/changelog.d/1124-uk-size-stage-l2.added.md b/changelog.d/1124-uk-size-stage-l2.added.md new file mode 100644 index 0000000000..fafa21358e --- /dev/null +++ b/changelog.d/1124-uk-size-stage-l2.added.md @@ -0,0 +1 @@ +The UK dataset-size search and refit each take their own L2 penalty (microcosm#1124): `select_uk_dataset_size` gains `selection_l2_lambda/anchor/basis` and `refit_uk_dataset_size` also `refit_l2_lambda/anchor/basis`, validated by `uk_size_l2` (chi-square basis only, since the record basis only shrinks mass under free mass; anchors `initial` and `uniform`, where the refit's `initial` is the normalized, optionally floored Horvitz–Thompson baseline). An off penalty passes no L2 keyword to the solver, so default runs, their solver options and their size receipts are unchanged; an on penalty is receipted (`selection_l2`, `refit_l2` with the chi-square distance to its anchor, `refit_iterate_selection`). A reused selection must have been searched under the requested L2 and loss weights; a full-pool size refuses size-stage settings. The size checkpoint binds `selection_l2` in its identity only when it is on, so existing checkpoints keep loading, and its loader and writer check it against the search's own options. A typed `UKStageTargetWeighting` seam runs either stage under a named rule other than the dense solve's (the size-experiment harness; graph builds never pass it), and `UKSizeDraw.from_payload` parses a stored draw for the refit node and the harness alike. diff --git a/changelog.d/1124-uk-target-weight-rules.added.md b/changelog.d/1124-uk-target-weight-rules.added.md new file mode 100644 index 0000000000..16fa4f8eb6 --- /dev/null +++ b/changelog.d/1124-uk-target-weight-rules.added.md @@ -0,0 +1 @@ +Four named UK local target-weight rules that read row metadata beyond the grain (microcosm#1124): `grain_family_equal` (equal grain shares, then equal family shares inside each grain), `nation_grain_family_equal` (a fourth `nation_region` grain for every national row below the UK-wide aggregates, splitting on the new shared `UK_WIDE_GEOGRAPHY_IDS`), and a `*_sqrt_count` variant of each that shares a cell's count budget by the square root of the target value while keeping every grain, family and cell budget. They are computed in `uk_runtime.target_weights` from one row-aligned carrier (`UKTargetRows`) built identically from declared targets and from a stored ordered problem, with a weights digest and a share receipt by grain, family and nation, and a typed stage seam (`UKStageTargetWeighting`) for the size experiments. `uniform` and `grain_equal` are unchanged bit for bit. `nation_grain_family_equal_sqrt_count` is the doctrine's rule (`1124-uk-local-doctrine.changed.md`), and release candidates refuse every other rule. diff --git a/changelog.d/882-uc-housing-element-renewal.changed.md b/changelog.d/882-uc-housing-element-renewal.changed.md new file mode 100644 index 0000000000..fc97d1ff75 --- /dev/null +++ b/changelog.d/882-uc-housing-element-renewal.changed.md @@ -0,0 +1 @@ +The `dwp.uc.households_housing_element` measure exclusion is renewed for one month, to 2026-11-14 (microcosm#882; renewed by juaristi22 on 2026-10-14 for the microcosm#1124 local release): the model still has no other/unknown tenure, so DWP's any-tenure housing-element count stays structurally above the bound social and private tenure rows. diff --git a/changelog.d/telemetry-emitter-startup-timeout.fixed.md b/changelog.d/telemetry-emitter-startup-timeout.fixed.md new file mode 100644 index 0000000000..93230214b2 --- /dev/null +++ b/changelog.d/telemetry-emitter-startup-timeout.fixed.md @@ -0,0 +1 @@ +The UK build's telemetry emitter adapter now waits up to 15 seconds, rather than 3, for the local emitter service to become ready. The service took about four seconds to start on an M1 Max even with a warm cache, so every build on such a machine fell back to local-only telemetry. The value is a ceiling: a faster start is not delayed. diff --git a/changelog.d/uk-full-inspection-export.added.md b/changelog.d/uk-full-inspection-export.added.md new file mode 100644 index 0000000000..964118f50d --- /dev/null +++ b/changelog.d/uk-full-inspection-export.added.md @@ -0,0 +1 @@ +`microcosm-build-uk --release-role dense --inspection-export` exports, packages and stages a candidate's H5 for inspection even when a structural full-build gate blocks the artifact. It is candidate-only (refused with `--release-candidate` and by the national role), records `inspection_export` in the request, the export descriptor and the manifest parameters, leaves the bundle unreleasable and keeps the blocked exit status. The kernel lives in `uk_runtime.inspection_export`, so default node keys are unchanged. diff --git a/docs/evidence/spec-engine/us-f0-coverage.json b/docs/evidence/spec-engine/us-f0-coverage.json index bbced0b031..e04a5d0d0a 100644 --- a/docs/evidence/spec-engine/us-f0-coverage.json +++ b/docs/evidence/spec-engine/us-f0-coverage.json @@ -1656,13 +1656,13 @@ "compiler_ir.node_slices" ], "expected": { - "map_sha256": "41a64b8820bf161e316c49e3f2b666a247b58f6f6d50dd4395b89c8f5b1399f6", - "protocol_sha256": "0e4c8a4ace5e10747065669a009691d0eee9179ca42b344f8912c5ca239ebde6" + "map_sha256": "1efa1551b0711e69fd89e5fbfa00384a209a158989bcc830b594b93a1c5a71fc", + "protocol_sha256": "73cd2b524a057e74620e48072edaf192d158e29b6358c6ff59af6eeac46a4269" }, "failures": [], "observed": { - "map_sha256": "41a64b8820bf161e316c49e3f2b666a247b58f6f6d50dd4395b89c8f5b1399f6", - "protocol_sha256": "0e4c8a4ace5e10747065669a009691d0eee9179ca42b344f8912c5ca239ebde6" + "map_sha256": "1efa1551b0711e69fd89e5fbfa00384a209a158989bcc830b594b93a1c5a71fc", + "protocol_sha256": "73cd2b524a057e74620e48072edaf192d158e29b6358c6ff59af6eeac46a4269" }, "status": "covered" }, @@ -1677,7 +1677,7 @@ "compiler_ir.seed_stream_map" ], "expected": { - "implementation_sha256": "0e4c8a4ace5e10747065669a009691d0eee9179ca42b344f8912c5ca239ebde6", + "implementation_sha256": "73cd2b524a057e74620e48072edaf192d158e29b6358c6ff59af6eeac46a4269", "protocol": "legacy-v1", "streams": [ "build_model", @@ -1698,7 +1698,7 @@ }, "failures": [], "observed": { - "implementation_sha256": "0e4c8a4ace5e10747065669a009691d0eee9179ca42b344f8912c5ca239ebde6", + "implementation_sha256": "73cd2b524a057e74620e48072edaf192d158e29b6358c6ff59af6eeac46a4269", "protocol": "legacy-v1", "streams": [ "build_model", @@ -2599,7 +2599,7 @@ "country": "us", "schema_id": "country_spec", "schema_version": 1, - "spec_sha256": "3080fe938df62216dc4853c1163639817de3c537e0e932309642fbd3f7d8f266" + "spec_sha256": "4d08ff77d2118ec4f7d63fcf17983f774b9d1142b0b8308bfdc30767ece76f1f" } }, "report_schema_version": 3, @@ -2609,7 +2609,7 @@ "country": "us", "schema_id": "country_spec", "schema_version": 1, - "spec_sha256": "3080fe938df62216dc4853c1163639817de3c537e0e932309642fbd3f7d8f266" + "spec_sha256": "4d08ff77d2118ec4f7d63fcf17983f774b9d1142b0b8308bfdc30767ece76f1f" }, "status": "pass" } diff --git a/docs/shared-constants.md b/docs/shared-constants.md index 8d76706522..069fef8aaa 100644 --- a/docs/shared-constants.md +++ b/docs/shared-constants.md @@ -36,6 +36,9 @@ and build packages. It currently defines: - `UK_REGION_TIER`, `UK_REGION_TIER_ENUM` and `UK_LADDER_NATION_REGION_CODES` (the twelve-area region tier UK national references fan out over, its spine `region` enum names, and the ladder's nation pseudo-codes; microcosm#905) +- `UK_WIDE_GEOGRAPHY_IDS` (the K-prefixed UK-wide aggregates among + `UK_GEOGRAPHY_ID_TO_LABEL`, which the nation grain of the UK target-weight + rules splits on; microcosm#1124) - `US_STATE_FIPS_TO_POSTAL` - `US_STATE_NUMERIC_FIPS_TO_POSTAL` - `US_STATE_POSTAL_TO_NUMERIC_FIPS` diff --git a/docs/uk-dataset-size-plan-355.md b/docs/uk-dataset-size-plan-355.md index 926d8d56dc..9926075819 100644 --- a/docs/uk-dataset-size-plan-355.md +++ b/docs/uk-dataset-size-plan-355.md @@ -8,6 +8,11 @@ on [#870](https://github.com/PolicyEngine/microcosm/pull/870)'s branch. The PR i ## Decisions retained +(2026-10-13: microcosm#1124 moved the local doctrine to 2,000 epochs and +`nation_grain_family_equal_sqrt_count`, kept K=15, and gave sized builds the +defaults π_hi 0.5, refit floor 0.5 and a uniform refit L2 penalty; see +`uk_runtime/local_doctrine.py`. The values below are this plan's, as decided.) + - Pool generation keeps clone count **K=15**. Requested output households is a different parameter, applied after cloning and materialization. - The joint surface retains national, constituency and local-authority rows, @@ -175,15 +180,22 @@ Three should-fix items and two questions changed the machinery: - **Zero-valued targets are counted.** `zero_target_rows` in the size receipt says how many compiled rows carry a zero target, the case where the contribution prior uses a unit denominator. -## Certification and publication still required - -The implementation produces **candidates**, not a new certified UK default. -A size request refuses `--release-candidate`, and its manifest records -`releasable=false` even if the diagnostic gate run passes. The dense assembler -must not interpret a compact candidate as the already reviewed dense line. -The existing registry, production pointers and pe.py default are unchanged. - -Before any size can be promoted: +## Certification and publication + +**Ruling of 2026-10-13 (microcosm#1124):** the local release ships the +100,000-household selection under the dense line's publication requirements +(every release-blocking gate, the rotated holdout, the pre-flight and the +incumbent-surface evaluation on one engine over the released file). The whole +pool is too large for one engine and for the consumers who load the file. A +release candidate may request exactly that size with the size doctrine +(`local_doctrine.uk_local_size_doctrine`); the pre-flight checks the size +receipt, and the assembler takes the sized candidate on the same contract. The +matched-size scorecard and promotion protocol below are no longer required for +that size; they stay the route for any other size, which remains candidate-only +with `releasable=false`. The existing registry, production pointers and pe.py +default are unchanged. + +Before any other size can be promoted: 1. Run the licensed full-input build, retain all gate failures and measure local ESS and fit. A nominal 50k size is not guaranteed to clear the floors diff --git a/docs/uk-dense-release-assembly-runbook-762.md b/docs/uk-dense-release-assembly-runbook-762.md index 4e6f378ffe..f9583000de 100644 --- a/docs/uk-dense-release-assembly-runbook-762.md +++ b/docs/uk-dense-release-assembly-runbook-762.md @@ -1,7 +1,7 @@ # UK dense line release assembly runbook (#762) The dense line `microcosm-uk-2024-25-dense` is the spine cloned K=15 times -through the OA geography ladder and calibrated to the national and local +through the atomic output-area assignment and calibrated to the national and local target surfaces in one solve (`microcosm-build-uk --release-role dense`, the graph driver `tools/build_uk_full.py`; `tools/build_uk_rowwise_candidate.py` is a stub over it). It ships @@ -10,6 +10,9 @@ on the **inspect lane only**: a constant release id, an immutable per-cut tag, `--no-latest` at publication, so it can never displace the default artifact. It is registered as `("uk", 2025, "dense")` in the private repo `policyengine/populace-uk-private`. Publication is a separate human step. +Since microcosm#1124 (ruling of 2026-10-13) the line may ship the +100,000-household exact-k selection from that pool instead of the whole pool, +under the same requirements; see [Sized release](#sized-release-microcosm1124). The R16/R17 release verdicts recorded in the historical receipts used the previous gate policy. They do not satisfy the current contract: four quality @@ -23,6 +26,12 @@ No historical run was re-signed or recalibrated by the PR #870 review fixes. - The Ledger consumer artifact, the spine H5 with its sidecar, the OA ladder. - The incumbent extraction (`tools/extract_uk_local_incumbent_surface.py`) for the head-to-head score. +- The licensed input-mass reference the national line keeps with its runs + (`--input-mass-reference`). `uk_input_mass_parity` is release-blocking on + this line too and records `evidence_absent` without it. The sidecar's totals + digest must match the registry's `totals_sha256`; the totals depend on the + installed policyengine-uk, so re-emitting them under a newer engine does not + reproduce the pinned file. ## 1. Pre-flight the environment @@ -43,15 +52,17 @@ uv run --no-sync python tools/build_uk_full.py --release-role dense \ --ladder build/uk/uk_oa_ladder_2021.npz --ladder-sha256 \ --ledger-facts --ledger-facts-sha256 \ --ledger-manifest-sha256 --seed 42 \ + --input-mass-reference \ --logbook-prev-row-digest --out ``` `--release-role dense` is required (microcosm#823): the role supplies the -solve defaults (K=15, seed 42, 1,500 epochs, learning rate 0.15, -`grain_equal`), names the outputs `microcosm_uk_2024_25_local.h5` and +solve defaults (K=15, seed 42, 2,000 epochs, learning rate 0.15, +`nation_grain_family_equal_sqrt_count`), names the outputs `microcosm_uk_2024_25_local.h5` and `microcosm_uk_2024_25_local.local_gates.json`, records itself in the manifest, and refuses the national role's flags. `--release-candidate` pins -the doctrine (bound 10, `grain_equal`, K=15, 1500 epochs) and runs the +the doctrine (bound 10, `nation_grain_family_equal_sqrt_count`, K=15, 2000 +epochs) and runs the rotated holdout. The engine may resolve in a single block or per clone (`--engine-blocks K`): each block's engine weights are scaled to the pool so weight-share formulas see the pool's denominator, and the measures receipt @@ -59,8 +70,14 @@ records whether the blocks were identical copies (an exact representation); a per-block run is releasable only when it was. Best-effort staging telemetry uploads every 300 s by default on this driver (the Hub allows about 128 commits per hour per repository). -Expect about 3.5 hours and 10 GB at K=15 (measured on the rowwise tool; the -graph driver's dense build has not been timed yet). Since microcosm#901 the +For scale: at K=25 with `--engine-blocks 25` the graph driver's dense solve took +about 3 hours (2,000 epochs over 1.6 million households, M1 Max), each of three +size-search probes about as long, and the build peaked at 24.7 GiB (the +2026-10-11 100,000-household candidate). At K=15 the pool is 60 % of that size, +and the rowwise tool measured about 3.5 hours and 10 GB for an unsized build. +A release candidate also runs the rotated holdout, which repeats the dense +solve in each of its five folds, and in a sized build the size search and +refit too. Since microcosm#901 the dense role runs through the [full-build graph](uk-full-build-graph.md): the candidate directory also holds `candidate.json`, `certification.json`, `operations.json` and the `build.json` completion marker, the graph store sits @@ -68,6 +85,21 @@ at `/.graph-store` unless `--graph-store` says otherwise, and `--input-h5` must be a spine whose gate manifest matches the branch's gate declarations. +### Sized release (microcosm#1124) + +A release candidate may add `--dataset-households 100000` to ship the exact-k +selection instead of the whole pool. It must carry the local size doctrine +(`local_doctrine.uk_local_size_doctrine`): 100,000 households, an exact-count +draw at π_hi 0.5, the refit baseline floored at 0.5 and a chi-square refit L2 +penalty toward uniform weights at λ 0.0216, drawn with the solve seed. The role +fills those values when the request leaves them unset and refuses any other. +`--selection-initial-lambda` warm-starts the budget search, and every holdout +fold's search starts from the same value. The requirements are the dense +line's (ruling of 2026-10-13; no matched-size scorecard): every +release-blocking gate passed, the rotated holdout, the pre-flight, which also +checks the size receipt, and the incumbent-surface evaluation on one engine +over the 100,000-household file. The release identity is unchanged. + ## 3. Pre-flight the finished run, then score it ```bash @@ -75,10 +107,17 @@ uv run --no-sync python tools/preflight_uk_local_release_candidate.py --candidat uv run --no-sync python tools/score_uk_local_candidate.py ... --output-json /score_vs_incumbent.json ``` +The scorer's `--registry-json` is the candidate's target selection, +`/microcosm_uk_2024_25_local.target_selection.json` (the +manifest's `outputs.target_registry`); it scores that selection's local rows, +which must be the 21,804 compiled local references of the #1123 surface and +must all appear in the candidate's diagnostics, so scoring needs the full rung. + The pre-flight checks the manifest and the signed gate report for everything the contract will demand: release posture attested, shippable, every release-blocking gate passed, an exact engine population (single block, or -per-block with identical copies scaled to the pool), the doctrine values, the +per-block with identical copies scaled to the pool), the doctrine values (for +a sized run, the size doctrine and a realized count equal to the request), the A15 census household uprating and A17 tenure application, the measure exclusions and their windows, the holdout, the Logbook row, the artifact digest. @@ -98,6 +137,17 @@ uv run --no-sync python tools/evaluate_uk_incumbent_surface.py \ --out-md /incumbent_surface_evaluation.md ``` +This evaluation is also where the released file's fit is gated on the engine +a consumer runs. The build's release-blocking fit gates read the measures it +resolved, per clone when `--engine-blocks K`; the build records that +representation as exact only for the weight-share scaling +(`engine_population_representation.exact_scope`), and engine draws that depend +on a record's position can differ between per-clone and single-block runs. The +evaluator re-measures the released file on one engine, which the data contract +requires, and its assessment fails release for any national or local row more +than 25% from its target, or any family of five or more rows with fewer than +half within 25%. Signed deferrals and measure exclusions do not exempt a row. + Use the actual metrics and weights filenames from the extraction manifest. The evaluator remains diagnostic: missing optional incumbent inputs and poor fit produce a failed assessment, not permission to publish. Only one engine diff --git a/docs/uk-full-build-graph.md b/docs/uk-full-build-graph.md index bc11e51252..f460df4894 100644 --- a/docs/uk-full-build-graph.md +++ b/docs/uk-full-build-graph.md @@ -50,7 +50,7 @@ To include raw spine construction in the same execution, pass `--spine-request / The request declares existing source adapters and lazy transforms; the spine stages execute in the same graph, and the driver supplies `--spine-h5` when the request omits it. A spine request has no checkpoint H5 to pin, so `--input-sha256` is not required on that path. The spine-only command remains available to materialise an execution checkpoint. -For smaller outputs, add `--dataset-households 100000` to use the common informed L0 search, exact-count draw and refit. `--baseline-pi-floor` floors the inclusion probability the refit's Horvitz-Thompson baseline divides each selected row's dense weight by (0, the default, is the untrimmed baseline), and `--no-size-checkpoint` skips materialising `size_selection_checkpoint.{npz,json}` into `--out`. `--n-clones K` sets the number of geographic copies in the pool; `--candidate-clone-counts 5,10,15` with `--dry-run` prints the compiled operation inventory for each K without solving. K and k are independent, and neither narrows target scope. A country-only run may explicitly request a smaller K, but this is never inferred. `--households-only` binds only the Chronicle census-household constituency targets (a target-selection family filter). +For smaller outputs, add `--dataset-households 100000` to use the common informed L0 search, exact-count draw and refit. A sized dense build takes the doctrine's size defaults where the request leaves them unset (microcosm#1124): the exact-count draw at `--selection-pi-hi 0.5`, the refit baseline floored at 0.5 and a chi-square refit L2 penalty toward uniform weights at λ 0.0216 (`--refit-l2-lambda 0` turns it off). At 100,000 households with exactly those settings a sized build can be a release candidate and releasable; any other size stays candidate-only (see the [dense runbook](uk-dense-release-assembly-runbook-762.md#sized-release-microcosm1124)). `--baseline-pi-floor` floors the inclusion probability the refit's Horvitz-Thompson baseline divides each selected row's dense weight by (0, the default, is the untrimmed baseline), and `--no-size-checkpoint` skips materialising `size_selection_checkpoint.{npz,json}` into `--out`. `--n-clones K` sets the number of geographic copies in the pool; `--candidate-clone-counts 5,10,15` with `--dry-run` prints the compiled operation inventory for each K without solving. K and k are independent, and neither narrows target scope. A country-only run may explicitly request a smaller K, but this is never inferred. `--households-only` binds only the Chronicle census-household constituency targets (a target-selection family filter). `--sample-fraction` samples before geographic cloning. Raw-spine sampling in the JSON request instead occurs at the original ingest boundary, before enrichment. An already sampled spine cannot be sampled a second time. The effective sample fraction controls development gate and target-admission policy. diff --git a/docs/uk-staging-operations.md b/docs/uk-staging-operations.md index a71bb3de3f..3d4b1ab4b9 100644 --- a/docs/uk-staging-operations.md +++ b/docs/uk-staging-operations.md @@ -187,7 +187,7 @@ rejected before remote storage is called. ## Staged datasets Telemetry is not the dataset. The build command (`microcosm-build-uk`; dense -K=15, exact-count `--dataset-households` and national runs alike) also stages the bundle its +exact-count `--dataset-households` and national runs alike) also stages the bundle its manifest vouches for, so a run can be inspected by the team without being published. The two destinations share one run id: diff --git a/packages/microcosm-build/src/microcosm/build/spec_engine/inventory_coverage.py b/packages/microcosm-build/src/microcosm/build/spec_engine/inventory_coverage.py index b68874986d..7e1b6e05f1 100644 --- a/packages/microcosm-build/src/microcosm/build/spec_engine/inventory_coverage.py +++ b/packages/microcosm-build/src/microcosm/build/spec_engine/inventory_coverage.py @@ -361,9 +361,11 @@ "primary_tuples": "fdf23da429f8c501e198a2f5b719763d523a41565c2153fdf17b0344f21bcd7c", # #1178: puf_support's source bytes attest CPS Social Security preservation. # RNG rules and the ordered QRF targets are unchanged. microcosm#104 moves - # both again: the protocol attests microcosm.calibrate.solve. - "seed_map": "41a64b8820bf161e316c49e3f2b666a247b58f6f6d50dd4395b89c8f5b1399f6", - "seed_protocol": "0e4c8a4ace5e10747065669a009691d0eee9179ca42b344f8912c5ca239ebde6", + # both again: the protocol attests microcosm.calibrate.solve. microcosm#1124 + # moves both once more: solve gains the opt-in retain_best_objective, off + # by default, so no US solve, RNG rule or QRF target changes. + "seed_map": "1efa1551b0711e69fd89e5fbfa00384a209a158989bcc830b594b93a1c5a71fc", + "seed_protocol": "73cd2b524a057e74620e48072edaf192d158e29b6358c6ff59af6eeac46a4269", "source_manifest": "f4f8bfeb79268a4043a843b47cec1d1c0e58e9f5dd6bc23417ada95446dd7b8e", "take_up": "9522ce40f7dea569312dd7a7beb474e5a5afadd2f1e10cb5876534c3ef623d35", "tail": "ac92829c88a1a4fb6460d61190918d5d99c6c377fc8dd8f62f02b332d09bf59c", diff --git a/packages/microcosm-build/src/microcosm/build/telemetry_emitter.py b/packages/microcosm-build/src/microcosm/build/telemetry_emitter.py index 559e77838e..31f3c893ef 100644 --- a/packages/microcosm-build/src/microcosm/build/telemetry_emitter.py +++ b/packages/microcosm-build/src/microcosm/build/telemetry_emitter.py @@ -66,7 +66,9 @@ def start( run_kind: str = "build", development_collector_url: str | None = None, heartbeat_seconds: float = DEFAULT_HEARTBEAT_SECONDS, - startup_timeout_seconds: float = 3.0, + # The service took about four seconds to become ready on an M1 Max, + # past the provider's three-second default; this is only a ceiling. + startup_timeout_seconds: float = 15.0, spool_path: Path | str | None = None, close_service_on_finish: bool = True, ) -> LocalTelemetryEmitter: diff --git a/packages/microcosm-build/src/microcosm/build/uk/calibration_measure_exclusions.json b/packages/microcosm-build/src/microcosm/build/uk/calibration_measure_exclusions.json index 913ec73e06..cdc2ca5b6d 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/calibration_measure_exclusions.json +++ b/packages/microcosm-build/src/microcosm/build/uk/calibration_measure_exclusions.json @@ -420,9 +420,9 @@ "reason": "Structural bias, not a weight question: DWP's any-tenure housing-entitlement count is social plus private plus 'Other or unknown' tenure, and the model has no other/unknown tenure (uc_housing_costs_element pays only council, housing-association and private renters), so the model's any-tenure count is identically the social plus private rows, which are bound and fit; the row overstates the reachable total by the other/unknown cell, 112,518 of 4,037,650 in calendar 2025 (2.8 percent; the published any-tenure mean, which the three tenure cells reproduce to the unit). Fitting all three housing rows would ask the solver to over-fill the tenure rows by that amount. Measured on every evaluation as the housing-element caseload diagnostic (Q50f -0.8 percent, D2 +0.6 percent at the baseline; v20 -2.8 percent unbound, the structural gap itself, docs/evidence/uk-a16-v20/).", "tracking": "microcosm#882", "approved_by": "juaristi22", - "adjudication": "microcosm#882 (element sizing at the 10 September 2026 baseline, docs/uk-uc-baseline-2026-09-10.md; María's sign-off in review of microcosm#921 on 2026-09-15, setting the one-month window)", - "approved_on": "2026-09-15", - "expires_on": "2026-10-15" + "adjudication": "microcosm#882 (element sizing at the 10 September 2026 baseline, docs/uk-uc-baseline-2026-09-10.md; María's sign-off in review of microcosm#921 on 2026-09-15, setting the one-month window); renewed one month on 2026-10-14 by juaristi22 in session (microcosm#1124 local release): the model still has no other/unknown tenure, so the any-tenure count stays structurally above the bound tenure rows", + "approved_on": "2026-10-14", + "expires_on": "2026-11-14" }, { "name": "dwp.hb.households", diff --git a/packages/microcosm-build/src/microcosm/build/uk/degenerate_reviewed_exclusions.json b/packages/microcosm-build/src/microcosm/build/uk/degenerate_reviewed_exclusions.json index 6ddd4fc6e0..9cfe129d7b 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/degenerate_reviewed_exclusions.json +++ b/packages/microcosm-build/src/microcosm/build/uk/degenerate_reviewed_exclusions.json @@ -1,7 +1,17 @@ { "schema_version": 2, - "description": "Reviewed degenerate-release-surface exclusions for the UK terminal battery (#630, schema 2 per #610). Each entry is a complete approval receipt: reason, approver, adjudication reference, approval date, and expiry. The gate reports dormant entries, FAILS stale ones (columns that regain signal), and FAILS expired ones until the adjudication is renewed, so this register cannot rot.", + "description": "Reviewed degenerate-release-surface exclusions for the UK terminal battery (#630, schema 2 per #610). Each entry is a complete approval receipt: reason, approver, adjudication reference, approval date, and expiry. The gate reports dormant entries, FAILS stale ones (columns that regain signal), and FAILS expired ones until the adjudication is renewed, so this register cannot rot. An entry may carry release_lines, the release lines whose evidence earned it (national, local); on any other line it is out of line: it neither suppresses a failure nor reads as stale, and the gate details name it.", "exclusions": { + "household.atomic_area_basis": { + "reason": "Constant by construction on the local line: the UK assignment spec draws every household's atomic area from census support and declares no observed-area system (microcosm#932), so the shared operator's basis output is always 'assigned'. Retained deliberately: it tells a consumer that every area code is a modelled assignment, not an observed address. Not signal.", + "approved_by": "juaristi22", + "adjudication": "microcosm#1124 local release blockers (ruling 2026-10-13 in session; the assignment columns stay at export per microcosm#932)", + "approved_on": "2026-10-13", + "expires_on": "2027-04-13", + "release_lines": [ + "local" + ] + }, "household.source_year": { "reason": "Constant at the build vintage by construction on a single-vintage artifact, and fully derivable from the artifact's time_period metadata (the rowwise clone path re-derives it when absent). Retained deliberately for row-level vintage documentation and UK/US schema symmetry \u2014 the US pools three ASEC vintages, where the same column is irreducible per-row identity. Not signal.", "approved_by": "juaristi22", diff --git a/packages/microcosm-build/src/microcosm/build/uk/qrf_tail_reviewed_exclusions.json b/packages/microcosm-build/src/microcosm/build/uk/qrf_tail_reviewed_exclusions.json index c95b8a07b3..88a1f7879f 100644 --- a/packages/microcosm-build/src/microcosm/build/uk/qrf_tail_reviewed_exclusions.json +++ b/packages/microcosm-build/src/microcosm/build/uk/qrf_tail_reviewed_exclusions.json @@ -1,13 +1,36 @@ { "schema_version": 2, - "description": "Reviewed QRF tail-concentration exclusions for the UK terminal battery (#609). Each entry maps a declared fit_weighted_qrf_stage* output column to a non-empty reason naming the tracked defect or the genuinely concentrated instrument. The shared gate reports dormant entries (columns absent from the declared surface) and FAILS stale ones (columns now below the concentration threshold), so this register cannot rot. Empty until an incident or adjudication earns an entry.", + "description": "Reviewed QRF tail-concentration exclusions for the UK terminal battery (#609). Each entry maps a declared fit_weighted_qrf_stage* output column to a non-empty reason naming the tracked defect or the genuinely concentrated instrument. The shared gate reports dormant entries (columns absent from the declared surface) and FAILS stale ones (columns now below the concentration threshold), so this register cannot rot. Empty until an incident or adjudication earns an entry. An entry may carry release_lines, the release lines whose evidence earned it (national, local); on any other line it is out of line: it neither suppresses a failure nor reads as stale, and the gate details name it.", "exclusions": { "charitable_investment_gifts": { - "reason": "A genuinely concentrated instrument: the SPI stage-1 QRF leaf for gifts of investments to charity has 168 weighted carriers on the 2026-09-30 national build, so its top 100 weighted records carry the whole mass by construction (the published SPI population itself is a few thousand donors of listed shares and property). Not the microcosm#462 broadcast signature: the carriers are SPI-channel rows drawn from the tape's own sparse leaf, not a point mass copied across a donor tail. An earlier entry for this column was retired on 2026-09-21 when the first national candidate measured it inside the fence; the 2026-09-30 build measures it at the threshold again, and the gate fails this entry as stale the day a build brings it back inside.", + "reason": "A genuinely concentrated instrument: the SPI stage-1 QRF leaf for gifts of investments to charity has few weighted carriers on the 2026-09-30 national build, so its top 100 weighted records carry nearly the whole mass by construction (the published SPI population itself is a small group of donors of listed shares and property). Not the microcosm#462 broadcast signature: the carriers are SPI-channel rows drawn from the tape's own sparse leaf, not a point mass copied across a donor tail. An earlier entry for this column was retired on 2026-09-21 when the first national candidate measured it inside the fence; the 2026-09-30 build measures it at the threshold again, and the gate fails this entry as stale the day a build brings it back inside. The re-arm on independent draws is microcosm#1277.", "approved_by": "juaristi22", "adjudication": "microcosm#1063 c9 (certifier rehearsal R5 on the 2026-09-30 build, 2026-10-01; ruling 2026-10-02 in session)", "approved_on": "2026-10-02", - "expires_on": "2027-04-02" + "expires_on": "2027-04-02", + "release_lines": [ + "national" + ] + }, + "hmrc_spi_other_income": { + "reason": "A genuinely concentrated instrument: a real stage-1 draw, rare on the SPI tape itself, whose top 100 records carry nearly its whole mass whether the gate counts rows or independent records. Not the microcosm#462 donor-tail signature: the carriers are SPI-channel rows drawn from the tape's own sparse leaf. The CGT clones and SPI support copies take it past the carrier floor, so the row count also overstates its support. Deferred, time-boxed, until the gate is re-armed on independent draws (microcosm#1277).", + "approved_by": "juaristi22", + "adjudication": "national release-cut certification of 2026-10-14 (ruling 2026-10-14 in session; microcosm#1124, PR #1242); the re-arm on independent draws, and this entry's renewal or retirement, is microcosm#1277", + "approved_on": "2026-10-14", + "expires_on": "2027-01-14", + "release_lines": [ + "national" + ] + }, + "hmrc_spi_taxable_termination_pay": { + "reason": "A genuinely concentrated instrument: a real stage-1 draw, rare on the SPI tape itself, whose top 100 records carry nearly its whole mass whether the gate counts rows or independent records. Not the microcosm#462 donor-tail signature: the carriers are SPI-channel rows drawn from the tape's own sparse leaf. The CGT clones and SPI support copies take it past the carrier floor, so the row count also overstates its support. Deferred, time-boxed, until the gate is re-armed on independent draws (microcosm#1277).", + "approved_by": "juaristi22", + "adjudication": "national release-cut certification of 2026-10-14 (ruling 2026-10-14 in session; microcosm#1124, PR #1242); the re-arm on independent draws, and this entry's renewal or retirement, is microcosm#1277", + "approved_on": "2026-10-14", + "expires_on": "2027-01-14", + "release_lines": [ + "national" + ] } } } diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/__init__.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/__init__.py index 2423a2b9fa..d25e2c22d0 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/__init__.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/__init__.py @@ -310,10 +310,12 @@ from microcosm.build.uk_runtime.local_doctrine import ( UK_LOCAL_CLONE_COUNT, UK_LOCAL_MAX_WEIGHT_RATIO, + UK_LOCAL_ROW_METADATA_RULES, UK_LOCAL_SOLVE_DOCTRINE, UK_LOCAL_SOLVE_EPOCHS, UK_LOCAL_TARGET_LOSS_CAP, UK_LOCAL_TARGET_WEIGHT_RULE, + UK_LOCAL_TARGET_WEIGHT_RULES, UKLocalSolveDoctrine, uk_local_doctrine_with_overrides, uk_local_target_loss_weights, @@ -607,6 +609,8 @@ "UK_LOCAL_MAX_WEIGHT_RATIO", "UK_LOCAL_SOLVE_EPOCHS", "UK_LOCAL_TARGET_WEIGHT_RULE", + "UK_LOCAL_TARGET_WEIGHT_RULES", + "UK_LOCAL_ROW_METADATA_RULES", "UK_LOCAL_BINDING_ADJUDICATION_REGISTER_RESOURCE", "UK_LOCAL_HOLDOUT_FOLDS", "UK_LOCAL_HOLDOUT_SEED", diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/battery_bindings.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/battery_bindings.py index 0255dc5f46..3ef6d827b8 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/battery_bindings.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/battery_bindings.py @@ -110,6 +110,7 @@ UK_INPUT_MASS_EXCLUSION_REGISTER_RESOURCE, UK_INPUT_MASS_REFERENCE_REGISTRY, UK_QRF_TAIL_EXCLUSION_REGISTER_RESOURCE, + UK_RELEASE_LINES, UK_TARGET_FIT_EXCLUSION_REGISTER_RESOURCE, UKInputMassParityPolicy, UKQRFTailConcentrationPolicy, @@ -959,6 +960,46 @@ def _exclusion_clock(context: EvidenceContext) -> date: return clock +def _release_line(context: EvidenceContext) -> str | None: + """The release line the gate context declares (``release_line``), if any.""" + + line = context.artifacts.get("release_line") + if line is not None and line not in UK_RELEASE_LINES: + raise ValueError( + f"release_line must be one of {sorted(UK_RELEASE_LINES)}, got {line!r}." + ) + return line + + +def _in_line( + records: Mapping[str, UKReviewedExclusion], context: EvidenceContext +) -> tuple[Mapping[str, UKReviewedExclusion], list[str]]: + """The records in scope on the context's line, and the names out of line.""" + + line = _release_line(context) + applied = { + name: record for name, record in records.items() if record.applies_on(line) + } + return MappingProxyType(applied), sorted(set(records) - set(applied)) + + +def _with_out_of_line( + result: GateResult, context: EvidenceContext, out_of_line: list[str] +) -> GateResult: + """Record the register entries a line-scoped evaluation set aside.""" + + if not out_of_line: + return result + return replace( + result, + details={ + **dict(result.details), + "release_line": _release_line(context), + "out_of_line_exclusions": out_of_line, + }, + ) + + def _resolve_degenerate_exclusions( context: EvidenceContext, ) -> tuple[Mapping[str, UKReviewedExclusion], str]: @@ -1073,7 +1114,8 @@ def _evaluate_degenerate_release_surface( f"runtime loads {UK_DEGENERATE_EXCLUSION_REGISTER_RESOURCE!r}." ) resolved, _source = _resolve_degenerate_exclusions(context) - return uk_degenerate_release_surface_gate( + resolved, out_of_line = _in_line(resolved, context) + result = uk_degenerate_release_surface_gate( _uk_gate_surface(context.frame), reviewed_exclusions=resolved, now=_exclusion_clock(context), @@ -1083,6 +1125,7 @@ def _evaluate_degenerate_release_surface( dropped_at_export=UK_RELEASE_EXPORT_DROPPED_COLUMNS, **kwargs, ) + return _with_out_of_line(result, context, out_of_line) def _degenerate_exclusions_evidence( @@ -1531,13 +1574,15 @@ def _evaluate_target_fit( f"runtime loads {UK_TARGET_FIT_EXCLUSION_REGISTER_RESOURCE!r}." ) exclusions, _source = _resolve_target_fit_exclusions(context) + exclusions, out_of_line = _in_line(exclusions, context) parity = context.artifacts["parity_evidence"] - return uk_target_fit_gate( + result = uk_target_fit_gate( parity.target_relative_errors, reviewed_exclusions=exclusions, now=_exclusion_clock(context), **kwargs, ) + return _with_out_of_line(result, context, out_of_line) def _target_fit_required_artifacts( @@ -1579,6 +1624,7 @@ def _evaluate_input_mass_parity( exclusions, _source = _resolve_input_mass_exclusions( context, reference=reference_name ) + exclusions, out_of_line = _in_line(exclusions, context) policy = UKInputMassParityPolicy( relative_tolerance=relative_tolerance, minimum_reference_total=minimum_reference_total, @@ -1590,7 +1636,7 @@ def _evaluate_input_mass_parity( # carries the spine's WAS property_wealth, which the reference also # stores, so comparing it there would call the signed exclusion stale # (microcosm#1106). - return uk_input_mass_parity_gate( + result = uk_input_mass_parity_gate( uk_input_mass_totals(uk_release_export_frame(context.frame)), context.artifacts["input_mass_reference"], descriptor=coerced_registry[reference_name], @@ -1598,6 +1644,7 @@ def _evaluate_input_mass_parity( now=_exclusion_clock(context), **kwargs, ) + return _with_out_of_line(result, context, out_of_line) def _input_mass_reference_evidence( @@ -1630,6 +1677,7 @@ def _evaluate_tail_concentration( f"runtime loads {UK_QRF_TAIL_EXCLUSION_REGISTER_RESOURCE!r}." ) exclusions, _source = _resolve_qrf_tail_exclusions(context) + exclusions, out_of_line = _in_line(exclusions, context) policy = UKQRFTailConcentrationPolicy( top_k=top_k, max_top_share=max_top_share, @@ -1657,7 +1705,7 @@ def _evaluate_tail_concentration( ], "export_dropped_checked_on_spine": sorted(checked_on_spine), } - return uk_qrf_tail_concentration_gate( + result = uk_qrf_tail_concentration_gate( values, weights, policy=policy, @@ -1665,6 +1713,7 @@ def _evaluate_tail_concentration( now=_exclusion_clock(context), **kwargs, ) + return _with_out_of_line(result, context, out_of_line) def _evaluate_ledger_compile_parity( diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/dataset_size.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/dataset_size.py index 71a264b8da..0d7f5d3d7e 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/dataset_size.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/dataset_size.py @@ -2,7 +2,7 @@ import hashlib import json -from collections.abc import Callable +from collections.abc import Callable, Mapping from dataclasses import dataclass, replace from typing import Any @@ -10,17 +10,23 @@ import pandas as pd from scipy import sparse +from microcosm.build.uk_runtime.target_weights import UKStageTargetWeighting from microcosm.calibrate import ( CalibrationResult, TargetSet, assert_exact_k_support, calibrate, + chi_square_distance, exact_k_design_feasibility, refit_l0_selection, + relative_error_loss, select_exact_k, ) from microcosm.calibrate.initialization import contribution_initialization -from microcosm.calibrate.solve import BUDGET_BASIS_OPEN_PROBABILITY_MASS +from microcosm.calibrate.solve import ( + BUDGET_BASIS_OPEN_PROBABILITY_MASS, + L2_BASIS_CHI_SQUARE, +) from microcosm.frame import Frame @@ -67,6 +73,34 @@ class UKSizeDraw: pi_hi: float probabilities_sha256: str + @classmethod + def from_payload( + cls, payload: bytes | str | Mapping[str, object], *, problem_sha256: str + ) -> "UKSizeDraw | None": + """Rebuild a stored draw artifact; ``None`` for a full-pool draw. + + The graph's size-draw node writes this JSON; the refit node and the + size-experiment harness read it back through this one parser. + """ + + draw = dict(payload if isinstance(payload, Mapping) else json.loads(payload)) + if draw.pop("problem_sha256", None) != problem_sha256: + raise ValueError("Exact-count draw belongs to another ordered problem.") + method = draw.pop("method", None) + if method == "full_pool": + return None + if method != "exact_count": + raise ValueError("Compact refit requires a completed exact-count draw.") + return cls( + **{ + **draw, + "support": np.asarray(draw["support"]), + "inclusion_probabilities": np.asarray( + draw["inclusion_probabilities"], dtype=np.float64 + ), + } + ) + def _probability_digest(probabilities: np.ndarray) -> str: return hashlib.sha256(np.asarray(probabilities, dtype=" float: return value +#: The L2 anchors a UK size stage admits (microcosm#1124). ``initial`` is the +#: stage's own starting weights: the pool design weights in the search, the +#: normalized (optionally floored) Horvitz–Thompson baseline in the refit. +#: ``uniform`` is their mean, a direct Kish-ESS control. +UK_SIZE_L2_ANCHORS = ("initial", "uniform") +#: The L2 basis a UK size stage admits. Under the UK's free mass the record +#: basis has its target-free optimum at every weight zero, so it only shrinks +#: mass; the chi-square basis is zero at the anchor (penalized GREG). +UK_SIZE_L2_BASES = (L2_BASIS_CHI_SQUARE,) +UK_SIZE_L2_STAGES = ("selection", "refit") +#: The flat keyword names a size stage's L2 travels under (graph node +#: parameters and the size functions' keywords). +UK_SIZE_L2_PARAM_KEYS = tuple( + f"{stage}_l2_{field}" + for stage in UK_SIZE_L2_STAGES + for field in ("lambda", "anchor", "basis") +) + + +@dataclass(frozen=True) +class UKSizeL2: + """One size stage's L2 concentration penalty; an instance means it is on.""" + + stage: str + l2_lambda: float + anchor: str = "initial" + basis: str = L2_BASIS_CHI_SQUARE + + def __post_init__(self) -> None: + if self.stage not in UK_SIZE_L2_STAGES: + raise ValueError(f"L2 stage must be one of {UK_SIZE_L2_STAGES}.") + value = self.l2_lambda + if ( + isinstance(value, bool) + or not isinstance(value, int | float) + or not np.isfinite(value) + or value <= 0.0 + ): + raise ValueError(f"{self.stage}_l2_lambda of an L2 penalty must be > 0.") + if self.anchor not in UK_SIZE_L2_ANCHORS: + raise ValueError( + f"{self.stage}_l2_anchor must be one of {UK_SIZE_L2_ANCHORS}, " + f"got {self.anchor!r}." + ) + if self.basis not in UK_SIZE_L2_BASES: + raise ValueError( + f"{self.stage}_l2_basis must be one of {UK_SIZE_L2_BASES}, " + f"got {self.basis!r}." + ) + object.__setattr__(self, "l2_lambda", float(value)) + + def solver_kwargs(self) -> dict[str, object]: + return { + "l2_lambda": self.l2_lambda, + "l2_anchor": self.anchor, + "l2_basis": self.basis, + } + + def params(self) -> dict[str, object]: + """The flat keywords (node parameters) this penalty travels under.""" + + return { + f"{self.stage}_l2_lambda": self.l2_lambda, + f"{self.stage}_l2_anchor": self.anchor, + f"{self.stage}_l2_basis": self.basis, + } + + def as_dict(self) -> dict[str, object]: + return {"lambda": self.l2_lambda, "anchor": self.anchor, "basis": self.basis} + + +def uk_size_l2( + stage: str, + *, + l2_lambda: object = 0.0, + anchor: object = None, + basis: object = None, +) -> UKSizeL2 | None: + """Validate one stage's flat L2 settings; ``None`` when the penalty is off. + + ``l2_lambda`` 0 (the default) is off, and then no anchor or basis may be + given. The record basis and a ``"design"`` anchor are refused with their + reasons: the first only shrinks mass under free mass; the second is, on + the exact-count refit, the same vector as ``"initial"`` (the normalized + Horvitz–Thompson baseline), and in the search ``"initial"`` already names + the pool design weights. + """ + + if stage not in UK_SIZE_L2_STAGES: + raise ValueError(f"L2 stage must be one of {UK_SIZE_L2_STAGES}.") + if ( + isinstance(l2_lambda, bool) + or not isinstance(l2_lambda, int | float) + or not np.isfinite(l2_lambda) + or l2_lambda < 0.0 + ): + raise ValueError(f"{stage}_l2_lambda must be a finite number >= 0.") + if float(l2_lambda) == 0.0: + if anchor is not None or basis is not None: + raise ValueError( + f"{stage} L2 anchor or basis given without a positive " + f"{stage}_l2_lambda." + ) + return None + anchor = "initial" if anchor is None else str(anchor) + basis = L2_BASIS_CHI_SQUARE if basis is None else str(basis) + if anchor == "design": + raise ValueError( + f"{stage}_l2_anchor 'design' is not a size-stage anchor: on the " + "exact-count refit it is the same vector as 'initial' (the " + "normalized Horvitz–Thompson baseline), and in the search 'initial' " + "already names the pool design weights." + ) + if basis == "record": + raise ValueError( + f"{stage}_l2_basis 'record' is refused under the UK's free mass: its " + "target-free optimum is every weight at zero, so it only shrinks " + "mass. Use 'chi_square'." + ) + return UKSizeL2(stage, float(l2_lambda), anchor, basis) + + +def uk_size_l2_from_options(options: Mapping[str, object]) -> dict[str, object] | None: + """The L2 penalty a solved stage recorded in its options (``None`` when off).""" + + value = float(options.get("l2_lambda", 0.0) or 0.0) + if value == 0.0: + return None + return { + "lambda": value, + "anchor": str(options.get("l2_anchor", "initial")), + "basis": str(options.get("l2_basis", "record")), + } + + +def _l2_solver_kwargs(l2: UKSizeL2 | None) -> dict[str, object]: + # Off means no L2 keyword reaches the solver: its recorded defaults (and + # every byte of a default run) stay what they were before #1124. + return {} if l2 is None else l2.solver_kwargs() + + +def _stage_loss_weights( + dense: CalibrationResult, weighting: object +) -> np.ndarray | None: + """A stage's loss weights from a named rule, or ``None`` (the dense's).""" + + if weighting is None: + return None + if not isinstance(weighting, UKStageTargetWeighting): + raise TypeError( + "a size stage's target weighting must be a UKStageTargetWeighting " + "(a named rule on the problem's rows), never a raw vector." + ) + return weighting.loss_weights(dense.problem.names) + + def _solver_common( - dense: CalibrationResult, *, epochs: int, learning_rate: float, seed: int + dense: CalibrationResult, + *, + epochs: int, + learning_rate: float, + seed: int, + target_loss_weights: np.ndarray | None = None, ) -> dict[str, Any]: return dict( weight_entity="household", @@ -214,7 +409,11 @@ def _solver_common( mass=dense.options["mass"], max_weight_ratio=dense.options["max_weight_ratio"], seed=seed, - target_loss_weights=dense.target_loss_weights, + target_loss_weights=( + dense.target_loss_weights + if target_loss_weights is None + else target_loss_weights + ), target_loss_scales=dense.target_loss_scales, target_loss_cap=dense.target_loss_cap, ) @@ -230,6 +429,10 @@ def select_uk_dataset_size( seed: int, pi_hi: float = 1.0, initial_lambda: float | None = None, + selection_l2_lambda: float = 0.0, + selection_l2_anchor: str | None = None, + selection_l2_basis: str | None = None, + target_weighting: UKStageTargetWeighting | None = None, progress_callback: ProgressCallback | None = None, ) -> UKSizeSelection: """Run the informed L0 budget search for an exact-count draw of ``households``. @@ -247,10 +450,23 @@ def select_uk_dataset_size( every probe, so a stale hint costs probes, never feasibility; any penalty whose draw lands inside the budget window is a valid stop, so a warm and a cold search can settle on different penalties and select different rows. + + ``selection_l2_*`` add a chi-square L2 penalty on the search's pre-gate + weights (microcosm#1124; see :func:`uk_size_l2`); off by default, and an + off penalty passes no L2 keyword to the solver. ``target_weighting`` + searches under a named rule other than the dense solve's weights (the + size-experiment seam; graph builds never pass it). """ n = _check_size_inputs(frame, dense, households) pi_hi = _check_pi_hi(pi_hi) initial_lambda = _check_initial_lambda(initial_lambda) + selection_l2 = uk_size_l2( + "selection", + l2_lambda=selection_l2_lambda, + anchor=selection_l2_anchor, + basis=selection_l2_basis, + ) + search_weights = _stage_loss_weights(dense, target_weighting) if households == n: raise ValueError("a full-pool size needs no selection.") problem = dense.problem @@ -289,7 +505,14 @@ def select_uk_dataset_size( feasible_draw_pi_hi=pi_hi, l0_lambda=0.0 if initial_lambda is None else initial_lambda, progress_callback=_phased(progress_callback, "size_search"), - **_solver_common(dense, epochs=epochs, learning_rate=learning_rate, seed=seed), + **_l2_solver_kwargs(selection_l2), + **_solver_common( + dense, + epochs=epochs, + learning_rate=learning_rate, + seed=seed, + target_loss_weights=search_weights, + ), ) if selection.gate_open_probabilities is None: raise RuntimeError("informed L0 returned no selection probabilities.") @@ -316,6 +539,14 @@ def refit_uk_dataset_size( pi_hi: float = 1.0, initial_lambda: float | None = None, baseline_pi_floor: float = 0.0, + selection_l2_lambda: float = 0.0, + selection_l2_anchor: str | None = None, + selection_l2_basis: str | None = None, + refit_l2_lambda: float = 0.0, + refit_l2_anchor: str | None = None, + refit_l2_basis: str | None = None, + selection_target_weighting: UKStageTargetWeighting | None = None, + refit_target_weighting: UKStageTargetWeighting | None = None, selection: UKSizeSelection | None = None, draw: UKSizeDraw | None = None, progress_callback: ProgressCallback | None = None, @@ -352,11 +583,43 @@ def refit_uk_dataset_size( searched for the same size, epochs, learning rate and seed on this pool. ``draw`` additionally reuses an authenticated completed draw without consuming its random stream again; the probability binding must match. + + ``selection_l2_*`` are the search's L2 settings (a reused selection must + have been searched under the same ones) and ``refit_l2_*`` the refit's + own, independent of the search's (microcosm#1124; see :func:`uk_size_l2`). + On the exact-count path the refit's ``"initial"`` anchor is the + normalized Horvitz–Thompson baseline, floored by ``baseline_pi_floor``. + ``selection_target_weighting`` and ``refit_target_weighting`` run a stage + under a named rule other than the dense solve's weights (the + size-experiment seam; graph builds never pass them). Every new setting is + off by default and then leaves the call, its solver options and its + receipt exactly as before. """ n = _check_size_inputs(frame, dense, households) pi_hi = _check_pi_hi(pi_hi) baseline_pi_floor = _check_baseline_pi_floor(baseline_pi_floor) + selection_l2 = uk_size_l2( + "selection", + l2_lambda=selection_l2_lambda, + anchor=selection_l2_anchor, + basis=selection_l2_basis, + ) + refit_l2 = uk_size_l2( + "refit", l2_lambda=refit_l2_lambda, anchor=refit_l2_anchor, basis=refit_l2_basis + ) + search_weights = _stage_loss_weights(dense, selection_target_weighting) + refit_weights = _stage_loss_weights(dense, refit_target_weighting) if households == n: + if ( + selection_l2 is not None + or refit_l2 is not None + or search_weights is not None + or refit_weights is not None + ): + raise ValueError( + "a full-pool size returns the dense solve unchanged, so it cannot " + "honour a size-stage L2 penalty or target weighting." + ) return UKDatasetSize( dense, np.arange(n), @@ -379,6 +642,10 @@ def refit_uk_dataset_size( seed=seed, pi_hi=pi_hi, initial_lambda=initial_lambda, + selection_l2_lambda=selection_l2_lambda, + selection_l2_anchor=selection_l2_anchor, + selection_l2_basis=selection_l2_basis, + target_weighting=selection_target_weighting, progress_callback=progress_callback, ) reused = False @@ -404,6 +671,25 @@ def refit_uk_dataset_size( or selection.selection.l0_lambda <= 0.0 ): mismatched.append("pool: the selection was not searched on this pool") + stored_l2 = uk_size_l2_from_options(selection.selection.options) + requested_l2 = None if selection_l2 is None else selection_l2.as_dict() + if stored_l2 != requested_l2 or selection.selection.options.get( + "l2_anchor_weights_supplied", False + ): + mismatched.append( + f"selection L2: searched under {stored_l2!r}, requested " + f"{requested_l2!r}" + ) + expected_search_weights = ( + dense.target_loss_weights if search_weights is None else search_weights + ) + if not np.array_equal( + np.asarray(selection.selection.target_loss_weights, dtype=np.float64), + np.asarray(expected_search_weights, dtype=np.float64), + ): + mismatched.append( + "target loss weights: the selection was searched under other weights" + ) if mismatched: raise ValueError( "cannot draw from a selection searched for different inputs: " @@ -412,7 +698,11 @@ def refit_uk_dataset_size( ) init_protected = selection.protected common = _solver_common( - dense, epochs=epochs, learning_rate=learning_rate, seed=seed + dense, + epochs=epochs, + learning_rate=learning_rate, + seed=seed, + target_loss_weights=refit_weights, ) probabilities = selection.selection.gate_open_probabilities assert probabilities is not None @@ -460,6 +750,11 @@ def refit_uk_dataset_size( support_inclusion_probabilities=baseline_q, mass_reason=dense.options["mass_reason"], progress_callback=_phased(progress_callback, "size_refit"), + # The refit's weights are the released weights: keep its best epoch on + # the objective it minimizes, as the dense solve keeps its best on the + # target loss, not whatever the last update left (microcosm#1124). + retain_best_objective=True, + **_l2_solver_kwargs(refit_l2), **common, ).refit if ( @@ -478,6 +773,15 @@ def refit_uk_dataset_size( baseline_name = "normalized_horvitz_thompson_w_over_q" + ( "_floored" if baseline_pi_floor > 0.0 else "" ) + receipt_extras = _size_stage_receipt( + dense=dense, + search_result=search_result, + refit=refit, + refit_l2=refit_l2, + baseline_name=baseline_name, + selection_weighting=selection_target_weighting, + refit_weighting=refit_target_weighting, + ) return UKDatasetSize( refit, support, @@ -525,10 +829,81 @@ def refit_uk_dataset_size( np.max(np.abs(errors_small - errors_dense) / dense.target_loss_scales) ), "certification": "candidate_only_pending_matched_comparison_and_promotion_scorecard", + **receipt_extras, }, ) +def _size_stage_receipt( + *, + dense: CalibrationResult, + search_result: CalibrationResult, + refit: CalibrationResult, + refit_l2: UKSizeL2 | None, + baseline_name: str, + selection_weighting: UKStageTargetWeighting | None, + refit_weighting: UKStageTargetWeighting | None, +) -> dict[str, object]: + """The microcosm#1124 receipt fields, present only when a setting is on. + + A default run (no L2, the dense's weights in both stages) adds nothing, + so its size receipt is byte-identical to the one before these settings + existed. + """ + + extras: dict[str, object] = {} + searched_l2 = uk_size_l2_from_options(search_result.options) + if searched_l2 is not None: + extras["selection_l2"] = { + **searched_l2, + "penalty": search_result.options.get("l2_penalty"), + "anchor_reference": ( + "pool_design" + if searched_l2["anchor"] == "initial" + else "pool_design_mean" + ), + } + if refit_l2 is not None: + start = np.asarray(refit.initial_weights, dtype=np.float64) + anchor = ( + start if refit_l2.anchor == "initial" else np.full_like(start, start.mean()) + ) + extras["refit_l2"] = { + **refit_l2.as_dict(), + "penalty": refit.options.get("l2_penalty"), + "anchor_reference": ( + baseline_name + if refit_l2.anchor == "initial" + else f"{baseline_name}_mean" + ), + "chi_square_distance_to_anchor": float( + chi_square_distance(np.asarray(refit.weights, dtype=np.float64), anchor) + ), + } + extras["refit_iterate_selection"] = refit.options.get("iterate_selection") + names = dense.problem.names + if selection_weighting is not None: + extras["selection_target_weighting"] = selection_weighting.receipt(names) + if refit_weighting is not None: + extras["refit_target_weighting"] = refit_weighting.receipt(names) + # The compact loss under the dense solve's own weights keeps the + # dense-versus-compact comparison on one yardstick. + estimates = np.asarray( + [diagnostic.final_estimate for diagnostic in refit.diagnostics], + dtype=np.float64, + ) + extras["compact_loss_under_dense_weights"] = float( + relative_error_loss( + estimates, + np.asarray(refit.problem.target_vector, dtype=np.float64), + target_loss_weights=dense.target_loss_weights, + target_loss_scales=dense.target_loss_scales, + target_loss_cap=dense.target_loss_cap, + ) + ) + return extras + + def unsupported_nonzero_targets(problem: Any) -> list[str]: """Names of nonzero targets whose constraint row has no nonzero entry. diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/full_build_cli.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/full_build_cli.py index 1ed2d9ec12..49e63ea1a7 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/full_build_cli.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/full_build_cli.py @@ -108,6 +108,7 @@ spine_provenance_from_sidecar, ) from .chronicle_feed import load_uk_chronicle_feed +from .dataset_size import UK_SIZE_L2_ANCHORS, UK_SIZE_L2_BASES from .frs_release import load_uk_frs_release from .full_certification import ( append_uk_full_certification_node, @@ -156,6 +157,11 @@ register_uk_terminal_kernels, rowwise_candidate_manifest_from_graph, ) +from .inspection_export import ( + register_uk_inspection_export_kernel, + use_inspection_export, +) +from .local_doctrine import UK_LOCAL_ROW_METADATA_RULES from .national_doctrine import uk_doctrine_with_overrides from .national_frame import ( load_uk_national_frame, @@ -169,6 +175,7 @@ REPOSITORY, candidate_clone_counts_argument, candidate_identity_digest, + cli_size_l2, git_commit, git_dirty, json_text, @@ -508,7 +515,15 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace: ) parser.add_argument("--seed", type=int, help="Defaults to the role's seed.") parser.add_argument("--selection-seed", type=int) - parser.add_argument("--selection-pi-hi", type=float, default=1.0) + parser.add_argument( + "--selection-pi-hi", + type=float, + default=None, + help=( + "Inclusion-probability ceiling of the exact-count draw. Defaults to the " + "doctrine's 0.5 with --dataset-households, 1.0 otherwise." + ), + ) parser.add_argument( "--selection-initial-lambda", type=float, @@ -526,13 +541,49 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace: parser.add_argument( "--baseline-pi-floor", type=float, - default=0.0, + default=None, help=( "Floor on the inclusion probability the refit's Horvitz-Thompson " "baseline divides each selected row's dense weight by (microcosm#355). " - "0 (default) is the untrimmed baseline. Requires --dataset-households." + "Defaults to the doctrine's 0.5 with --dataset-households; 0 is the " + "untrimmed baseline. Requires --dataset-households." ), ) + for size_stage, weights, default_note in ( + ( + "selection", + "the informed L0 size search's pre-gate weights", + "off by default", + ), + ( + "refit", + "the exact-count refit's weights", + "with --dataset-households and no refit L2 flag, the doctrine's " + "chi-square penalty toward uniform weights at λ 0.0216; 0 turns it off", + ), + ): + parser.add_argument( + f"--{size_stage}-l2-lambda", + type=float, + default=None, + help=( + f"A chi-square L2 penalty on {weights} toward their anchor " + f"(microcosm#1124); {default_note}. Candidate-only; requires " + "--dataset-households." + ), + ) + parser.add_argument( + f"--{size_stage}-l2-anchor", + choices=UK_SIZE_L2_ANCHORS, + default=None, + help=( + "'initial' (the stage's starting weights; the refit's is the " + "normalized Horvitz-Thompson baseline) or 'uniform' (their mean)." + ), + ) + parser.add_argument( + f"--{size_stage}-l2-basis", choices=UK_SIZE_L2_BASES, default=None + ) parser.add_argument( "--no-size-checkpoint", action="store_true", @@ -550,7 +601,12 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace: ) parser.add_argument( "--target-weight-rule", - choices=("uniform", "grain_equal", "family_equal"), + choices=( + "uniform", + "grain_equal", + "family_equal", + *UK_LOCAL_ROW_METADATA_RULES, + ), help="Defaults to the role's doctrine; any other admitted rule is a receipted override.", ) parser.add_argument("--engine-blocks", type=int, default=1) @@ -562,6 +618,16 @@ def parse_args(argv: list[str] | None = None) -> argparse.Namespace: help="Dense role only: constituency vintage required from the ladder.", ) parser.add_argument("--skip-holdout", action="store_true") + parser.add_argument( + "--inspection-export", + action="store_true", + help=( + "Candidate-only: export, package and stage the H5 for inspection " + "even when a structural full-build gate blocks the artifact. The " + "bundle stays unreleasable and the exit status still reports the " + "block. Refused with --release-candidate." + ), + ) parser.add_argument("--release-candidate", action="store_true") parser.add_argument("--review-date", type=date.fromisoformat, default=date.today()) parser.add_argument( @@ -921,6 +987,8 @@ def prepare_full_build( selection_initial_lambda=args.selection_initial_lambda, baseline_pi_floor=args.baseline_pi_floor, target_weight_rule=args.target_weight_rule, + selection_l2=cli_size_l2(args, "selection"), + refit_l2=cli_size_l2(args, "refit"), ), ) args._calibration_year = int(config.calibration_year) @@ -1014,6 +1082,13 @@ def prepare_full_build( "engine_identity": engine_identity, "release_candidate": args.release_candidate, "skip_holdout": args.skip_holdout, + # Recorded only when on, so a default request (and the export node's + # key) is unchanged. + **( + {"inspection_export": True} + if getattr(args, "inspection_export", False) + else {} + ), "targets": { "chronicle": { "facts_sha256": feed.facts_sha256, @@ -1036,6 +1111,9 @@ def prepare_full_build( ), ), ) + if getattr(args, "inspection_export", False): + graph = use_inspection_export(graph) + register_uk_inspection_export_kernel(kernels) register_uk_full_kernels( kernels, progress_callback=None if args.dry_run else _solve_observer(args, telemetry), @@ -1914,7 +1992,15 @@ def _execute_full_build( local_gate_report=_local_gate_report_state(local_report), ) if not enforcement["artifact_permitted"]: - return 1 + if not getattr(args, "inspection_export", False): + return 1 + print( + "warning: structural gates blocked the artifact " + f"({', '.join(enforcement['structural_failures'])}); " + "--inspection-export continues to an unreleasable inspection bundle.", + file=sys.stderr, + flush=True, + ) stage(telemetry, "output_bundle", "started") manifest = _run_graph_with_progress( compile_graph(graph), diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/full_gates.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/full_gates.py index 1bc886834b..72bee79bc5 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/full_gates.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/full_gates.py @@ -291,6 +291,9 @@ def build_full_gate_context( "aggregate_admin": admin_totals, "aggregate_admin_measurement": admin_receipt, "full_gate_scope": uk_full_gate_scope_receipt(selection_receipt), + # The full build is the local line's K-clone pool; line-scoped + # register entries earned on the national seam stand aside here. + "release_line": "local", } ) if "rules_engine" not in artifacts and "coverage_engine" in artifacts: diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/full_measure.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/full_measure.py index fb82aa4a2e..03bf9f0aba 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/full_measure.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/full_measure.py @@ -282,6 +282,19 @@ def _engine_population_representation(frame, block_frames) -> dict[str, Any]: and inputs_match ), "blocks": blocks, + # What ``exact`` certifies, and what it does not: on a slice of the + # K=25 pool, per-clone and single-block resolution matched on every + # household column and constituency metric but differed on a small + # share of person records (state pension type, pension credit, SLC + # repayment), so engine draws that depend on a record's position in + # the simulation are outside it (microcosm#1124). The released file is + # re-measured on one engine by the incumbent-surface evaluation. + "exact_scope": ( + "weight-share formulas see the pool's denominator in every block: " + "identical clone copies, pool-scaled block weights and equal " + "allocation-key sums; engine draws that depend on a record's " + "position in the simulation are not covered" + ), "factor_by_block": factors, "checks": { "identity_key": identity_key, diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_calibration.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_calibration.py index a6881d49d1..1103094d6f 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_calibration.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_calibration.py @@ -52,9 +52,9 @@ from microcosm.graph.canonical import canonical_json from . import dataset_size, size_checkpoint -from .dataset_size import UKSizeDraw, UKSizeSelection +from .dataset_size import UKSizeDraw, UKSizeL2, UKSizeSelection from .graph_population import context_frame, population_slices -from .local_doctrine import UK_LOCAL_SOLVE_EPOCHS +from .local_doctrine import UK_LOCAL_SOLVE_EPOCHS, UK_LOCAL_TARGET_WEIGHT_RULES SIZE_SEARCH_TYPE = ArtifactType("microcosm.uk.size-search", 1) SIZE_DRAW_TYPE = ArtifactType("microcosm.uk.size-draw", 1) @@ -82,6 +82,12 @@ class UKGraphCalibrationConfig: selection_initial_lambda: float | None = None baseline_pi_floor: float = 0.0 target_weight_rule: str = "uniform" + #: microcosm#1124: each size stage's L2 penalty (``None``: off). The + #: selection's is a parameter of the search, draw and refit nodes; the + #: refit's of the refit node alone, like ``baseline_pi_floor``. Off adds + #: no node parameter, so a default build's node keys keep their params. + selection_l2: UKSizeL2 | None = None + refit_l2: UKSizeL2 | None = None def __post_init__(self): if type(self.epochs) is not int or self.epochs < 1: @@ -98,6 +104,23 @@ def __post_init__(self): dataset_size._check_pi_hi(self.selection_pi_hi) dataset_size._check_initial_lambda(self.selection_initial_lambda) dataset_size._check_baseline_pi_floor(self.baseline_pi_floor) + for stage, l2 in (("selection", self.selection_l2), ("refit", self.refit_l2)): + if l2 is None: + continue + if not isinstance(l2, UKSizeL2) or l2.stage != stage: + raise ValueError( + f"{stage}_l2 must be a {stage}-stage UKSizeL2 or None." + ) + if self.dataset_households is None: + raise ValueError(f"A {stage} L2 penalty requires a dataset size.") + if self.target_weight_rule not in ( + *UK_LOCAL_TARGET_WEIGHT_RULES, + "family_equal", + ): + raise ValueError( + f"Unknown target_weight_rule {self.target_weight_rule!r}; admitted: " + f"{(*UK_LOCAL_TARGET_WEIGHT_RULES, 'family_equal')}." + ) @dataclass(frozen=True) @@ -605,27 +628,18 @@ def run(self, context): dense = _dense(context, frame, problem) selection = _selection(context, frame, problem) draw = json.loads(context.artifacts["draw"].payload) - if draw.pop("problem_sha256") != problem.sha256: - raise ValueError("Exact-count draw belongs to another ordered problem.") - method = draw.pop("method") if selection is None: - if method != "full_pool" or draw["support"] != list( + if draw.get("problem_sha256") != problem.sha256: + raise ValueError("Exact-count draw belongs to another ordered problem.") + if draw.get("method") != "full_pool" or draw.get("support") != list( range(frame.n("household")) ): raise ValueError("Full-pool draw has a different support.") cached_draw = None else: - if method != "exact_count": + cached_draw = UKSizeDraw.from_payload(draw, problem_sha256=problem.sha256) + if cached_draw is None: raise ValueError("Compact refit requires a completed exact-count draw.") - cached_draw = UKSizeDraw( - **{ - **draw, - "support": np.asarray(draw["support"]), - "inclusion_probabilities": np.asarray( - draw["inclusion_probabilities"], dtype=np.float64 - ), - } - ) compact = dataset_size.refit_uk_dataset_size( frame, dense, @@ -816,6 +830,7 @@ def uk_calibration_nodes( # search verifies it like any probe, so it is a parameter of the # search node and of the nodes that reuse its selection. "initial_lambda": config.selection_initial_lambda, + **({} if config.selection_l2 is None else config.selection_l2.params()), } dense_input = ArtifactInput("dense", dense_id, "result", RESULT_TYPE) search_id, draw_id, refit_id = ( @@ -876,7 +891,11 @@ def uk_calibration_nodes( kernel=UKSizeRefitKernel.ref, population=base, inputs=inputs, - params={**params, "baseline_pi_floor": config.baseline_pi_floor}, + params={ + **params, + "baseline_pi_floor": config.baseline_pi_floor, + **({} if config.refit_l2 is None else config.refit_l2.params()), + }, artifact_inputs=( *search_inputs, ArtifactInput("draw", draw_id, "draw", SIZE_DRAW_TYPE), diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_national.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_national.py index f479783e6f..7dd1d1ca8d 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_national.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_national.py @@ -943,6 +943,7 @@ def run(self, context: KernelContext) -> KernelResult: "exclusions_evaluated_on": date.fromisoformat( str(context.params["review_date"]) ), + "release_line": "national", } report = evaluate_phase( gates, diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_targets.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_targets.py index bdc2c1edbf..11feee0b20 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_targets.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_targets.py @@ -45,7 +45,16 @@ from microcosm.graph.canonical import canonical_json from microcosm.graph.codecs import SOURCE_CODECS -from . import full_measure, full_problem, ladder_targets, ledger_targets, local_doctrine +from . import ( + full_measure, + full_problem, + ladder_targets, + ledger_targets, + local_doctrine, + local_rowwise, + national_doctrine, + target_weights, +) from .full_measure import resolve_uk_full_national_problem from .full_problem import build_uk_full_local_problem from .full_targets import CHRONICLE_SOURCE_CODEC, load_chronicle_source_bytes @@ -572,6 +581,22 @@ def reconstruct_uk_full_problem_inputs(context: KernelContext) -> UKFullProblemI class UKFullProblemKernel(_TargetKernel): ref = "uk.full.problem@1" + def implementation_hash(self) -> str: + # The loss weights the problem binds are built by + # ``local_rowwise.prepare_uk_full_solve`` from the rules in + # ``target_weights`` and ``national_doctrine`` (microcosm#1124), so an + # edit there re-keys the problem instead of replaying a stale vector. + return hashlib.sha256( + canonical_json( + { + "targets": super().implementation_hash(), + "weights": source_hash( + local_rowwise, national_doctrine, target_weights + ), + } + ) + ).hexdigest() + def run(self, context: KernelContext) -> KernelResult: inputs = reconstruct_uk_full_problem_inputs(context) frame, local_problem = inputs.frame, inputs.local_problem @@ -604,6 +629,17 @@ def run(self, context: KernelContext) -> KernelResult: } ) doctrine = local_doctrine.UK_LOCAL_SOLVE_DOCTRINE + weight_receipt = ( + {} + if prepared.target_rows is None + else { + "target_loss_weight_receipt": target_weights.uk_target_loss_weight_receipt( + prepared.target_rows, + prepared.target_loss_weights, + rule=str(context.params["target_weight_rule"]), + ) + } + ) payload = encode_problem( problem, entity_ids=ids, @@ -629,6 +665,7 @@ def run(self, context: KernelContext) -> KernelResult: "cross_geography": cross, "measure_exclusions": full["measure_exclusions"], "calibration_year": full["calibration_year"], + **weight_receipt, }, ) return KernelResult(artifacts={"problem": payload}) diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_terminal.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_terminal.py index 9e3ad12acd..d5f769af0a 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_terminal.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/graph_terminal.py @@ -48,6 +48,7 @@ from ..artifact_files import file_artifact from . import geography_ladder, national_frame from .atomic_area_support import uk_area_code_frames, without_uk_native_alias_columns +from .dataset_size import UK_SIZE_L2_PARAM_KEYS from .geography_ladder import ( UK_EXPORT_AREA_CODE_COLUMNS, export_area_code_columns, @@ -1090,7 +1091,14 @@ class UKFullHoldoutKernel(KernelBase): def implementation_hash(self) -> str: from ..country_spec import load_country_spec - from . import dataset_size, graph_targets, local_rowwise + from . import ( + dataset_size, + graph_targets, + local_doctrine, + local_rowwise, + national_doctrine, + target_weights, + ) return hashlib.sha256( canonical_json( @@ -1100,6 +1108,9 @@ def implementation_hash(self) -> str: graph_targets, local_rowwise, dataset_size, + local_doctrine, + national_doctrine, + target_weights, dependencies=self.capabilities.dependencies, ), "country_resources": load_country_spec("uk").fingerprint, @@ -1152,6 +1163,12 @@ def run(self, context: KernelContext) -> KernelResult: selection_seed=context.params.get("selection_seed"), selection_pi_hi=float(context.params["selection_pi_hi"]), baseline_pi_floor=float(context.params["baseline_pi_floor"]), + selection_initial_lambda=context.params.get("selection_initial_lambda"), + **{ + key: context.params[key] + for key in UK_SIZE_L2_PARAM_KEYS + if key in context.params + }, ) report = { **report, @@ -1195,6 +1212,21 @@ def uk_full_holdout_node( "baseline_pi_floor": ( 0.0 if size is None else size.params["baseline_pi_floor"] ), + # microcosm#1124: each fold's budget search starts where the main + # search started, so five folds do not each search from zero. + "selection_initial_lambda": ( + None if size is None else size.params.get("initial_lambda") + ), + # microcosm#1124: the size stages' L2 penalties, present only when on. + **( + {} + if size is None + else { + key: size.params[key] + for key in UK_SIZE_L2_PARAM_KEYS + if key in size.params + } + ), }, artifact_inputs=( *original.artifact_inputs, @@ -1364,6 +1396,7 @@ def rowwise_candidate_manifest_from_graph( local_vintage_census, release_verdict, rowwise_parameters, + size_doctrine_drift, ) nodes = final_manifest.nodes @@ -1449,6 +1482,7 @@ def rowwise_candidate_manifest_from_graph( representation = dict(bindings.get("measure_resolution", {})).get( "engine_population_representation" ) + size_drift = size_doctrine_drift(args) releasable, release_posture = release_verdict( sample_fraction=args.sample_fraction, engine_blocks=args.engine_blocks, @@ -1677,6 +1711,15 @@ def rowwise_candidate_manifest_from_graph( } } ), + **( + {} + if "target_loss_weight_receipt" not in bindings + else { + "target_loss_weight_receipt": dict( + bindings["target_loss_weight_receipt"] + ) + } + ), "measure_resolution": dict(bindings.get("measure_resolution", {})), "cross_grain": dict(bindings.get("cross_geography", {})), "binding_adjudications": dict(bindings.get("binding_adjudications", {})), @@ -1744,13 +1787,18 @@ def rowwise_candidate_manifest_from_graph( for gate_id, payload in gate_rows.items() if not isinstance(payload, Mapping) or payload.get("status") != "passed" ), - "releasable": bool(releasable and args.dataset_households is None), + # microcosm#1124 (2026-10-13): a sized build is releasable only at the + # local size doctrine; no matched-size certification is required. + "releasable": bool(releasable and not size_drift), "release_posture": { **release_posture, **( {} if args.dataset_households is None - else {"size_certification_present": False} + else { + "size_certification_present": False, + "size_doctrine": not size_drift, + } ), }, "census_household_uprating": dict( diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/hmrc_source_contract.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/hmrc_source_contract.py index 3f2e96f5b3..e5f3a0542a 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/hmrc_source_contract.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/hmrc_source_contract.py @@ -64,6 +64,7 @@ "HMRC_DISTRIBUTIONAL_INPUTS", "assert_uk_hmrc_income_source_contract_current", "uk_hmrc_weighted_qrf_output_columns", + "uk_spi_coherence_superseded_columns", ] UK_CANONICAL_SOURCE_STAGES_RESOURCE = "source_stages.json" @@ -917,6 +918,62 @@ def uk_hmrc_weighted_qrf_output_columns( return tuple(outputs) +#: The stage whose declared report replacements take QRF outputs off the +#: tail-concentration surface, and the operations that replace them. +_SPI_COHERENCE_STAGE = "spi_benefit_coherence" +_SPI_COHERENCE_SUPERSEDING_KINDS = frozenset( + {"zero_spi_channel_reports", "restore_spi_channel_reports_from_source"} +) + + +def uk_spi_coherence_superseded_columns(resource: Any | None = None) -> tuple[str, ...]: + """Report columns ``spi_benefit_coherence`` replaces on every SPI-channel row. + + The stage (microcosm#1100) zeroes the income-tested legacy reports and + restores IIDB, AFCS and bereavement support from the FRS twin, so after it + these columns hold only FRS respondents' own reports: no stage-2 forest + draw survives in them, and the donor-tail broadcast the tail-concentration + gate looks for (#462) cannot occur there. Read from the stage's declared + operations in the committed source manifest, in declaration order. + """ + + target = ( + files("microcosm.build.uk").joinpath(UK_CANONICAL_SOURCE_STAGES_RESOURCE) + if resource is None + else resource + ) + raw = ( + target.read_text(encoding="utf-8") + if hasattr(target, "read_text") + else Path(target).read_text(encoding="utf-8") + ) + stages = json.loads(raw).get("stages") + matches = [ + stage + for stage in stages or () + if isinstance(stage, Mapping) and stage.get("stage") == _SPI_COHERENCE_STAGE + ] + if len(matches) != 1: + raise ValueError( + f"UK source manifest must contain exactly one {_SPI_COHERENCE_STAGE} stage." + ) + columns: dict[str, None] = {} + for operation in matches[0].get("operations", ()): + if operation.get("kind") not in _SPI_COHERENCE_SUPERSEDING_KINDS: + continue + for column in operation.get("columns", ()): + if not isinstance(column, str) or not column: + raise ValueError( + f"{_SPI_COHERENCE_STAGE} declares a non-string or empty column." + ) + columns[column] = None + if not columns: + raise ValueError( + f"{_SPI_COHERENCE_STAGE} declares no zeroed or restored report columns." + ) + return tuple(columns) + + def _load_payload(resource: Any | None) -> Mapping[str, Any]: target = ( files("microcosm.build.uk").joinpath(UK_CANONICAL_SOURCE_STAGES_RESOURCE) diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/inspection_export.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/inspection_export.py new file mode 100644 index 0000000000..60e8a7ba0e --- /dev/null +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/inspection_export.py @@ -0,0 +1,81 @@ +"""Candidate-only inspection export past a structural full-build gate block. + +``microcosm-build-uk --release-role dense --inspection-export`` exports, +packages and stages an unreleasable H5 for inspection when a structural gate +blocks the artifact, so a candidate can be studied rather than discarded. The +CLI refuses the flag for release candidates and for the national role; the +manifest stays unreleasable and the exit status still reports the block. + +The kernel lives in its own module: the gate, holdout and export kernels hash +:mod:`graph_terminal`'s source, so keeping the inspection path out of it +leaves every default node key unchanged. +""" + +from __future__ import annotations + +import json +import sys +from dataclasses import replace + +from microcosm.graph import ( + Graph, + KernelContext, + KernelRegistry, + KernelResult, + source_hash, +) +from microcosm.graph.canonical import canonical_json + +from . import graph_terminal +from .graph_population import context_frame + +#: The export description node the inspection kernel replaces. +EXPORT_PREPARE_NODE = "uk.full.export.prepare" + + +class UKInspectionExportPrepareKernel(graph_terminal.UKExportPrepareKernel): + """Describe the H5 as the export kernel does, without the gate refusal.""" + + ref = "uk.full-export.prepare-inspection@1" + + def implementation_hash(self) -> str: + return source_hash(sys.modules[__name__], graph_terminal) + + def run(self, context: KernelContext) -> KernelResult: + if context.params.get("inspection_export") is not True: + raise ValueError("The inspection export kernel requires inspection_export.") + bindings = json.loads(str(context.params["bindings"])) + if bindings.get("inspection_export") is not True: + raise ValueError( + "An inspection export must record inspection_export in its request." + ) + bindings["artifacts"] = { + name: value.key for name, value in context.artifacts.items() + } + descriptor = graph_terminal.describe_uk_export( + context_frame(context), bindings=bindings + ) + return KernelResult(artifacts={"export_descriptor": canonical_json(descriptor)}) + + +def use_inspection_export(graph: Graph) -> Graph: + """Swap the export description node onto the inspection kernel.""" + + nodes = [] + swapped = 0 + for node in graph.nodes: + if node.id == EXPORT_PREPARE_NODE: + node = replace( + node, + kernel=UKInspectionExportPrepareKernel.ref, + params={**dict(node.params), "inspection_export": True}, + ) + swapped += 1 + nodes.append(node) + if swapped != 1: + raise ValueError(f"Expected one {EXPORT_PREPARE_NODE} node, found {swapped}.") + return replace(graph, nodes=tuple(nodes)) + + +def register_uk_inspection_export_kernel(registry: KernelRegistry) -> None: + registry.register(UKInspectionExportPrepareKernel()) diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/local_doctrine.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/local_doctrine.py index 556c96ef57..dffc88c217 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/local_doctrine.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/local_doctrine.py @@ -33,13 +33,22 @@ __all__ = [ "UK_LOCAL_CLONE_COUNT", + "UK_LOCAL_DATASET_HOUSEHOLDS", "UK_LOCAL_MAX_WEIGHT_RATIO", + "UK_LOCAL_ROW_METADATA_RULES", + "UK_LOCAL_SIZE_BASELINE_PI_FLOOR", + "UK_LOCAL_SIZE_REFIT_L2_ANCHOR", + "UK_LOCAL_SIZE_REFIT_L2_BASIS", + "UK_LOCAL_SIZE_REFIT_L2_LAMBDA", + "UK_LOCAL_SIZE_SELECTION_PI_HI", "UK_LOCAL_SOLVE_DOCTRINE", "UK_LOCAL_SOLVE_EPOCHS", "UK_LOCAL_TARGET_LOSS_CAP", "UK_LOCAL_TARGET_WEIGHT_RULE", + "UK_LOCAL_TARGET_WEIGHT_RULES", "UKLocalSolveDoctrine", "uk_local_doctrine_with_overrides", + "uk_local_size_doctrine", "uk_local_target_loss_weights", ] @@ -71,7 +80,19 @@ #: row matrix and are effectively ignored (78 % within 10 %; OBR aggregates #: 43 %); ``grain_equal`` holds them at 92–93 % for about two points of #: census-household and tenure fit. ``uniform`` stays a receipted override. -UK_LOCAL_TARGET_WEIGHT_RULE = "grain_equal" +#: +#: microcosm#1124 (PR #1163 experiments on the K=25 pool, and the K=25 / +#: 100,000-household build of 2026-10-11): ``nation_grain_family_equal_sqrt_count`` +#: — each grain's loss share is split equally across its target families +#: (national rows by nation as a fourth grain), and count rows within a +#: family by the square root of their count. Under ``grain_equal`` the 662 +#: HMRC rows took 18.9 % of the loss and the size selection kept a lone-person +#: share of 23 % against the dense solve's 28.6 %; under this rule the selection +#: keeps the composition (28.8 %) and the Northern Ireland share (2.73 %). It +#: dilutes the HMRC top-income bands, which is where the national misses +#: remain (microcosm#1123). ``grain_equal`` and ``uniform`` stay receipted +#: overrides; the other row-metadata rules too. +UK_LOCAL_TARGET_WEIGHT_RULE = "nation_grain_family_equal_sqrt_count" #: Declared solve length for the UK local solve (Adam epochs). #: @@ -79,21 +100,90 @@ #: ruling, matching the national certified-cut posture. The solver has no #: stopping criterion; at 512 the loss was still falling 2–6 % per 128 #: epochs, at 1500 it is 0.4 % (``uniform``) / 2.8 % (``grain_equal``). -UK_LOCAL_SOLVE_EPOCHS = 1500 +#: +#: microcosm#1124: 2000, the length every #1124 measurement ran at (the +#: 2026-10-06 and 2026-10-11 builds and the #1163 experiments). On the +#: 2026-10-11 build's stored trajectories the dense solve's best loss by +#: epoch 1500 is 3.5 % above its best by 2000, and the size search is still +#: falling at 2000 (7.5 % above at 1500); the refit is within 0.2 % of its +#: best by epoch 500. The K=15 rebuild re-measures it per stage. +UK_LOCAL_SOLVE_EPOCHS = 2000 #: Declared clone count K for the UK local candidate (rows per spine #: household through the OA ladder). #: -#: microcosm#762 adjudication (2026-09-03, receipt R9): 15. The plan's -#: K=4 default refused 172 UC child-band cells with zero support; K=10 -#: refused 86 local-authority cells (84 band-H plus the two micro-authorities; -#: A14) and, under ``grain_equal`` at bound 10, left five constituencies -#: 0.3–7.7 ESS under the floor; K=15 clears the -#: floor (min constituency ESS 54.1) at unchanged fit and flat memory. +#: microcosm#762 adjudication (2026-09-03, receipt R9): 15. Smaller K left +#: target cells without supporting households and constituencies under the +#: ESS floor; K=15 cleared the floor at unchanged fit and flat memory +#: (receipts R8-R10 in experiments/762-uk-rowwise-candidate-receipts.md). +#: +#: microcosm#1124 (ruling of 2026-10-13): stays 15. Across the dense pools of +#: the uk-dataset-evaluation reports (``55k-candidates`` and ``55k-rebuild`` at +#: K=15, ``55k-k20`` at K=20, ``k25-60k`` and ``k25-100k`` at K=25), more clones +#: did not lift the weakest seat's support, and the 100,000-household draw's +#: support gain on the K=25 pool came from the size settings below, not from +#: K. A K=15 / 100,000-household build re-measures it. UK_LOCAL_CLONE_COUNT = 15 +#: Target-weighting rules that read more row metadata than the grain +#: (microcosm#1124): the row's target family, its geography and its value +#: basis. They are computed in :mod:`microcosm.build.uk_runtime.target_weights` +#: from one row-aligned carrier, and each is a declared function of that +#: metadata, never a per-target vector. ``nation_grain_family_equal_sqrt_count`` +#: is the doctrine's rule; the others are receipted overrides, which a release +#: candidate refuses like any rule but the default. +UK_LOCAL_ROW_METADATA_RULES = ( + "grain_family_equal", + "grain_family_equal_sqrt_count", + "nation_grain_family_equal", + "nation_grain_family_equal_sqrt_count", +) +#: The closed target-weighting vocabulary of the UK local solve. +UK_LOCAL_TARGET_WEIGHT_RULES = ("uniform", "grain_equal", *UK_LOCAL_ROW_METADATA_RULES) + +#: Declared defaults of a sized (``--dataset-households``) local build, applied +#: when the request leaves them unset (microcosm#1124; PR #1163's +#: ``SAN_f0.5_u0.022`` and the 2026-10-11 K=25 / 100,000-household build): +#: the exact-k draw's inclusion-probability ceiling, the refit's +#: Horvitz–Thompson baseline floor, and a chi-square L2 penalty on the refit's +#: weights toward their uniform mean. The penalty is a Kish-ESS control (with +#: a uniform anchor it equals n / ESS − 1, so its λ means the same at any +#: size); λ is 10⁻² times the doctrine rule's loss ratio to ``grain_equal`` +#: (2.1575). A sized release candidate must use exactly these settings at +#: :data:`UK_LOCAL_DATASET_HOUSEHOLDS`. +UK_LOCAL_SIZE_SELECTION_PI_HI = 0.5 +UK_LOCAL_SIZE_BASELINE_PI_FLOOR = 0.5 +UK_LOCAL_SIZE_REFIT_L2_LAMBDA = 0.0215748772773509 +UK_LOCAL_SIZE_REFIT_L2_ANCHOR = "uniform" +UK_LOCAL_SIZE_REFIT_L2_BASIS = "chi_square" + +#: The household count a sized local release ships (microcosm#1124, ruling +#: of 2026-10-13): the local release is the exact-k selection from the +#: K-clone pool at this size, under the dense line's publication +#: requirements (rotated holdout, signed gates, incumbent-surface evaluation +#: on one engine). The pool itself is too large for one engine and for the +#: consumers who load the file. A release candidate may also be unsized +#: (the whole pool); any other size stays candidate-only. +UK_LOCAL_DATASET_HOUSEHOLDS = 100_000 + + +def uk_local_size_doctrine() -> dict[str, object]: + """The settings a sized local release candidate must carry, by name.""" + + return { + "dataset_households": UK_LOCAL_DATASET_HOUSEHOLDS, + "selection_pi_hi": UK_LOCAL_SIZE_SELECTION_PI_HI, + "baseline_pi_floor": UK_LOCAL_SIZE_BASELINE_PI_FLOOR, + "refit_l2_lambda": UK_LOCAL_SIZE_REFIT_L2_LAMBDA, + "refit_l2_anchor": UK_LOCAL_SIZE_REFIT_L2_ANCHOR, + "refit_l2_basis": UK_LOCAL_SIZE_REFIT_L2_BASIS, + # The selection (search) stage carries no L2 penalty. + "selection_l2": None, + } + + _ALLOWED_SCALE_RULES = ("default_target_loss_scales",) -_ALLOWED_TARGET_WEIGHT_RULES = ("uniform", "grain_equal") +_ALLOWED_TARGET_WEIGHT_RULES = UK_LOCAL_TARGET_WEIGHT_RULES _OVERRIDABLE_FIELDS = frozenset({"target_weight_rule"}) @@ -103,9 +193,12 @@ class UKLocalSolveDoctrine: ``scale_rule`` and ``target_weight_rule`` are closed vocabularies: the only admissible scale rule is the canonical target-defined default, and - the admissible target weightings are ``grain_equal`` (the reviewed - default) and ``uniform`` (a receipted override). Any other weighting - would be a new reviewed rule name here — never a per-target vector. + the admissible target weightings are ``uniform``, ``grain_equal`` and the + row-metadata rules of :data:`UK_LOCAL_ROW_METADATA_RULES`, of which + ``nation_grain_family_equal_sqrt_count`` is the reviewed default and the + rest are receipted overrides. Any other + weighting would be a new reviewed rule name here — never a per-target + vector. """ target_loss_cap: float = UK_LOCAL_TARGET_LOSS_CAP @@ -156,13 +249,25 @@ def uk_local_target_loss_weights( *, rule: str, ) -> np.ndarray | None: - """Derive doctrine weights from row-grain labels, never target knobs.""" + """Derive doctrine weights from row-grain labels, never target knobs. + + Only the grain rules (``uniform``, ``grain_equal``) are computable from + grain labels alone; a row-metadata rule refuses here and is computed by + :func:`microcosm.build.uk_runtime.target_weights.uk_target_loss_weights_for_rows`. + """ if rule not in _ALLOWED_TARGET_WEIGHT_RULES: raise ValueError( f"target_weight_rule must be one of {_ALLOWED_TARGET_WEIGHT_RULES}, " f"got {rule!r}." ) + if rule in UK_LOCAL_ROW_METADATA_RULES: + raise ValueError( + f"target_weight_rule {rule!r} derives from row metadata (grain, " + "family, geography and value basis), not grain labels alone; compute " + "it with microcosm.build.uk_runtime.target_weights." + "uk_target_loss_weights_for_rows." + ) if rule == "uniform": return None from microcosm.build.uk_runtime.national_doctrine import ( diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/local_rowwise.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/local_rowwise.py index e559ac9d3a..1267f1d80e 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/local_rowwise.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/local_rowwise.py @@ -34,6 +34,7 @@ from microcosm.build.holdout import rotated_folds, summarize_rotations from microcosm.build.uk_runtime import local_target_census from microcosm.build.uk_runtime.local_doctrine import ( + UK_LOCAL_ROW_METADATA_RULES, UK_LOCAL_SOLVE_DOCTRINE, UK_LOCAL_TARGET_LOSS_CAP, uk_local_target_loss_weights, @@ -44,6 +45,11 @@ uk_time_period, validate_uk_national_frame, ) +from microcosm.build.uk_runtime.target_weights import ( + UKTargetRows, + uk_target_loss_weights_for_rows, + uk_target_rows_from_targets, +) from microcosm.build.uk_runtime.weighted_integrity import ( coerce_reviewed_exclusions, exclusion_evaluation_date, @@ -1123,6 +1129,9 @@ class UKPreparedFullSolve: target_loss_weights: np.ndarray binding_adjudications: Mapping[str, Any] mass_reason: str + #: The row carrier a row-metadata weighting rule was computed from + #: (microcosm#1124); ``None`` under the grain rules. + target_rows: UKTargetRows | None = None def prepare_uk_full_solve( @@ -1243,10 +1252,27 @@ def prepare_uk_full_solve( *problem.target_frame["area_type"].astype(str).tolist(), *(["national"] * national_count), ] - target_loss_weights = uk_local_target_loss_weights( - grain_labels, - rule=target_weight_rule, - ) + target_rows = None + if target_weight_rule in UK_LOCAL_ROW_METADATA_RULES: + # microcosm#1124: the rule reads each row's family, geography and + # value basis, carried in the solve's own row order. + target_rows = uk_target_rows_from_targets( + local_target_set.targets, + () if national_rows is None else national_rows.registry.specs, + ) + if target_rows.grain != tuple(grain_labels): + raise ValueError( + "the target-weight row carrier disagrees with the solve's grain " + "labels; the rule cannot be computed on this surface." + ) + target_loss_weights = uk_target_loss_weights_for_rows( + target_rows, rule=target_weight_rule + ) + else: + target_loss_weights = uk_local_target_loss_weights( + grain_labels, + rule=target_weight_rule, + ) return UKPreparedFullSolve( frame=frame, problem=problem, @@ -1255,6 +1281,7 @@ def prepare_uk_full_solve( target_loss_weights=target_loss_weights, binding_adjudications=binding_adjudications, mass_reason=mass_reason, + target_rows=target_rows, ) @@ -1311,6 +1338,13 @@ def solve_uk_rowwise_weights_under_doctrine( selection_seed: int | None = None, selection_pi_hi: float = 1.0, baseline_pi_floor: float = 0.0, + selection_initial_lambda: float | None = None, + selection_l2_lambda: float = 0.0, + selection_l2_anchor: str | None = None, + selection_l2_basis: str | None = None, + refit_l2_lambda: float = 0.0, + refit_l2_anchor: str | None = None, + refit_l2_basis: str | None = None, size_checkpoint_dir: Path | None = None, resume_size_checkpoint: Path | None = None, checkpoint_identity: Mapping[str, Any] | None = None, @@ -1375,6 +1409,20 @@ def solve_uk_rowwise_weights_under_doctrine( dataset_households is None ): raise ValueError("size checkpoints apply to a dataset_households solve.") + if dataset_households is None and ( + selection_l2_lambda + or refit_l2_lambda + or any( + value is not None + for value in ( + selection_l2_anchor, + selection_l2_basis, + refit_l2_anchor, + refit_l2_basis, + ) + ) + ): + raise ValueError("size-stage L2 penalties apply to a dataset_households solve.") if size_checkpoint_dir is not None and resume_size_checkpoint is not None: raise ValueError("a resumed solve does not write a second checkpoint.") progress_callback = _progress_fan_out(progress, progress_events) @@ -1450,6 +1498,10 @@ def solve_uk_rowwise_weights_under_doctrine( learning_rate=learning_rate, seed=size_seed, pi_hi=selection_pi_hi, + initial_lambda=selection_initial_lambda, + selection_l2_lambda=selection_l2_lambda, + selection_l2_anchor=selection_l2_anchor, + selection_l2_basis=selection_l2_basis, progress_callback=progress_callback, ) if size_checkpoint_dir is not None: @@ -1478,6 +1530,12 @@ def solve_uk_rowwise_weights_under_doctrine( seed=size_seed, pi_hi=selection_pi_hi, baseline_pi_floor=baseline_pi_floor, + selection_l2_lambda=selection_l2_lambda, + selection_l2_anchor=selection_l2_anchor, + selection_l2_basis=selection_l2_basis, + refit_l2_lambda=refit_l2_lambda, + refit_l2_anchor=refit_l2_anchor, + refit_l2_basis=refit_l2_basis, selection=size_selection, progress_callback=progress_callback, ) @@ -1867,8 +1925,20 @@ def rotated_uk_local_holdout( selection_seed: int | None = None, selection_pi_hi: float = 1.0, baseline_pi_floor: float = 0.0, + selection_initial_lambda: float | None = None, + selection_l2_lambda: float = 0.0, + selection_l2_anchor: str | None = None, + selection_l2_basis: str | None = None, + refit_l2_lambda: float = 0.0, + refit_l2_anchor: str | None = None, + refit_l2_basis: str | None = None, ) -> dict[str, object]: - """Run five local-row rotations with national rows fixed in training.""" + """Run five local-row rotations with national rows fixed in training. + + A sized fold reruns the whole size selection on its training rows; + ``selection_initial_lambda`` starts each fold's budget search where the + main build's search started. + """ if not len(problem.targets): return { @@ -1888,6 +1958,13 @@ def rotated_uk_local_holdout( seed=UK_LOCAL_HOLDOUT_SEED, ) all_indices = np.arange(len(problem.targets), dtype=np.int64) + # A row-metadata rule weights the held rows from their own row carrier + # (microcosm#1124); the grain rules keep the grain-label path. + local_rows = ( + uk_target_rows_from_targets(_rowwise_target_set(problem).targets) + if target_weight_rule in UK_LOCAL_ROW_METADATA_RULES + else None + ) fold_rows: list[dict[str, object]] = [] declared_national = [] if bound_families is not None: @@ -1927,6 +2004,13 @@ def rotated_uk_local_holdout( selection_seed=selection_seed, selection_pi_hi=selection_pi_hi, baseline_pi_floor=baseline_pi_floor, + selection_initial_lambda=selection_initial_lambda, + selection_l2_lambda=selection_l2_lambda, + selection_l2_anchor=selection_l2_anchor, + selection_l2_basis=selection_l2_basis, + refit_l2_lambda=refit_l2_lambda, + refit_l2_anchor=refit_l2_anchor, + refit_l2_basis=refit_l2_basis, ) held_targets = problem.targets[holdout_indices] held_estimates = np.asarray( @@ -1939,10 +2023,15 @@ def rotated_uk_local_holdout( @ train_solve.weights, dtype=np.float64, ).reshape(-1) - held_weights = uk_local_target_loss_weights( - problem.target_frame.iloc[holdout_indices]["area_type"].astype(str), - rule=target_weight_rule, - ) + if local_rows is not None: + held_weights = uk_target_loss_weights_for_rows( + local_rows.take(holdout_indices), rule=target_weight_rule + ) + else: + held_weights = uk_local_target_loss_weights( + problem.target_frame.iloc[holdout_indices]["area_type"].astype(str), + rule=target_weight_rule, + ) loss = relative_error_loss( held_estimates, held_targets, diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/release_certification.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/release_certification.py index 9f2b3e1582..d6fdaec221 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/release_certification.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/release_certification.py @@ -352,6 +352,7 @@ def run_uk_release_cut_battery( "parity_evidence": parity_evidence, "aggregate_admin": admin_totals, "input_mass_reference": input_mass_reference, + "release_line": "national", } if fit_weight_records is not None: terminal_artifacts["fit_weight_records"] = fit_weight_records diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_cli.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_cli.py index baa49c4156..1647dd9473 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_cli.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_cli.py @@ -38,7 +38,16 @@ record_terminal_attempt, write_error_receipt, ) +from microcosm.build.uk_runtime.dataset_size import UKSizeL2, uk_size_l2 from microcosm.build.uk_runtime.ledger_targets import _spec_geography +from microcosm.build.uk_runtime.local_doctrine import ( + UK_LOCAL_SIZE_BASELINE_PI_FLOOR, + UK_LOCAL_SIZE_REFIT_L2_ANCHOR, + UK_LOCAL_SIZE_REFIT_L2_BASIS, + UK_LOCAL_SIZE_REFIT_L2_LAMBDA, + UK_LOCAL_SIZE_SELECTION_PI_HI, + uk_local_size_doctrine, +) from microcosm.build.uk_runtime.national_sampling import ( UK_SAMPLE_RUNG_TOKENS, UK_SAMPLE_SEED_DEFAULT, @@ -89,6 +98,7 @@ "refuse_national_role_arguments", "release_verdict", "resolve_role_arguments", + "size_doctrine_drift", "rowwise_parameters", "validate_cli_args", ] @@ -213,6 +223,35 @@ def doctrine_bounds(posture: UKRowwisePosture) -> dict[str, Any]: _doctrine_bounds = doctrine_bounds +def _size_l2_flags(args: argparse.Namespace, stage: str) -> dict[str, object]: + return { + field: getattr(args, f"{stage}_l2_{field}", None) + for field in ("lambda", "anchor", "basis") + } + + +def cli_size_l2(args: argparse.Namespace, stage: str) -> UKSizeL2 | None: + """One size stage's L2 penalty from the CLI flags (``None``: off).""" + + flags = _size_l2_flags(args, stage) + return uk_size_l2( + stage, + l2_lambda=0.0 if flags["lambda"] is None else flags["lambda"], + anchor=flags["anchor"], + basis=flags["basis"], + ) + + +def _size_l2_parameters(args: argparse.Namespace) -> dict[str, Any]: + # microcosm#1124: recorded only when a penalty is on, so a default run's + # parameters and identity digest are unchanged. + return { + f"{stage}_l2": l2.as_dict() + for stage in ("selection", "refit") + if (l2 := cli_size_l2(args, stage)) is not None + } + + def rowwise_parameters(args: argparse.Namespace, *, source_year: int) -> dict[str, Any]: """The run parameters a rowwise manifest, plan and identity digest record.""" @@ -265,6 +304,13 @@ def rowwise_parameters(args: argparse.Namespace, *, source_year: int) -> dict[st "l0_lambda": L0_LAMBDA, "budget_iters": BUDGET_ITERS, }, + **_size_l2_parameters(args), + # Recorded only when on, so a default run's parameters are unchanged. + **( + {"inspection_export": True} + if getattr(args, "inspection_export", False) + else {} + ), } @@ -429,13 +475,73 @@ def resolve_role_arguments(args: argparse.Namespace) -> UKRowwisePosture: args.target_weight_rule = posture.target_weight_rule if args.expected_constituency_vintage is None: args.expected_constituency_vintage = posture.expected_constituency_vintage + _resolve_size_defaults(args, posture) args._posture = posture return posture +def _resolve_size_defaults(args: argparse.Namespace, posture: UKRowwisePosture) -> None: + """Fill a sized dense build's unset size-stage settings from the doctrine. + + A dense build with ``--dataset-households`` takes the doctrine's draw + ceiling, refit floor and refit L2 penalty where the request leaves them + unset (microcosm#1124); explicit values, including ``--refit-l2-lambda 0`` + to turn the penalty off, are kept. Every other build keeps the untrimmed, + unpenalized defaults. + """ + + if not hasattr(args, "selection_pi_hi"): + return + sized = posture.role == "dense" and args.dataset_households is not None + if args.selection_pi_hi is None: + args.selection_pi_hi = UK_LOCAL_SIZE_SELECTION_PI_HI if sized else 1.0 + if args.baseline_pi_floor is None: + args.baseline_pi_floor = UK_LOCAL_SIZE_BASELINE_PI_FLOOR if sized else 0.0 + if sized and all( + getattr(args, f"refit_l2_{field}", None) is None + for field in ("lambda", "anchor", "basis") + ): + args.refit_l2_lambda = UK_LOCAL_SIZE_REFIT_L2_LAMBDA + args.refit_l2_anchor = UK_LOCAL_SIZE_REFIT_L2_ANCHOR + args.refit_l2_basis = UK_LOCAL_SIZE_REFIT_L2_BASIS + + _resolve_role_arguments = resolve_role_arguments +def size_doctrine_drift(args: argparse.Namespace) -> list[str]: + """How a sized build's settings depart from the local size doctrine. + + Empty when the build requests :data:`UK_LOCAL_DATASET_HOUSEHOLDS` with + the doctrine's draw ceiling, refit floor and refit L2 penalty, no + selection L2 penalty, draws with the solve seed and selects afresh rather + than resuming a size checkpoint (microcosm#1124, ruling of 2026-10-13). + Only such a sized build can be a release candidate, or be releasable. + """ + + if args.dataset_households is None: + return [] + doctrine = uk_local_size_doctrine() + drift = [ + f"{name} {getattr(args, name, None)!r} != doctrine {value!r}" + for name, value in doctrine.items() + if name != "selection_l2" and getattr(args, name, None) != value + ] + selection_l2 = _size_l2_flags(args, "selection") + if selection_l2["lambda"] not in (None, 0, 0.0) or any( + selection_l2[field] is not None for field in ("anchor", "basis") + ): + drift.append(f"selection L2 penalty {selection_l2!r} != doctrine off") + selection_seed = getattr(args, "selection_seed", None) + if selection_seed is not None and selection_seed != args.seed: + drift.append(f"selection_seed {selection_seed!r} != the solve seed") + if getattr(args, "resume_size_checkpoint", None) is not None: + drift.append( + "resume_size_checkpoint is set: the release draws its own selection" + ) + return drift + + def validate_cli_args(args: argparse.Namespace) -> None: posture = posture_of(args) # The declared role is checked against the parameters first: the other @@ -456,6 +562,15 @@ def validate_cli_args(args: argparse.Namespace) -> None: raise ValueError("--baseline-pi-floor requires --dataset-households.") if args.no_size_checkpoint and args.dataset_households is None: raise ValueError("--no-size-checkpoint requires --dataset-households.") + for stage in ("selection", "refit"): + given = [ + f"--{stage}-l2-{field}" + for field, value in _size_l2_flags(args, stage).items() + if value is not None + ] + if given and args.dataset_households is None: + raise ValueError(f"{', '.join(given)} requires --dataset-households.") + cli_size_l2(args, stage) if args.resume_size_checkpoint is not None: if args.dataset_households is None: raise ValueError("--resume-size-checkpoint requires --dataset-households.") @@ -464,12 +579,19 @@ def validate_cli_args(args: argparse.Namespace) -> None: "--resume-size-checkpoint already implies no new checkpoint; " "drop --no-size-checkpoint." ) + if getattr(args, "inspection_export", False) and args.release_candidate: + raise ValueError( + "--inspection-export is candidate-only: a release candidate never " + "exports past a blocking gate." + ) if args.dataset_households is not None: if args.dataset_households <= 0: raise ValueError("--dataset-households must be positive.") - if args.release_candidate: + drift = size_doctrine_drift(args) + if args.release_candidate and drift: raise ValueError( - "--dataset-households is candidate-only: size-specific matched comparison and promotion scorecard are required before release." + "A sized --release-candidate must carry the local size " + "doctrine (microcosm#1124): " + "; ".join(drift) + "." ) # Size-selection arguments are validated first so their refusals name # the size flag at fault; the pinned Ledger inputs are then mandatory. @@ -682,10 +804,18 @@ def refuse_dense_role_arguments( refused.append("--no-size-checkpoint") if args.resume_size_checkpoint is not None: refused.append("--resume-size-checkpoint") + for stage in ("selection", "refit"): + refused.extend( + f"--{stage}-l2-{field}" + for field, value in _size_l2_flags(args, stage).items() + if value is not None + ) if args.sample_fraction != 1.0: refused.append("--sample-fraction") if "sample_seed" in explicit: refused.append("--sample-seed") + if getattr(args, "inspection_export", False): + refused.append("--inspection-export") if "seed" in explicit and args.seed != posture.seed: # The seam doctrine's seed is a reviewed constant, not a knob. refused.append(f"--seed != doctrine {posture.seed}") diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_posture.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_posture.py index 59c6b02160..58de62f843 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_posture.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_posture.py @@ -36,6 +36,7 @@ ) from microcosm.build.uk_runtime.local_doctrine import ( UK_LOCAL_CLONE_COUNT, + UK_LOCAL_ROW_METADATA_RULES, UK_LOCAL_SOLVE_DOCTRINE, UK_LOCAL_SOLVE_EPOCHS, UKLocalSolveDoctrine, @@ -264,7 +265,13 @@ def _vintage(vintage: str) -> str: learning_rate=UK_DENSE_LEARNING_RATE, seed=UK_DENSE_SEED, target_weight_rule=UK_LOCAL_SOLVE_DOCTRINE.target_weight_rule, - allowed_target_weight_rules=("grain_equal", "uniform"), + # microcosm#1124: the row-metadata rules are receipted candidate-only + # overrides; a release candidate still refuses any rule but the default. + allowed_target_weight_rules=( + "grain_equal", + "uniform", + *UK_LOCAL_ROW_METADATA_RULES, + ), expected_constituency_vintage=UK_DENSE_CONSTITUENCY_VINTAGE, gate_scope=tuple(UK_LOCAL_GATE_SCOPE), gate_posture="local_candidate", diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/size_checkpoint.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/size_checkpoint.py index 1b6870be46..fdecee56eb 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/size_checkpoint.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/size_checkpoint.py @@ -30,7 +30,11 @@ from microcosm.build.uk_runtime.atomic_area_support import ( SOURCES as ATOMIC_SUPPORT_SOURCES, ) -from microcosm.build.uk_runtime.dataset_size import UKSizeSelection +from microcosm.build.uk_runtime.dataset_size import ( + UKSizeSelection, + uk_size_l2, + uk_size_l2_from_options, +) from microcosm.build.uk_runtime.rowwise_cli import doctrine_bounds, posture_of from microcosm.calibrate import ( CalibrationProblem, @@ -106,6 +110,18 @@ def uk_size_checkpoint_identity( # bound while the manifest declares the new one. "doctrine": doctrine_bounds(posture), } + # microcosm#1124: a search under an L2 penalty binds it; an unpenalized + # search adds nothing, so every existing checkpoint keeps its identity. + # The refit's L2 is never bound: re-refitting a stored search under + # another refit penalty is the point. + selection_l2 = uk_size_l2( + "selection", + l2_lambda=getattr(args, "selection_l2_lambda", None) or 0.0, + anchor=getattr(args, "selection_l2_anchor", None), + basis=getattr(args, "selection_l2_basis", None), + ) + if selection_l2 is not None: + identity["selection_l2"] = selection_l2.as_dict() # microcosm#932: an identity-keyed atomic run also binds the assignment # and the three support pins (flat pin roles named by graph source); a # legacy run adds nothing, so its checkpoints keep their identity. @@ -211,6 +227,22 @@ def write_uk_size_checkpoint( n = frame.n("household") if len(dense.weights) != n or len(search.weights) != n: raise ValueError("checkpoint weights must align with the pool households.") + if not np.array_equal( + np.asarray(search.target_loss_weights, dtype=np.float64), + np.asarray(dense.target_loss_weights, dtype=np.float64), + ): + raise ValueError( + "a size checkpoint stores one loss-weight vector: a selection searched " + "under other target weights than the dense solve's cannot be " + "checkpointed." + ) + if uk_size_l2_from_options(search.options) != _normalised(identity).get( + "selection_l2" + ): + raise ValueError( + "size checkpoint identity's selection_l2 disagrees with the L2 penalty " + "the selection was searched under." + ) np.savez( arrays_path, dense_weights=np.asarray(dense.weights, dtype=np.float64), @@ -401,6 +433,12 @@ def load_uk_size_checkpoint_files( _check_closing_loss( "selection", float(selection_meta["closing_loss"]), search.final_loss ) + searched_l2 = uk_size_l2_from_options(search.options) + if searched_l2 != requested_identity.get("selection_l2"): + raise ValueError( + f"size checkpoint selection was searched under L2 {searched_l2!r}; this " + f"run requests {requested_identity.get('selection_l2')!r}." + ) protected = np.asarray(stored["protected"], dtype=bool) if int(np.count_nonzero(protected)) != int(selection_meta["protected_carriers"]): raise ValueError("size checkpoint protected-carrier mask is inconsistent.") diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/size_evaluation.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/size_evaluation.py index 546d44d3e1..1e3f7984e2 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/size_evaluation.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/size_evaluation.py @@ -290,9 +290,16 @@ def add( } ) + # A sized release candidate at the size doctrine may be releasable + # (microcosm#1124, 2026-10-13); every other sized run stays a candidate. + size_release_candidate = bool( + _mapping(manifest.get("parameters")).get("release_candidate") + ) add( "releasable_false", - manifest.get("releasable") is False if is_size else None, + manifest.get("releasable") is False + if is_size and not size_release_candidate + else None, manifest.get("releasable"), False, ) diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/target_weights.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/target_weights.py new file mode 100644 index 0000000000..117d5c2220 --- /dev/null +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/target_weights.py @@ -0,0 +1,621 @@ +"""UK target loss-weight rules derived from row metadata (microcosm#1124). + +The UK local doctrine admits named weighting rules, never per-target vectors +(#762 A2). ``uniform`` and ``grain_equal`` need only each row's grain and stay +in :mod:`microcosm.build.uk_runtime.local_doctrine`. The rules here also read +the row's target family, its geography and its value basis, so this module +carries that metadata in one row-aligned carrier, :class:`UKTargetRows`. The +carrier is built the same way from the declared targets +(:func:`uk_target_rows_from_targets`, used by ``prepare_uk_full_solve``) and +from a stored ordered problem (:func:`uk_target_rows_from_problem`, used by +the size-experiment harness), so a rule computed either way is one vector. + +Rules (G grains, F_g families in grain g, n rows in a (grain, family) cell; +every vector sums to one): + +``grain_family_equal`` + Each grain (national, constituency, local authority) takes one equal + share, each family inside a grain one equal share of it, rows inside a + family equal shares: ``w = 1 / (G * F_g * n)``. +``nation_grain_family_equal`` + The same with a fourth grain, ``nation_region``: every national row whose + geography is not one of the UK-wide aggregates + (:data:`~microcosm.calibrate.geography_constants.UK_WIDE_GEOGRAPHY_IDS`: + the United Kingdom, Great Britain, England and Wales) — the four nations + and the English regions, one ITL1 tier — leaves the national grain for + it. The four grains share equally. + + Family budgets are set per grain, so a family with rows at UK level and + at nation or region level takes a budget in each: its nation rows, which + are sub-aggregates of the UK rows, are weighted twice over. A family with + a single nation-level row takes a whole family share on that row, and an + England-only family sits in ``nation_region`` because its geography is + England. This double budget is known and kept for now (review of + microcosm#1124), so that the K=15 build changes only K against the K=25 + candidate; assigning each family to one grain is the alternative. +``*_sqrt_count`` + Inside each (grain, family) cell the count rows share the cell's count + budget in proportion to ``max(|value|, 1) ** 0.5`` (the US value-weight + power, microcosm#1104); amount rows keep equal shares. Every grain, family + and cell budget is unchanged: the modifier only moves weight from small + count cells to large ones inside one family. + +Local families are the census families of +:mod:`microcosm.build.uk_runtime.local_target_census`, not the Ledger source +families, which split council tax by nation. +""" + +from __future__ import annotations + +import hashlib +import json +from collections.abc import Mapping, Sequence +from dataclasses import dataclass +from typing import Any + +import numpy as np + +from microcosm.build.uk_runtime.local_doctrine import ( + UK_LOCAL_ROW_METADATA_RULES, + UK_LOCAL_TARGET_WEIGHT_RULES, + uk_local_target_loss_weights, +) +from microcosm.build.uk_runtime.local_targets import AREA_TYPES, area_groups_from_codes +from microcosm.calibrate.geography_constants import UK_WIDE_GEOGRAPHY_IDS + +__all__ = [ + "NATIONAL_GRAIN", + "NATION_REGION_GRAIN", + "UK_TARGET_WEIGHT_RULE_VERSION", + "UKStageTargetWeighting", + "UKTargetRows", + "uk_rule_loss_weights", + "uk_target_loss_weight_receipt", + "uk_target_loss_weights_for_rows", + "uk_target_loss_weights_sha256", + "uk_target_rows", + "uk_target_rows_from_problem", + "uk_target_rows_from_targets", + "uk_target_value_basis", +] + +NATIONAL_GRAIN = "national" +NATION_REGION_GRAIN = "nation_region" +UK_WIDE_LABEL = "UK-wide" +#: Bumped whenever a rule's arithmetic changes, so receipts stay comparable. +UK_TARGET_WEIGHT_RULE_VERSION = 1 +LOCAL_SURFACE_MATERIALIZATION = "uk_local_surface" + +_COUNT, _AMOUNT = "count", "amount" +_LOCAL_AMOUNT_METRIC_PREFIXES = ("rent/", "ons/equiv_") +_LOCAL_COUNT_METRIC_PREFIXES = ("age/", "council_tax/", "tenure/", "uc_") +_LOCAL_COUNT_METRICS = frozenset({"households"}) +_NATIONAL_COUNT_UNITS = frozenset({"count"}) +_NATIONAL_AMOUNT_UNIT_PREFIX = "gbp" + + +@dataclass(frozen=True) +class UKTargetRows: + """The metadata a target-weight rule may read, aligned to the target axis. + + ``grain`` is ``"constituency"`` or ``"la"`` for local rows and + ``"national"`` for every national row; ``family`` the census family + (local) or the register family (national); ``geography`` the area code + (local) or the national geography id; ``measure`` the local metric or + the national ``ledger_measure_unit``; ``values`` the target values. + """ + + names: tuple[str, ...] + local: tuple[bool, ...] + grain: tuple[str, ...] + family: tuple[str, ...] + geography: tuple[str, ...] + measure: tuple[str, ...] + values: np.ndarray + + def __post_init__(self) -> None: + n = len(self.names) + for label in ("local", "grain", "family", "geography", "measure"): + if len(getattr(self, label)) != n: + raise ValueError( + f"target rows: {label} has {len(getattr(self, label))} " + f"entries for {n} names." + ) + if len(set(self.names)) != n: + raise ValueError("target rows: row names must be unique.") + values = np.array(self.values, dtype=np.float64) + if values.shape != (n,) or not np.isfinite(values).all(): + raise ValueError("target rows: values must be n finite numbers.") + values.flags.writeable = False + object.__setattr__(self, "values", values) + for name, local, grain, family in zip( + self.names, self.local, self.grain, self.family, strict=True + ): + if local and grain not in AREA_TYPES: + raise ValueError( + f"local target {name!r} has grain {grain!r}, expected one of " + f"{AREA_TYPES}." + ) + if not local and grain != NATIONAL_GRAIN: + raise ValueError( + f"national target {name!r} has grain {grain!r}, expected " + f"{NATIONAL_GRAIN!r}." + ) + if not family: + raise ValueError(f"target {name!r} declares no family.") + + def __len__(self) -> int: + return len(self.names) + + def take(self, indices: Sequence[int] | np.ndarray) -> UKTargetRows: + """The rows at ``indices`` (unique, in the given order).""" + + index = np.asarray(indices, dtype=np.int64) + if index.ndim != 1 or len(np.unique(index)) != len(index): + raise ValueError("target rows: take() needs unique row positions.") + if len(index) and (index.min() < 0 or index.max() >= len(self)): + raise ValueError("target rows: take() position out of range.") + positions = index.tolist() + return UKTargetRows( + names=tuple(self.names[i] for i in positions), + local=tuple(self.local[i] for i in positions), + grain=tuple(self.grain[i] for i in positions), + family=tuple(self.family[i] for i in positions), + geography=tuple(self.geography[i] for i in positions), + measure=tuple(self.measure[i] for i in positions), + values=self.values[index], + ) + + +def uk_target_rows( + names: Sequence[str], + metadata: Sequence[Mapping[str, Any]], + values: Sequence[float] | np.ndarray, + *, + local: Sequence[bool], +) -> UKTargetRows: + """Build the carrier from per-row metadata, the one core both builders use. + + A local row needs ``area_type`` (a local grain), ``area_code`` and + ``metric``; its ``family`` falls back to the census classifier of its + metric. A national row needs a ``family``; its geography is the local + spelling ``geography_id`` or the Ledger ``ledger_geography_id``, its basis + the ``ledger_measure_unit``. + """ + + names = tuple(str(name) for name in names) + rows = tuple(metadata) + flags = tuple(bool(flag) for flag in local) + if not (len(names) == len(rows) == len(flags)): + raise ValueError( + "target rows: names, metadata and local flags must align " + f"({len(names)}, {len(rows)}, {len(flags)})." + ) + grain: list[str] = [] + family: list[str] = [] + geography: list[str] = [] + measure: list[str] = [] + for name, row, is_local in zip(names, rows, flags, strict=True): + area_type = row.get("area_type") + if is_local: + if area_type not in AREA_TYPES: + raise ValueError( + f"local target {name!r} has area_type {area_type!r}, expected " + f"one of {AREA_TYPES}." + ) + metric = str(row.get("metric") or "") + code = str(row.get("area_code") or "") + if not metric or not code: + raise ValueError( + f"local target {name!r} needs a metric and an area code." + ) + declared = row.get("family") + if not declared: + from microcosm.build.uk_runtime import local_target_census + + declared = local_target_census.family_for_metric(metric) + grain.append(str(area_type)) + family.append(str(declared)) + geography.append(code) + measure.append(metric) + else: + if area_type in AREA_TYPES: + raise ValueError( + f"national target {name!r} carries the local area_type " + f"{area_type!r}." + ) + grain.append(NATIONAL_GRAIN) + family.append(str(row.get("family") or "")) + geography.append( + str(row.get("geography_id") or row.get("ledger_geography_id") or "") + ) + measure.append(str(row.get("ledger_measure_unit") or "")) + return UKTargetRows( + names=names, + local=flags, + grain=tuple(grain), + family=tuple(family), + geography=tuple(geography), + measure=tuple(measure), + values=np.asarray(values, dtype=np.float64), + ) + + +def uk_target_rows_from_targets( + local_targets: Sequence[Any], + national_specs: Sequence[Any] = (), +) -> UKTargetRows: + """The carrier for a joint solve: local targets first, then national specs. + + ``local_targets`` are the rowwise surface's :class:`~microcosm.calibrate.Target` + rows (their metadata names the area type, area code, metric and family); + ``national_specs`` the national register's + :class:`~microcosm.calibrate.TargetSpec` rows, in the solve's order. + """ + + local_targets = tuple(local_targets) + national_specs = tuple(national_specs) + names = [target.row_name for target in local_targets] + [ + spec.to_target().row_name for spec in national_specs + ] + metadata = [dict(target.metadata) for target in local_targets] + [ + {**spec.metadata, "family": spec.family} for spec in national_specs + ] + values = [float(target.value) for target in local_targets] + [ + float(spec.value) for spec in national_specs + ] + local = [True] * len(local_targets) + [False] * len(national_specs) + return uk_target_rows(names, metadata, values, local=local) + + +def uk_target_rows_from_problem(problem: Any) -> UKTargetRows: + """The carrier for a stored ordered problem (``calibrate.artifacts``). + + A row is local when its metadata says ``materialization == + "uk_local_surface"``; a row whose materialization and area type disagree + is refused rather than guessed. + """ + + names = tuple(problem.problem.names) + metadata = tuple(problem.target_metadata) + local = [] + for name, row in zip(names, metadata, strict=True): + is_local = row.get("materialization") == LOCAL_SURFACE_MATERIALIZATION + if is_local != (row.get("area_type") in AREA_TYPES): + raise ValueError( + f"target {name!r}: materialization {row.get('materialization')!r} " + f"disagrees with area_type {row.get('area_type')!r}." + ) + local.append(is_local) + return uk_target_rows( + names, + metadata, + np.asarray(problem.problem.target_vector, dtype=np.float64), + local=local, + ) + + +def _local_basis(metric: str) -> str | None: + if metric.startswith("hmrc/"): + if metric.endswith("/amount"): + return _AMOUNT + if metric.endswith("/count"): + return _COUNT + return None + if metric.startswith(_LOCAL_AMOUNT_METRIC_PREFIXES): + return _AMOUNT + if metric in _LOCAL_COUNT_METRICS or metric.startswith( + _LOCAL_COUNT_METRIC_PREFIXES + ): + return _COUNT + return None + + +def _national_basis(unit: str) -> str | None: + if unit in _NATIONAL_COUNT_UNITS: + return _COUNT + if unit.startswith(_NATIONAL_AMOUNT_UNIT_PREFIX): + return _AMOUNT + return None + + +def uk_target_value_basis(rows: UKTargetRows) -> tuple[str, ...]: + """Each row's value basis, ``"count"`` or ``"amount"``, refusing unknowns. + + Local rows by their metric (amounts: ``hmrc/*/amount``, ``rent/*``, + ``ons/equiv_*``; counts: ``hmrc/*/count``, ``age/*``, ``households``, + ``uc_*``, ``council_tax/*``, ``tenure/*``), national rows by their + ``ledger_measure_unit`` (``count``; ``gbp`` and its ``gbp_*`` variants). + """ + + bases = [] + for name, local, measure in zip(rows.names, rows.local, rows.measure, strict=True): + basis = _local_basis(measure) if local else _national_basis(measure) + if basis is None: + kind = "metric" if local else "ledger_measure_unit" + raise ValueError( + f"target {name!r}: {kind} {measure!r} has no declared count or " + "amount basis; the *_sqrt_count rules refuse an undeclared basis." + ) + bases.append(basis) + return tuple(bases) + + +def _rule_grains(rows: UKTargetRows, *, nation: bool) -> tuple[str, ...]: + if not nation: + return rows.grain + grains = [] + for name, grain, geography in zip( + rows.names, rows.grain, rows.geography, strict=True + ): + if grain != NATIONAL_GRAIN: + grains.append(grain) + continue + if not geography: + raise ValueError( + f"national target {name!r} names no geography; the nation grain " + "needs one to place it." + ) + grains.append( + NATIONAL_GRAIN + if geography in UK_WIDE_GEOGRAPHY_IDS + else NATION_REGION_GRAIN + ) + return tuple(grains) + + +def _check_row_rule(rule: str) -> None: + if rule not in UK_LOCAL_ROW_METADATA_RULES: + raise ValueError( + f"row-metadata target_weight_rule must be one of " + f"{UK_LOCAL_ROW_METADATA_RULES}, got {rule!r}." + ) + + +def uk_target_loss_weights_for_rows(rows: UKTargetRows, *, rule: str) -> np.ndarray: + """The loss-weight vector of a row-metadata rule (sums to one).""" + + _check_row_rule(rule) + if not len(rows): + raise ValueError("target weighting requires at least one target.") + nation = rule.startswith("nation_") + sqrt_count = rule.endswith("_sqrt_count") + grains = np.asarray(_rule_grains(rows, nation=nation), dtype=object) + families = np.asarray(rows.family, dtype=object) + bases = ( + np.asarray(uk_target_value_basis(rows), dtype=object) if sqrt_count else None + ) + magnitudes = np.maximum(np.abs(rows.values), 1.0) ** 0.5 if sqrt_count else None + grain_order = list(dict.fromkeys(grains.tolist())) + weights = np.zeros(len(rows), dtype=np.float64) + for grain in grain_order: + in_grain = grains == grain + family_order = list(dict.fromkeys(families[in_grain].tolist())) + budget = 1.0 / (len(grain_order) * len(family_order)) + for family in family_order: + cell = np.flatnonzero(in_grain & (families == family)) + if bases is None: + weights[cell] = budget / cell.size + continue + counts = cell[bases[cell] == _COUNT] + amounts = cell[bases[cell] == _AMOUNT] + weights[amounts] = budget / cell.size + if counts.size: + share = magnitudes[counts] + weights[counts] = ( + (budget * counts.size / cell.size) * share / share.sum() + ) + return weights + + +def uk_rule_loss_weights(rows: UKTargetRows, *, rule: str) -> np.ndarray: + """The loss-weight vector of any rule in the local vocabulary. + + ``uniform`` is ones (the vector a graph problem binds for it); + ``grain_equal`` is :func:`~microcosm.build.uk_runtime.local_doctrine.uk_local_target_loss_weights` + on the rows' grains, bit for bit the vector ``prepare_uk_full_solve`` + binds; the row-metadata rules come from + :func:`uk_target_loss_weights_for_rows`. + """ + + if rule not in UK_LOCAL_TARGET_WEIGHT_RULES: + raise ValueError( + f"target_weight_rule must be one of {UK_LOCAL_TARGET_WEIGHT_RULES}, " + f"got {rule!r}." + ) + if rule == "uniform": + return np.ones(len(rows), dtype=np.float64) + if rule == "grain_equal": + return np.asarray( + uk_local_target_loss_weights(list(rows.grain), rule="grain_equal"), + dtype=np.float64, + ) + return uk_target_loss_weights_for_rows(rows, rule=rule) + + +def uk_target_loss_weights_sha256(names: Sequence[str], weights: np.ndarray) -> str: + """Content digest of a row-aligned loss-weight vector. + + The canonical form of the US national release's ``loss_vector_sha256`` + (``us_runtime.target_loss_weights.target_loss_weights_sha256``): one + ``{"row_name", "weight_hex"}`` object per row, in row order, as compact + sorted-key JSON. Any change to a name, a weight bit or the order changes it. + """ + + weights = np.asarray(weights, dtype=np.float64) + if weights.shape != (len(names),): + raise ValueError( + f"{weights.shape} loss weights do not align with {len(names)} names." + ) + rows = [ + {"row_name": str(name), "weight_hex": float(weight).hex()} + for name, weight in zip(names, weights, strict=True) + ] + return hashlib.sha256( + json.dumps(rows, sort_keys=True, separators=(",", ":"), allow_nan=False).encode( + "utf-8" + ) + ).hexdigest() + + +def _nations(geographies: Sequence[str]) -> tuple[str, ...]: + """Each row's nation: UK-wide aggregates, else the area-code prefix rule.""" + + coded = sorted( + {code for code in geographies if code and code not in UK_WIDE_GEOGRAPHY_IDS} + ) + groups = area_groups_from_codes(coded) if coded else {} + return tuple( + UK_WIDE_LABEL if code in UK_WIDE_GEOGRAPHY_IDS else groups.get(code, "unknown") + for code in geographies + ) + + +def uk_target_loss_weight_receipt( + rows: UKTargetRows, + weights: np.ndarray, + *, + rule: str, +) -> dict[str, object]: + """Loss shares of a weight vector by rule grain, grain × family and nation. + + ``loss_share`` is a cell's share of the total weight (the share of the + loss it carries at equal misses); ``equal_share`` what it would carry + with every target weighted equally. + """ + + weights = np.asarray(weights, dtype=np.float64) + if weights.shape != (len(rows),): + raise ValueError( + f"{weights.shape} loss weights do not align with {len(rows)} rows." + ) + total = float(weights.sum()) + n = len(rows) + nation_rule = rule in UK_LOCAL_ROW_METADATA_RULES and rule.startswith("nation_") + grains = np.asarray(_rule_grains(rows, nation=nation_rule), dtype=object) + families = np.asarray(rows.family, dtype=object) + nations = np.asarray(_nations(rows.geography), dtype=object) + sqrt_count = rule in UK_LOCAL_ROW_METADATA_RULES and rule.endswith("_sqrt_count") + bases = ( + np.asarray(uk_target_value_basis(rows), dtype=object) if sqrt_count else None + ) + + def cell(mask: np.ndarray) -> dict[str, object]: + selected = weights[mask] + block: dict[str, object] = { + "n_targets": int(mask.sum()), + "loss_share": float(selected.sum()) / total if total else 0.0, + "equal_share": float(mask.sum()) / n if n else 0.0, + "min_weight": float(selected.min()) if selected.size else None, + "max_weight": float(selected.max()) if selected.size else None, + } + if bases is not None: + block["n_count"] = int((bases[mask] == _COUNT).sum()) + block["n_amount"] = int((bases[mask] == _AMOUNT).sum()) + return block + + grain_order = list(dict.fromkeys(grains.tolist())) + cells = [] + for grain in grain_order: + for family in dict.fromkeys(families[grains == grain].tolist()): + cells.append( + { + "grain": grain, + "family": family, + **cell((grains == grain) & (families == family)), + } + ) + return { + "rule": rule, + "rule_version": UK_TARGET_WEIGHT_RULE_VERSION, + "n_targets": n, + "weights_sha256": uk_target_loss_weights_sha256(rows.names, weights), + "grains": [{"grain": grain, **cell(grains == grain)} for grain in grain_order], + "cells": cells, + "nations": [ + {"nation": nation, **cell(nations == nation)} + for nation in sorted(set(nations.tolist())) + ], + } + + +@dataclass(frozen=True) +class UKStageTargetWeighting: + """One stage's loss weights, computed from a named rule (the size seam). + + The dataset-size search and refit inherit the dense solve's weights + unless a caller hands them one of these: the rule is computed on + :attr:`rows`, which must be the stage problem's own row axis (checked by + name). ``held_out`` rows get zero weight and the rule is applied to the + remaining rows, as the production holdout re-applies the rule to its + training problem. Never a raw vector: a per-target vector cannot enter. + """ + + rule: str + rows: UKTargetRows + held_out: tuple[int, ...] = () + label: str | None = None + + def __post_init__(self) -> None: + if self.rule not in UK_LOCAL_TARGET_WEIGHT_RULES: + raise ValueError( + f"target_weight_rule must be one of {UK_LOCAL_TARGET_WEIGHT_RULES}, " + f"got {self.rule!r}." + ) + if not isinstance(self.rows, UKTargetRows): + raise TypeError("stage target weighting needs a UKTargetRows carrier.") + held = tuple(sorted(int(index) for index in self.held_out)) + if len(set(held)) != len(held): + raise ValueError("held-out rows must be unique.") + if held and (held[0] < 0 or held[-1] >= len(self.rows)): + raise ValueError("held-out row position out of range.") + if len(held) >= len(self.rows): + raise ValueError("a stage weighting must train on at least one row.") + object.__setattr__(self, "held_out", held) + + def training_positions(self) -> np.ndarray: + return np.setdiff1d( + np.arange(len(self.rows), dtype=np.int64), + np.asarray(self.held_out, dtype=np.int64), + assume_unique=True, + ) + + def _require_axis(self, names: Sequence[str]) -> None: + if tuple(str(name) for name in names) != self.rows.names: + raise ValueError( + "stage target weighting was built for another row axis: the " + "stage problem's target names differ from the carrier's." + ) + + def loss_weights(self, names: Sequence[str]) -> np.ndarray: + """The full-length training weight vector for the problem rows ``names``.""" + + self._require_axis(names) + if not self.held_out: + return uk_rule_loss_weights(self.rows, rule=self.rule) + training = self.training_positions() + weights = np.zeros(len(self.rows), dtype=np.float64) + weights[training] = uk_rule_loss_weights( + self.rows.take(training), rule=self.rule + ) + return weights + + def held_out_weights(self) -> np.ndarray: + """The rule applied to the held-out rows alone (their scoring weights).""" + + if not self.held_out: + raise ValueError("no rows are held out.") + return uk_rule_loss_weights( + self.rows.take(np.asarray(self.held_out, dtype=np.int64)), rule=self.rule + ) + + def receipt(self, names: Sequence[str]) -> dict[str, object]: + weights = self.loss_weights(names) + return { + "rule": self.rule, + "rule_version": UK_TARGET_WEIGHT_RULE_VERSION, + "label": self.label, + "n_targets": len(self.rows), + "n_held_out": len(self.held_out), + "weights_sha256": uk_target_loss_weights_sha256(self.rows.names, weights), + } diff --git a/packages/microcosm-build/src/microcosm/build/uk_runtime/weighted_integrity.py b/packages/microcosm-build/src/microcosm/build/uk_runtime/weighted_integrity.py index 9e4a64966b..9bb6306496 100644 --- a/packages/microcosm-build/src/microcosm/build/uk_runtime/weighted_integrity.py +++ b/packages/microcosm-build/src/microcosm/build/uk_runtime/weighted_integrity.py @@ -140,6 +140,7 @@ "UKQRFTailConcentrationPolicy", "UKReviewedExclusion", "UK_INPUT_MASS_REFERENCE_REGISTRY", + "UK_RELEASE_LINES", "coerce_input_mass_reference_registry", "coerce_reviewed_exclusions", "exclusion_evaluation_date", @@ -262,6 +263,11 @@ def spec_payload(self) -> dict[str, object]: _REVIEWED_EXCLUSION_FIELDS = frozenset( {"reason", "approved_by", "adjudication", "approved_on", "expires_on"} ) +#: An entry may name the release lines whose evidence earned it. +_REVIEWED_EXCLUSION_OPTIONAL_FIELDS = frozenset({"release_lines"}) +#: The UK release lines a gate context can declare (``release_line``): the +#: national seam and the geographic K-clone pool of the local line. +UK_RELEASE_LINES = frozenset({"national", "local"}) @dataclass(frozen=True) @@ -285,8 +291,24 @@ class UKReviewedExclusion: adjudication: str approved_on: str expires_on: str + release_lines: tuple[str, ...] | None = None def __post_init__(self) -> None: + if self.release_lines is not None: + lines = self.release_lines + if ( + isinstance(lines, str) + or not isinstance(lines, tuple | list) + or not lines + or any(line not in UK_RELEASE_LINES for line in lines) + or len(set(lines)) != len(lines) + ): + raise ValueError( + "UK reviewed exclusion release_lines must be a non-empty " + f"list of distinct lines from {sorted(UK_RELEASE_LINES)}; " + f"got {lines!r}." + ) + object.__setattr__(self, "release_lines", tuple(sorted(lines))) for name in ("reason", "approved_by", "adjudication"): value = getattr(self, name) if not isinstance(value, str) or not value.strip(): @@ -335,16 +357,38 @@ def premature(self, now: date) -> bool: return now < date.fromisoformat(self.approved_on) - def policy_payload(self) -> dict[str, str]: + def applies_on(self, line: str | None) -> bool: + """Whether the entry is in scope on a release line. + + An entry without ``release_lines`` applies on every line. A scoped + entry applies only where the context declares one of its lines, and + stands aside elsewhere, including where no line is declared: it then + neither suppresses a failure nor reads as stale, because that context's + evidence never earned it (a register is shared by the national seam and + the K-clone local pool, whose record counts differ by a factor of K). + Failing closed keeps a scoped entry from applying in a context that + forgot to declare its line. + """ + + if self.release_lines is None: + return True + return line is not None and line in self.release_lines + + def policy_payload(self) -> dict[str, object]: """The complete record, sealed into the policy digest.""" - return { + payload: dict[str, object] = { "reason": self.reason, "approved_by": self.approved_by, "adjudication": self.adjudication, "approved_on": self.approved_on, "expires_on": self.expires_on, } + # Unscoped records keep their pre-scope payload, and with it every + # policy digest already pinned. + if self.release_lines is not None: + payload["release_lines"] = list(self.release_lines) + return payload def coerce_reviewed_exclusions( @@ -384,16 +428,28 @@ def coerce_reviewed_exclusions( f"fields {sorted(_REVIEWED_EXCLUSION_FIELDS)} (schema 2); " f"got {type(entry).__name__}." ) - if set(entry) != _REVIEWED_EXCLUSION_FIELDS: + if not _REVIEWED_EXCLUSION_FIELDS <= set(entry) or ( + set(entry) + - _REVIEWED_EXCLUSION_FIELDS + - _REVIEWED_EXCLUSION_OPTIONAL_FIELDS + ): raise ValueError( f"{label} reviewed exclusion {name!r} fields must be exactly " - f"{sorted(_REVIEWED_EXCLUSION_FIELDS)}, got {sorted(entry)}." + f"{sorted(_REVIEWED_EXCLUSION_FIELDS)}, optionally with " + f"{sorted(_REVIEWED_EXCLUSION_OPTIONAL_FIELDS)}; got " + f"{sorted(entry)}." ) try: records[name] = UKReviewedExclusion( **{ field_name: entry[field_name] - for field_name in sorted(_REVIEWED_EXCLUSION_FIELDS) + for field_name in sorted( + set(entry) + & ( + _REVIEWED_EXCLUSION_FIELDS + | _REVIEWED_EXCLUSION_OPTIONAL_FIELDS + ) + ) } ) except ValueError as exc: @@ -1141,7 +1197,12 @@ def uk_qrf_tail_concentration_columns( """Assemble the tail-concentration evidence for the declared QRF surface. The surface defaults to every output declared by a - ``fit_weighted_qrf_stage*`` operation in the HMRC source manifest. + ``fit_weighted_qrf_stage*`` operation in the HMRC source manifest, less the + report columns ``spi_benefit_coherence`` replaces on every SPI-channel row + (microcosm#1100, read from that stage's declaration): after it those + columns hold only FRS respondents' own reports, no forest draw, so the + #462 broadcast cannot occur in them. They are listed in the surface as + ``superseded_by_spi_benefit_coherence``. Unlike the US wrapper, there is **no sparsity filter**: on the UK staging pool the SPI channel is a large fraction of all persons, so a nonzero-share cutoff would silently skip the gate exactly where the risk @@ -1155,11 +1216,16 @@ def uk_qrf_tail_concentration_columns( # at import time. from microcosm.build.uk_runtime.hmrc_source_contract import ( uk_hmrc_weighted_qrf_output_columns, + uk_spi_coherence_superseded_columns, ) - declared = uk_hmrc_weighted_qrf_output_columns() + superseded_columns = set(uk_spi_coherence_superseded_columns()) + all_declared = uk_hmrc_weighted_qrf_output_columns() + declared = tuple(c for c in all_declared if c not in superseded_columns) + superseded = [c for c in all_declared if c in superseded_columns] else: declared = tuple(str(name) for name in output_columns) + superseded = [] if not declared: raise ValueError("UK QRF tail-concentration surface must be non-empty.") @@ -1196,6 +1262,7 @@ def uk_qrf_tail_concentration_columns( "absent_columns": absent, "non_numeric_columns": non_numeric, "density_filter": "none: every declared output is checked (#609)", + "superseded_by_spi_benefit_coherence": superseded, } return values, weights, surface diff --git a/packages/microcosm-build/tests/engine_free/shared/test_emitter_adapter.py b/packages/microcosm-build/tests/engine_free/shared/test_emitter_adapter.py index 159ffb743c..ab5c70fd8f 100644 --- a/packages/microcosm-build/tests/engine_free/shared/test_emitter_adapter.py +++ b/packages/microcosm-build/tests/engine_free/shared/test_emitter_adapter.py @@ -66,7 +66,7 @@ def start(modules, **options): assert modules["telemetry"]["registration"]["run_id"] == "run-one" assert modules["telemetry"]["heartbeat_seconds"] == 123 assert modules["telemetry"]["identity"] == {"commit": "synthetic"} - assert options["startup_timeout"] == 3 + assert options["startup_timeout"] == 15 # The service emits one run-start event from its startup configuration. # The build adapter must not send a second run-start request. assert messages == [] diff --git a/packages/microcosm-build/tests/engine_free/shared/test_spec_engine_loader.py b/packages/microcosm-build/tests/engine_free/shared/test_spec_engine_loader.py index 070bbc6d23..b8c2c7eb5e 100644 --- a/packages/microcosm-build/tests/engine_free/shared/test_spec_engine_loader.py +++ b/packages/microcosm-build/tests/engine_free/shared/test_spec_engine_loader.py @@ -10,9 +10,11 @@ def test_semantic_hash_has_golden_vector_and_surface_separation(tmp_path) -> Non kernel_registry=KernelRegistry.from_ids(SELECTION_KERNEL_IDS), ) # Pin the domain separator, normalization rules, schema-set receipt, and - # exact normative projection as one reviewable golden vector. + # exact normative projection as one reviewable golden vector. The seed + # protocol inside it attests microcosm.calibrate.solve's bytes, so + # microcosm#1124's opt-in retain_best_objective moves it. assert first.spec_sha256 == ( - "0c0c8eea8f5d350f3a73867aebfcb82450cb606124dc85cc3772cc2eae971ddb" + "0beed241fbee407e76639d830853aa46aabbdb33c57df22a942dfee85ca1f404" ) second_root = _rich_minimal(tmp_path / "xy", note="second", store="local:b") diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_battery_bindings.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_battery_bindings.py index 499711977f..84f947f2b7 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_battery_bindings.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_battery_bindings.py @@ -65,7 +65,8 @@ # Midlands target-fit deferral is retired with microcosm#1106), and the # microcosm#1106 property_wealth input-mass entry on 2026-10-11. The earliest # expiry is 2026-10-15. -CLOCK = date(2026, 10, 11) +# On or after the newest committed receipt's approval (2026-10-14). +CLOCK = date(2026, 10, 14) VALIDATE_REFERENCE = ( "microcosm.build.uk_runtime.weighted_integrity." @@ -1203,6 +1204,65 @@ def test_resupplying_the_committed_register_is_not_an_override(self) -> None: assert resupplied == committed assert resupplied["exclusions_policy"] == "committed" + @pytest.mark.parametrize( + ("line", "stands_aside"), + # Where no line is declared a scoped entry stands aside (fails closed). + [("local", True), ("national", False), (None, True)], + ) + def test_a_line_scoped_entry_stands_aside_off_its_line( + self, line, stands_aside + ) -> None: + # employment_income carries signal in the fixture, so an exclusion for + # it is stale wherever it applies. + scoped = { + "person.employment_income": { + "reason": "injected: earned on the national line's evidence only", + "approved_by": "juaristi22", + "adjudication": "microcosm#1124", + "approved_on": "2026-08-26", + "expires_on": "2027-02-26", + "release_lines": ["national"], + } + } + extra = {"reviewed_degenerate_exclusions": scoped} + if line is not None: + extra["release_line"] = line + battery = _run_battery( + _tables(), + parity=_parity(), + fit_records=(FitWeightRecord("spi_qrf", "importance"),), + extra_artifacts=extra, + ) + outcome = {o.entry.id: o for o in battery.outcomes}[ + "uk_degenerate_release_surface" + ] + details = outcome.result.details + stale = "person.employment_income" in details["stale_exclusions"] + assert stale is not stands_aside + if stands_aside: + assert details["release_line"] == line + assert details["out_of_line_exclusions"] == ["person.employment_income"] + assert outcome.status is GateStatus.PASSED + else: + assert "out_of_line_exclusions" not in details + assert outcome.status is GateStatus.FAILED + + def test_an_unknown_release_line_is_refused(self) -> None: + binding = UK_GATE_REGISTRY["tail_concentration"] + context = EvidenceContext( + artifacts={"exclusions_evaluated_on": CLOCK, "release_line": "regional"} + ) + with pytest.raises(ValueError, match="release_line must be one of"): + binding.evaluate( + context, + { + "reviewed_exclusions_resource": "qrf_tail_reviewed_exclusions.json", + "top_k": 1, + "max_top_share": 0.5, + "min_nonzero_records": 2, + }, + ) + def test_a_datetime_clock_is_refused(self, uk_gates) -> None: person, benunit, household = _tables() frame = uk_national_frame( diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_dense_release_assembler.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_dense_release_assembler.py index 95bfa6228f..7ff3f726ab 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_dense_release_assembler.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_dense_release_assembler.py @@ -192,13 +192,13 @@ def _candidate_dir(root: Path) -> tuple[Path, Path, Path]: "git_commit": "b" * 40, "parameters": { "release_candidate": True, - "epochs": 1500, + "epochs": 2000, "n_clones": 15, "skip_holdout": False, "doctrine": { "max_weight_ratio": 10.0, - "target_weight_rule": "grain_equal", - "solve_epochs": 1500, + "target_weight_rule": "nation_grain_family_equal_sqrt_count", + "solve_epochs": 2000, "clone_count": 15, "target_loss_cap": 10.0, "scale_rule": "default_target_loss_scales", @@ -865,11 +865,19 @@ def test_assembler_refuses_a_national_role_manifest( ) +@pytest.mark.parametrize( + "sized", + [False, True], + ids=["whole-pool", "doctrine-sized"], +) def test_assembler_stages_a_graph_built_dense_candidate( - tmp_path: Path, monkeypatch, capsys + tmp_path: Path, monkeypatch, capsys, sized: bool ) -> None: """The graph driver's bundle assembles like a rowwise-tool bundle. + A release candidate sized at the size doctrine (microcosm#1124, ruling of + 2026-10-13) assembles on the same contract as the whole pool. + ``main`` runs the synthetic graph with local-only staging, so the manifest carries the staging receipt, the staged-dataset sidecars and the Logbook row the assembler hash-joins. The build signs its own local battery report, @@ -888,10 +896,12 @@ def test_assembler_stages_a_graph_built_dense_candidate( monkeypatch.setenv("MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY", KEY) monkeypatch.setattr(cli, "git_commit", lambda: "b" * 40) monkeypatch.setattr(cli, "git_dirty", lambda: False) + size = ("--dataset-households", "100000") if sized else () candidate = graph_dense_bundle( tmp_path, monkeypatch, "--release-candidate", + *size, staging="--staging-local-only", ) # The graph driver prints the finished manifest to stdout, as the tool does. @@ -903,6 +913,14 @@ def test_assembler_stages_a_graph_built_dense_candidate( manifest = json.loads(manifest_path.read_text()) assert manifest["staging_delivery"]["mode"] == "local_only" assert manifest["staged_dataset"]["status"] == "skipped" + assert manifest["releasable"] is True + if sized: + assert manifest["parameters"]["dataset_households"] == 100_000 + assert manifest["release_posture"]["size_doctrine"] is True + assert manifest["solve"]["dataset_size"]["realized_households"] == 100_000 + else: + assert manifest["solve"]["dataset_size"] is None + assert "size_doctrine" not in manifest["release_posture"] report_path = candidate / f"{STEM}.local_gates.json" report = json.loads(report_path.read_text()) assert report["release_id"] == row.build_id and report["shippable"] is True diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_full_build_cli.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_full_build_cli.py index 46547b9eb7..91512f7e4f 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_full_build_cli.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_full_build_cli.py @@ -85,8 +85,8 @@ def test_release_role_is_required_and_supplies_the_dense_posture(tmp_path): posture.epochs, posture.learning_rate, ) - assert (args.n_clones, args.epochs, args.learning_rate) == (15, 1500, 0.15) - assert args.target_weight_rule == "grain_equal" + assert (args.n_clones, args.epochs, args.learning_rate) == (15, 2000, 0.15) + assert args.target_weight_rule == "nation_grain_family_equal_sqrt_count" assert args.expected_constituency_vintage == "2024_pcon" assert args.sample_seed == 578 assert args.staging_upload_interval_seconds == 300.0 @@ -113,6 +113,88 @@ def test_dense_role_refuses_the_national_knobs(tmp_path, extra, needle): assert needle in str(excinfo.value) +def test_dense_role_admits_the_row_metadata_rules_as_candidate_overrides(tmp_path): + """microcosm#1124: candidate-only rules; release candidates and the national role refuse them.""" + for rule in ( + "grain_family_equal", + "grain_family_equal_sqrt_count", + "nation_grain_family_equal", + "nation_grain_family_equal_sqrt_count", + ): + args = arguments(tmp_path, "--target-weight-rule", rule) + cli.validate_cli_args(args) + assert args.target_weight_rule == rule + with pytest.raises(ValueError, match="--release-candidate refuses non-release"): + cli.validate_cli_args( + arguments( + tmp_path, + "--release-candidate", + "--target-weight-rule", + "grain_family_equal", + ) + ) + national = cli.parse_args( + _national_argv(tmp_path, "--target-weight-rule", "grain_family_equal") + ) + with pytest.raises(ValueError, match="--target-weight-rule grain_family_equal"): + cli.validate_cli_args(national) + + +def test_size_stage_l2_flags_are_validated_and_recorded_only_when_on(tmp_path): + """microcosm#1124: candidate-only, size-run-only, refused by the national role.""" + from microcosm.build.uk_runtime.rowwise_cli import rowwise_parameters + + # A sized build takes the doctrine's refit L2 unless it is turned off. + defaulted = arguments(tmp_path, "--dataset-households", "2") + cli.validate_cli_args(defaulted) + assert set(rowwise_parameters(defaulted, source_year=2024)) >= {"refit_l2"} + plain = arguments(tmp_path, "--dataset-households", "2", "--refit-l2-lambda", "0") + cli.validate_cli_args(plain) + assert not {"selection_l2", "refit_l2"} & set( + rowwise_parameters(plain, source_year=2024) + ) + on = arguments( + tmp_path, + "--dataset-households", + "2", + "--refit-l2-lambda", + "0.03", + "--refit-l2-anchor", + "uniform", + "--selection-l2-lambda", + "1e-3", + ) + cli.validate_cli_args(on) + parameters = rowwise_parameters(on, source_year=2024) + assert parameters["refit_l2"] == { + "lambda": 0.03, + "anchor": "uniform", + "basis": "chi_square", + } + assert parameters["selection_l2"]["lambda"] == 1e-3 + with pytest.raises( + ValueError, match="--refit-l2-lambda requires --dataset-households" + ): + cli.validate_cli_args(arguments(tmp_path, "--refit-l2-lambda", "0.03")) + with pytest.raises(ValueError, match="without a positive"): + cli.validate_cli_args( + arguments( + tmp_path, "--dataset-households", "2", "--refit-l2-anchor", "uniform" + ) + ) + with pytest.raises(ValueError, match="finite number"): + cli.validate_cli_args( + arguments( + tmp_path, "--dataset-households", "2", "--selection-l2-lambda", "-1" + ) + ) + with pytest.raises(SystemExit): + arguments(tmp_path, "--refit-l2-basis", "record") + national = cli.parse_args(_national_argv(tmp_path, "--refit-l2-lambda", "0.03")) + with pytest.raises(ValueError, match="--refit-l2-lambda"): + cli.validate_cli_args(national) + + def test_engine_blocks_must_be_positive_and_equal_the_clone_count(tmp_path): """The per-clone engine block count is either one or the clone count.""" with pytest.raises(ValueError, match="must equal --n-clones"): @@ -557,7 +639,7 @@ def test_dense_run_projects_the_rowwise_candidate_manifest(tmp_path): ) assert (manifest["parameters"]["n_clones"], manifest["parameters"]["epochs"]) == ( 15, - 1500, + 2000, ) for key in ( "solve", @@ -2099,3 +2181,125 @@ def terminated(prepared, args, *, telemetry=None, attempt=None): ) rows = load_spool_rows(out / "logbook-spool") assert [row.disposition for row in rows] == ["discarded"] + + +def test_inspection_export_is_candidate_only_and_dense_only(tmp_path): + args = arguments(tmp_path, "--inspection-export") + cli.validate_cli_args(args) + assert args.inspection_export + assert not arguments(tmp_path).inspection_export + with pytest.raises(ValueError, match="--inspection-export is candidate-only"): + cli.validate_cli_args( + arguments(tmp_path, "--release-candidate", "--inspection-export") + ) + national = cli.parse_args(_national_argv(tmp_path, "--inspection-export")) + with pytest.raises(ValueError, match="--inspection-export"): + cli.validate_cli_args(national) + + +def test_inspection_export_swaps_only_the_export_node(): + from dataclasses import dataclass, field + + from microcosm.build.uk_runtime import inspection_export + from microcosm.build.uk_runtime.graph_terminal import UKExportPrepareKernel + + @dataclass(frozen=True) + class Node: + id: str + kernel: str + params: dict = field(default_factory=dict) + + @dataclass(frozen=True) + class Graph: + nodes: tuple + + export = Node( + "uk.full.export.prepare", UKExportPrepareKernel.ref, {"bindings": "{}"} + ) + other = Node("uk.full.gates.calibrated", "gate@1") + swapped = inspection_export.use_inspection_export(Graph((other, export))) + nodes = {node.id: node for node in swapped.nodes} + assert nodes["uk.full.gates.calibrated"] is other + assert nodes["uk.full.export.prepare"].kernel == ( + inspection_export.UKInspectionExportPrepareKernel.ref + ) + assert nodes["uk.full.export.prepare"].params == { + "bindings": "{}", + "inspection_export": True, + } + assert export.params == {"bindings": "{}"} + with pytest.raises(ValueError, match="Expected one"): + inspection_export.use_inspection_export(Graph((other,))) + + +def test_inspection_kernel_describes_past_a_gate_block_only_when_requested(monkeypatch): + from microcosm.build.uk_runtime import inspection_export + + monkeypatch.setattr(inspection_export, "context_frame", lambda context: "frame") + monkeypatch.setattr( + inspection_export.graph_terminal, + "describe_uk_export", + lambda frame, *, bindings: {"frame": frame, "bindings": bindings}, + ) + kernel = inspection_export.UKInspectionExportPrepareKernel() + gate = SimpleNamespace(key="g" * 64) + + def context(params): + return SimpleNamespace(params=params, artifacts={"full_gates": gate}) + + result = kernel.run( + context({"bindings": '{"inspection_export":true}', "inspection_export": True}) + ) + descriptor = json.loads(result.artifacts["export_descriptor"]) + assert descriptor["bindings"]["inspection_export"] is True + assert descriptor["bindings"]["artifacts"] == {"full_gates": "g" * 64} + with pytest.raises(ValueError, match="requires inspection_export"): + kernel.run(context({"bindings": '{"inspection_export":true}'})) + with pytest.raises(ValueError, match="must record inspection_export"): + kernel.run(context({"bindings": "{}", "inspection_export": True})) + + +def test_sized_dense_builds_take_the_doctrine_size_defaults(tmp_path): + """microcosm#1124: draw ceiling, refit floor and refit L2 default only when sized.""" + from microcosm.build.uk_runtime import local_doctrine as doctrine + + unsized = arguments(tmp_path) + cli.validate_cli_args(unsized) + assert (unsized.selection_pi_hi, unsized.baseline_pi_floor) == (1.0, 0.0) + assert unsized.refit_l2_lambda is None + assert cli.cli_size_l2(unsized, "refit") is None + + sized = arguments(tmp_path, "--dataset-households", "100000") + cli.validate_cli_args(sized) + assert sized.selection_pi_hi == doctrine.UK_LOCAL_SIZE_SELECTION_PI_HI == 0.5 + assert sized.baseline_pi_floor == doctrine.UK_LOCAL_SIZE_BASELINE_PI_FLOOR == 0.5 + l2 = cli.cli_size_l2(sized, "refit") + assert l2.l2_lambda == doctrine.UK_LOCAL_SIZE_REFIT_L2_LAMBDA + assert (l2.anchor, l2.basis) == ("uniform", "chi_square") + assert cli.cli_size_l2(sized, "selection") is None + + explicit = arguments( + tmp_path, + "--dataset-households", + "100000", + "--selection-pi-hi", + "0.95", + "--baseline-pi-floor", + "0", + "--refit-l2-lambda", + "0", + ) + cli.validate_cli_args(explicit) + assert (explicit.selection_pi_hi, explicit.baseline_pi_floor) == (0.95, 0.0) + assert cli.cli_size_l2(explicit, "refit") is None + + +def test_the_doctrine_is_k15_2000_epochs_nation_grain(): + from microcosm.build.uk_runtime import local_doctrine as doctrine + + assert doctrine.UK_LOCAL_CLONE_COUNT == 15 + assert doctrine.UK_LOCAL_SOLVE_EPOCHS == 2000 + assert doctrine.UK_LOCAL_SOLVE_DOCTRINE.target_weight_rule == ( + "nation_grain_family_equal_sqrt_count" + ) + assert doctrine.UK_LOCAL_MAX_WEIGHT_RATIO == 10.0 diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_graph_terminal.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_graph_terminal.py index c576ffe05c..c57a5ac5cb 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_graph_terminal.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_graph_terminal.py @@ -473,7 +473,12 @@ def test_holdout_kernel_preserves_existing_rotations_and_seed_settings(monkeypat def solve(frame, training, **kwargs): calls.append( - (kwargs["seed"], kwargs["dataset_households"], kwargs["selection_seed"]) + ( + kwargs["seed"], + kwargs["dataset_households"], + kwargs["selection_seed"], + kwargs["selection_initial_lambda"], + ) ) return SimpleNamespace(weights=np.ones(2), selected_support=np.array([0, 2])) @@ -509,6 +514,8 @@ def solve(frame, training, **kwargs): "selection_seed": 17, "selection_pi_hi": 1.0, "baseline_pi_floor": 0.0, + # Every fold's budget search starts where the main search started. + "selection_initial_lambda": 1.7e-7, }, artifacts={ "preflight": SimpleNamespace(payload=preflight_payload(), key="a" * 64), @@ -525,11 +532,12 @@ def solve(frame, training, **kwargs): dataset_households=2, solve_seed=42, selection_seed=17, + selection_initial_lambda=1.7e-7, ) assert { key: value for key, value in result.items() if key != "graph_binding" } == expected - assert calls == [(42, 2, 17)] * 10 + assert calls == [(42, 2, 17, 1.7e-7)] * 10 def test_final_gate_kernel_owns_complete_diagnostics_and_reuses_decoded_result( diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_local_candidate_scorer.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_local_candidate_scorer.py index e26efa4005..8bfa718b57 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_local_candidate_scorer.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_local_candidate_scorer.py @@ -238,6 +238,54 @@ def test_local_scorer_refuses_a_non_frozen_surface() -> None: ) +def test_the_frozen_count_is_the_1123_compiled_local_reference_count() -> None: + # microcosm#1123's surface regeneration on the 942a1fa feed: + # register_completeness.compiled_local_reference_count 20,885 -> 21,804. + assert _load_scorer().UK_LOCAL_ACTIVE_REFERENCE_COUNT == 21_804 + + +def test_a_graph_target_selection_scores_its_local_rows(tmp_path) -> None: + """A graph build's target selection carries national and local rows; the + scorer reads it as written and keeps the local surface.""" + + import json + + from microcosm.build.uk_runtime.graph_targets import registry_payload + + scorer = _load_scorer() + registry, _candidate, _weights, _metrics = _case() + national = TargetSpec( + name="obr.income_tax", + entity="person", + value=1.0, + measure="income_tax", + period=2025, + source="fixture", + family="obr", + metadata={"geography_level": "country"}, + ) + selection = tmp_path / "target_selection.json" + selection.write_text( + json.dumps( + { + "receipt": {"schema": "microcosm.calibrate.target-selection.v1"}, + "registry": registry_payload( + TargetRegistry([*registry.specs, national], country="uk") + ), + "target_families": [], + } + ) + ) + scored = scorer.load_scored_registry(selection) + assert [spec.name for spec in scored.specs] == [ + spec.name for spec in registry.specs + ] + # A registry artifact still loads as one. + artifact = tmp_path / "registry.json" + registry.to_json(artifact) + assert len(scorer.load_scored_registry(artifact)) == len(registry) + + def test_local_scorer_accepts_candidate_superset_with_receipt() -> None: scorer = _load_scorer() registry, candidate, weights, metrics = _case() diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_local_doctrine.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_local_doctrine.py index 824b0a27bd..871c5a7a33 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_local_doctrine.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_local_doctrine.py @@ -83,18 +83,22 @@ def test_past_cap_census_defaults_scales_and_validates_shapes() -> None: def test_doctrine_constants_are_the_declared_contract() -> None: - # microcosm#762 rulings of 2026-09-03 (A2 rule, A3 bound, K, epochs); - # each is a measured choice recorded in the receipts (R6-R10). Moving - # one is a doctrine revision: edit the constant, its note, and this pin. + # microcosm#762 rulings of 2026-09-03 (A2 rule, A3 bound, K, epochs), + # revised for the rule, K and epochs by microcosm#1124; each is a measured + # choice recorded in the receipts. Moving one is a doctrine revision: edit + # the constant, its note, and this pin. assert UK_LOCAL_TARGET_LOSS_CAP == 10.0 assert UK_LOCAL_MAX_WEIGHT_RATIO == 10.0 - assert UK_LOCAL_TARGET_WEIGHT_RULE == "grain_equal" - assert UK_LOCAL_SOLVE_EPOCHS == 1500 + assert UK_LOCAL_TARGET_WEIGHT_RULE == "nation_grain_family_equal_sqrt_count" + assert UK_LOCAL_SOLVE_EPOCHS == 2000 assert UK_LOCAL_CLONE_COUNT == 15 assert UK_LOCAL_SOLVE_DOCTRINE.target_loss_cap == UK_LOCAL_TARGET_LOSS_CAP assert UK_LOCAL_SOLVE_DOCTRINE.max_weight_ratio == UK_LOCAL_MAX_WEIGHT_RATIO assert UK_LOCAL_SOLVE_DOCTRINE.scale_rule == "default_target_loss_scales" - assert UK_LOCAL_SOLVE_DOCTRINE.target_weight_rule == "grain_equal" + assert ( + UK_LOCAL_SOLVE_DOCTRINE.target_weight_rule + == "nation_grain_family_equal_sqrt_count" + ) assert UKLocalSolveDoctrine(target_weight_rule="uniform").target_weight_rule == ( "uniform" ) @@ -117,7 +121,10 @@ def test_local_doctrine_override_is_receipted_and_bounds_are_frozen() -> None: ) assert doctrine.target_weight_rule == "uniform" assert receipt == { - "target_weight_rule": {"default": "grain_equal", "effective": "uniform"} + "target_weight_rule": { + "default": "nation_grain_family_equal_sqrt_count", + "effective": "uniform", + } } with pytest.raises(ValueError, match="reviewed constants"): uk_local_doctrine_with_overrides( diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_local_release_preflight.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_local_release_preflight.py index 3a5247b2dc..0b561d2f09 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_local_release_preflight.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_local_release_preflight.py @@ -49,13 +49,13 @@ def _good_candidate(tmp_path: Path) -> Path: "release_role": "dense", "parameters": { "release_candidate": True, - "epochs": 1500, + "epochs": 2000, "n_clones": 15, "skip_holdout": False, "doctrine": { "max_weight_ratio": 10.0, - "target_weight_rule": "grain_equal", - "solve_epochs": 1500, + "target_weight_rule": "nation_grain_family_equal_sqrt_count", + "solve_epochs": 2000, "clone_count": 15, }, }, @@ -264,6 +264,12 @@ def test_preflight_refuses_a_national_role_manifest(tmp_path: Path, role) -> Non "max_weight_ratio", ), (lambda m, r: m["parameters"].__setitem__("skip_holdout", True), "holdout"), + ( + lambda m, r: m["parameters"].__setitem__( + "refit_l2", {"lambda": 0.03, "anchor": "initial", "basis": "chi_square"} + ), + "refit L2 penalty", + ), ( lambda m, r: m["census_household_uprating"]["tenure_cells"].__setitem__( "applied", False @@ -304,6 +310,126 @@ def test_each_missing_flag_is_named(tmp_path: Path, mutate, needle: str) -> None assert any(needle in failure for failure in failures), failures +def _per_clone(exact: bool = True, blocks: int = 15) -> dict: + return { + "blocks": blocks, + "engine_population_representation": { + "mode": "block_weights_scaled_to_pool", + "blocks": blocks, + "exact": exact, + }, + } + + +@pytest.mark.parametrize( + ("resolution", "passes"), + [ + (_per_clone(), True), + (_per_clone(exact=False), False), + # One block per clone, so the count must be the run's K. + (_per_clone(blocks=5), False), + ({"blocks": 15}, False), + ], +) +def test_one_engine_block_per_clone_passes_only_when_recorded_exact( + tmp_path: Path, resolution: dict, passes: bool +) -> None: + module = _load() + candidate = _good_candidate(tmp_path) + manifest_path = candidate / "rowwise_candidate_manifest.json" + manifest = json.loads(manifest_path.read_text()) + manifest["solve"]["measure_resolution"] = resolution + manifest_path.write_text(json.dumps(manifest)) + failures = module.check_candidate_dir(candidate, today=date(2026, 9, 4)) + assert (failures == []) is passes, failures + if not passes: + assert any("without an exact pool representation" in f for f in failures) + + +def _sized(manifest: dict) -> None: + """Make the fixture a sized run at the size doctrine.""" + + manifest["parameters"].update( + { + "dataset_households": 100_000, + "selection_pi_hi": 0.5, + "baseline_pi_floor": 0.5, + "refit_l2": { + "lambda": 0.0215748772773509, + "anchor": "uniform", + "basis": "chi_square", + }, + } + ) + manifest["solve"]["dataset_size"] = { + "requested_households": 100_000, + "realized_households": 100_000, + } + manifest.setdefault("release_posture", {})["size_doctrine"] = True + + +@pytest.mark.parametrize( + ("mutate", "needle"), + [ + (lambda m: None, None), + ( + lambda m: m["parameters"].__setitem__("dataset_households", 60_000), + "parameters.dataset_households", + ), + ( + lambda m: m["parameters"].__setitem__("selection_pi_hi", 0.95), + "parameters.selection_pi_hi", + ), + (lambda m: m["parameters"].__setitem__("refit_l2", None), "refit_l2"), + ( + lambda m: m["solve"]["dataset_size"].__setitem__( + "realized_households", 99_998 + ), + "realized", + ), + ( + lambda m: m["release_posture"].__setitem__("size_doctrine", False), + "size_doctrine", + ), + (lambda m: m["solve"].pop("dataset_size"), "dataset_size is missing"), + ( + lambda m: m["parameters"].__setitem__("resume_size_checkpoint", "ck"), + "resumed from a size checkpoint", + ), + ( + lambda m: m["parameters"].__setitem__("selection_l2", {"lambda": 0.5}), + "selection L2 penalty", + ), + ], +) +def test_a_sized_run_passes_only_at_the_size_doctrine( + tmp_path: Path, mutate, needle +) -> None: + module = _load() + candidate = _good_candidate(tmp_path) + manifest_path = candidate / "rowwise_candidate_manifest.json" + manifest = json.loads(manifest_path.read_text()) + _sized(manifest) + mutate(manifest) + manifest_path.write_text(json.dumps(manifest)) + failures = module.check_candidate_dir(candidate, today=date(2026, 9, 4)) + if needle is None: + assert failures == [] + else: + assert any(needle in failure for failure in failures), failures + + +def test_an_unsized_run_carries_no_refit_penalty(tmp_path: Path) -> None: + module = _load() + candidate = _good_candidate(tmp_path) + manifest_path = candidate / "rowwise_candidate_manifest.json" + manifest = json.loads(manifest_path.read_text()) + manifest["parameters"]["refit_l2"] = {"lambda": 0.01} + manifest_path.write_text(json.dumps(manifest)) + failures = module.check_candidate_dir(candidate, today=date(2026, 9, 4)) + assert any("without a size request" in failure for failure in failures) + + @pytest.mark.parametrize( "mutate, needle", [ diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_measure_simulation.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_measure_simulation.py index d9a8610b54..12e92b0b43 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_measure_simulation.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_measure_simulation.py @@ -826,11 +826,15 @@ def test_packaged_exclusions_load(): # retired on 2026-09-16 with their repairs; the any-tenure housing row (a # structural bias: DWP's 'Yes' includes an other/unknown tenure the model # cannot carry) remains, adjudicated to microcosm#882 and the committed - # baseline doc, signed in review of microcosm#921, windowed to one month. - elements = [e for e in exclusions if e["approved_on"] == "2026-09-15"] + # baseline doc, signed in review of microcosm#921, windowed to one month, + # and renewed one month on 2026-10-14 (microcosm#1124 local release), so + # nothing remains on the 2026-09-15 clock. + assert [e for e in exclusions if e["approved_on"] == "2026-09-15"] == [] + elements = [e for e in exclusions if e["approved_on"] == "2026-10-14"] assert sorted(e["name"] for e in elements) == sorted(_UC_ELEMENT_REGISTER_ROWS) for entry in elements: - assert entry["expires_on"] == "2026-10-15", entry["name"] + assert entry["expires_on"] == "2026-11-14", entry["name"] + assert "renewed one month on 2026-10-14" in entry["adjudication"] assert entry["tracking"] == "microcosm#882", entry["name"] assert entry["approved_by"] == "juaristi22", entry["name"] assert "microcosm#882" in entry["adjudication"], entry["name"] diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_rowwise_candidate.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_rowwise_candidate.py index ed35194466..c9ecc1f4d8 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_rowwise_candidate.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_rowwise_candidate.py @@ -97,7 +97,7 @@ def test_graph_driver_dry_run_prints_the_operation_inventory( assert plan["default_scope"] == "all_geographies" assert plan["configuration"]["n_clones"] == 2 assert plan["configuration"]["seed"] == 7 - assert plan["configuration"]["calibration"]["epochs"] == 1500 + assert plan["configuration"]["calibration"]["epochs"] == 2000 assert plan["configuration"]["target_families"] is None assert {node["id"] for node in plan["nodes"]} >= { "uk.full.dense", @@ -280,10 +280,10 @@ def test_release_candidate_refuses_non_doctrine_solve_settings(tmp_path) -> None args = builder._parse_args(base) builder._validate_cli_args(args) assert args.n_clones == builder.UK_ROWWISE_DENSE_POSTURE.clone_count == 15 - assert args.epochs == builder.UK_ROWWISE_DENSE_POSTURE.epochs == 1500 - assert args.target_weight_rule == "grain_equal" + assert args.epochs == builder.UK_ROWWISE_DENSE_POSTURE.epochs == 2000 + assert args.target_weight_rule == "nation_grain_family_equal_sqrt_count" - with pytest.raises(ValueError, match=r"--epochs != doctrine 1500"): + with pytest.raises(ValueError, match=r"--epochs != doctrine 2000"): builder._validate_cli_args(builder._parse_args([*base, "--epochs", "512"])) with pytest.raises(ValueError, match=r"--n-clones != doctrine 15"): builder._validate_cli_args(builder._parse_args([*base, "--n-clones", "10"])) @@ -376,7 +376,27 @@ def test_dense_candidate_manifest_has_no_size_sidecars(tmp_path): assert builder._SIZE_RUN_ONLY_OUTPUTS == {"dense_reference", "selection"} -def test_size_cli_refuses_promotion_without_separate_certification(tmp_path): +@pytest.mark.parametrize( + ("extra", "needle"), + [ + (["--dataset-households", "50000"], "dataset_households 50000"), + ( + ["--dataset-households", "100000", "--selection-pi-hi", "0.9"], + "selection_pi_hi 0.9", + ), + ( + ["--dataset-households", "100000", "--refit-l2-lambda", "0"], + "refit_l2_lambda", + ), + ( + ["--dataset-households", "100000", "--selection-seed", "7"], + "selection_seed 7", + ), + ], +) +def test_a_sized_release_candidate_must_carry_the_size_doctrine( + tmp_path, extra, needle +): builder = _load_builder_module() args = builder._parse_args( [ @@ -388,13 +408,66 @@ def test_size_cli_refuses_promotion_without_separate_certification(tmp_path): str(tmp_path / "ladder.npz"), "--out", str(tmp_path / "out"), - "--dataset-households", - "50000", + *extra, "--release-candidate", ] ) - with pytest.raises(ValueError, match="candidate-only"): + with pytest.raises(ValueError, match="local size doctrine") as refused: builder._validate_cli_args(args) + assert needle in str(refused.value) + + +def test_the_doctrine_size_is_a_release_size_and_any_other_a_candidate(tmp_path): + from microcosm.build.uk_runtime.local_doctrine import UK_LOCAL_DATASET_HOUSEHOLDS + from microcosm.build.uk_runtime.rowwise_cli import size_doctrine_drift + + builder = _load_builder_module() + + def parsed(*extra): + return builder._parse_args( + [ + "--input-h5", + str(tmp_path / "spine.h5"), + "--release-role", + "dense", + "--ladder", + str(tmp_path / "ladder.npz"), + "--out", + str(tmp_path / "out"), + *extra, + ] + ) + + assert UK_LOCAL_DATASET_HOUSEHOLDS == 100_000 + # The role fills the size doctrine, so the bare request carries it. + assert size_doctrine_drift(parsed("--dataset-households", "100000")) == [] + assert size_doctrine_drift(parsed()) == [] + assert size_doctrine_drift(parsed("--dataset-households", "60000")) + # A selection penalty or a resumed selection is not the doctrine's draw. + with_selection_l2 = size_doctrine_drift( + parsed( + "--dataset-households", + "100000", + "--selection-l2-lambda", + "0.5", + "--selection-l2-anchor", + "uniform", + "--selection-l2-basis", + "chi_square", + ) + ) + assert any("selection L2 penalty" in item for item in with_selection_l2) + resumed = parsed("--dataset-households", "100000") + resumed.resume_size_checkpoint = tmp_path / "checkpoint" + assert any( + "resume_size_checkpoint" in item for item in size_doctrine_drift(resumed) + ) + # Off the doctrine a sized build is still a candidate request: whatever + # else the bare request lacks, the size doctrine is not what refuses it. + try: + builder._validate_cli_args(parsed("--dataset-households", "60000")) + except ValueError as error: + assert "size doctrine" not in str(error) def test_staging_epoch_stride_keeps_the_forwarded_rows_bounded(): @@ -513,7 +586,7 @@ def test_release_role_supplies_the_solve_defaults(tmp_path) -> None: posture.epochs, posture.learning_rate, ) - assert dense.target_weight_rule == "grain_equal" + assert dense.target_weight_rule == "nation_grain_family_equal_sqrt_count" assert dense.expected_constituency_vintage == "2024_pcon" assert dense.staging_upload_interval_seconds == 300.0 assert dense._explicit_arguments == frozenset() diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_rowwise_posture.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_rowwise_posture.py index 8d01e3c608..827583d9c7 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_rowwise_posture.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_rowwise_posture.py @@ -40,11 +40,18 @@ def test_uk_dense_posture_mirrors_the_ruled_constants() -> None: assert posture.role == "dense" assert posture.doctrine is UK_LOCAL_SOLVE_DOCTRINE assert posture.clone_count == UK_LOCAL_CLONE_COUNT == 15 - assert posture.epochs == UK_LOCAL_SOLVE_EPOCHS == 1500 + assert posture.epochs == UK_LOCAL_SOLVE_EPOCHS == 2000 assert posture.learning_rate == 0.15 assert posture.seed == 42 - assert posture.target_weight_rule == "grain_equal" - assert posture.allowed_target_weight_rules == ("grain_equal", "uniform") + assert posture.target_weight_rule == "nation_grain_family_equal_sqrt_count" + assert posture.allowed_target_weight_rules == ( + "grain_equal", + "uniform", + "grain_family_equal", + "grain_family_equal_sqrt_count", + "nation_grain_family_equal", + "nation_grain_family_equal_sqrt_count", + ) assert posture.expected_constituency_vintage == "2024_pcon" assert posture.gate_scope == tuple(UK_LOCAL_GATE_SCOPE) assert (posture.gate_posture, posture.gate_policy_suffix) == ( @@ -71,8 +78,8 @@ def test_uk_dense_posture_mirrors_the_ruled_constants() -> None: "target_loss_cap": UK_LOCAL_TARGET_LOSS_CAP, "max_weight_ratio": UK_LOCAL_MAX_WEIGHT_RATIO, "scale_rule": "default_target_loss_scales", - "target_weight_rule": "grain_equal", - "solve_epochs": 1500, + "target_weight_rule": "nation_grain_family_equal_sqrt_count", + "solve_epochs": 2000, "clone_count": 15, } diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_size_l2.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_size_l2.py new file mode 100644 index 0000000000..410bda80ff --- /dev/null +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_size_l2.py @@ -0,0 +1,403 @@ +"""Per-stage L2 penalties and target weightings in the UK size chain (microcosm#1124).""" + +from __future__ import annotations + +import json + +import numpy as np +import pytest + +from microcosm.build.uk_runtime import dataset_size +from microcosm.build.uk_runtime.dataset_size import ( + UK_SIZE_L2_PARAM_KEYS, + UKSizeDraw, + UKSizeL2, + draw_uk_dataset_size, + refit_uk_dataset_size, + select_uk_dataset_size, + uk_size_l2, + uk_size_l2_from_options, +) +from microcosm.build.uk_runtime.target_weights import ( + UKStageTargetWeighting, + uk_target_rows, +) +from microcosm.calibrate import Target, TargetSet, calibrate +from test_support.microcosm_build.uk_local_rowwise import _clone_frame + +_COMMON = dict(households=2, epochs=2, learning_rate=0.02, seed=7, pi_hi=0.5) +_NEW_RECEIPT_KEYS = { + "selection_l2", + "refit_l2", + "refit_iterate_selection", + "selection_target_weighting", + "refit_target_weighting", + "compact_loss_under_dense_weights", +} + + +def _dense(frame): + return calibrate( + frame, + TargetSet( + [ + Target("count", "household", lambda f: np.ones(f.n("household")), 3), + Target( + "first", + "household", + lambda f: ( + f.table("household").household_id.to_numpy() == 101 + ).astype(float), + 1, + ), + ] + ), + epochs=2, + ) + + +def _weighting(dense, rule="grain_family_equal", held_out=()): + names = dense.problem.names + rows = uk_target_rows( + names, + [{"family": "count_family"}, {"family": "first_family"}], + np.asarray(dense.problem.target_vector, dtype=np.float64), + local=[False, False], + ) + return UKStageTargetWeighting(rule, rows, held_out=held_out) + + +def _capturing(monkeypatch): + calls = {"calibrate": [], "refit_l0_selection": []} + real_calibrate = dataset_size.calibrate + real_refit = dataset_size.refit_l0_selection + + def calibrate_spy(*args, **kwargs): + calls["calibrate"].append(kwargs) + return real_calibrate(*args, **kwargs) + + def refit_spy(*args, **kwargs): + calls["refit_l0_selection"].append(kwargs) + return real_refit(*args, **kwargs) + + monkeypatch.setattr(dataset_size, "calibrate", calibrate_spy) + monkeypatch.setattr(dataset_size, "refit_l0_selection", refit_spy) + return calls + + +def test_uk_size_l2_validates_and_is_off_by_default() -> None: + assert uk_size_l2("selection") is None + assert uk_size_l2("refit", l2_lambda=0) is None + on = uk_size_l2("refit", l2_lambda=0.03, anchor="uniform") + assert on == UKSizeL2("refit", 0.03, "uniform", "chi_square") + assert on.solver_kwargs() == { + "l2_lambda": 0.03, + "l2_anchor": "uniform", + "l2_basis": "chi_square", + } + assert on.params() == { + "refit_l2_lambda": 0.03, + "refit_l2_anchor": "uniform", + "refit_l2_basis": "chi_square", + } + assert set(on.params()) <= set(UK_SIZE_L2_PARAM_KEYS) + with pytest.raises(ValueError, match="without a positive"): + uk_size_l2("refit", anchor="uniform") + with pytest.raises(ValueError, match="'record' is refused"): + uk_size_l2("refit", l2_lambda=0.1, basis="record") + with pytest.raises(ValueError, match="'design' is not a size-stage anchor"): + uk_size_l2("refit", l2_lambda=0.1, anchor="design") + for bad in (-0.1, float("nan"), float("inf"), True, "0.1"): + with pytest.raises(ValueError, match="finite number"): + uk_size_l2("selection", l2_lambda=bad) + with pytest.raises(ValueError, match="stage"): + uk_size_l2("dense", l2_lambda=0.1) + + +def test_default_calls_pass_no_l2_keyword_and_keep_the_receipt(monkeypatch) -> None: + frame = _clone_frame() + dense = _dense(frame) + calls = _capturing(monkeypatch) + sized = refit_uk_dataset_size(frame, dense, **_COMMON) + for stage in ("calibrate", "refit_l0_selection"): + (kwargs,) = calls[stage] + assert not {"l2_lambda", "l2_anchor", "l2_basis"} & set(kwargs) + assert kwargs["target_loss_weights"] is dense.target_loss_weights + assert not _NEW_RECEIPT_KEYS & set(sized.receipt) + assert sized.result.options["l2_lambda"] == 0.0 + + +def test_refit_l2_reaches_the_refit_only_and_is_receipted(monkeypatch) -> None: + frame = _clone_frame() + dense = _dense(frame) + calls = _capturing(monkeypatch) + sized = refit_uk_dataset_size( + frame, dense, refit_l2_lambda=0.05, refit_l2_anchor="uniform", **_COMMON + ) + (search_kwargs,) = calls["calibrate"] + assert "l2_lambda" not in search_kwargs + (refit_kwargs,) = calls["refit_l0_selection"] + assert refit_kwargs["l2_lambda"] == 0.05 + assert refit_kwargs["l2_anchor"] == "uniform" + assert refit_kwargs["l2_basis"] == "chi_square" + options = sized.result.options + assert options["l2_lambda"] == 0.05 and options["l2_basis"] == "chi_square" + assert sized.receipt["refit_iterate_selection"] == options["iterate_selection"] + # The refit's weights ship: it keeps its best epoch on the penalized + # objective, not the closing state (microcosm#1124). + assert refit_kwargs["retain_best_objective"] is True + assert options["iterate_selection"] == "best_feasible_objective" + block = sized.receipt["refit_l2"] + assert block["lambda"] == 0.05 and block["anchor"] == "uniform" + assert block["anchor_reference"] == "normalized_horvitz_thompson_w_over_q_mean" + assert block["chi_square_distance_to_anchor"] >= 0.0 + assert "selection_l2" not in sized.receipt + floored = refit_uk_dataset_size( + frame, dense, refit_l2_lambda=0.05, baseline_pi_floor=0.5, **_COMMON + ) + assert floored.receipt["refit_l2"]["anchor_reference"] == ( + "normalized_horvitz_thompson_w_over_q_floored" + ) + + +def test_selection_l2_reaches_the_search_and_binds_reuse() -> None: + frame = _clone_frame() + dense = _dense(frame) + selection = select_uk_dataset_size( + frame, dense, selection_l2_lambda=0.01, **_COMMON + ) + assert uk_size_l2_from_options(selection.selection.options) == { + "lambda": 0.01, + "anchor": "initial", + "basis": "chi_square", + } + sized = refit_uk_dataset_size( + frame, dense, selection=selection, selection_l2_lambda=0.01, **_COMMON + ) + assert sized.receipt["selection_l2"]["anchor_reference"] == "pool_design" + with pytest.raises(ValueError, match="selection L2: searched under"): + refit_uk_dataset_size(frame, dense, selection=selection, **_COMMON) + plain = select_uk_dataset_size(frame, dense, **_COMMON) + with pytest.raises(ValueError, match="selection L2: searched under None"): + refit_uk_dataset_size( + frame, dense, selection=plain, selection_l2_lambda=0.01, **_COMMON + ) + + +def test_stage_weightings_are_named_rules_on_the_problem_axis() -> None: + frame = _clone_frame() + dense = _dense(frame) + weighting = _weighting(dense) + sized = refit_uk_dataset_size( + frame, dense, refit_target_weighting=weighting, **_COMMON + ) + expected = weighting.loss_weights(dense.problem.names) + np.testing.assert_array_equal(sized.result.target_loss_weights, expected) + assert sized.receipt["refit_target_weighting"]["rule"] == "grain_family_equal" + assert sized.receipt["compact_loss_under_dense_weights"] >= 0.0 + assert "selection_target_weighting" not in sized.receipt + with pytest.raises(TypeError, match="never a raw vector"): + refit_uk_dataset_size( + frame, dense, refit_target_weighting=np.array([0.5, 0.5]), **_COMMON + ) + reversed_rows = uk_target_rows( + dense.problem.names[::-1], + [{"family": "a"}, {"family": "b"}], + [1.0, 3.0], + local=[False, False], + ) + with pytest.raises(ValueError, match="another row axis"): + refit_uk_dataset_size( + frame, + dense, + refit_target_weighting=UKStageTargetWeighting("uniform", reversed_rows), + **_COMMON, + ) + # A search under a weighting must be reused with that same weighting. + searched = select_uk_dataset_size( + frame, dense, target_weighting=weighting, **_COMMON + ) + with pytest.raises(ValueError, match="target loss weights"): + refit_uk_dataset_size(frame, dense, selection=searched, **_COMMON) + reused = refit_uk_dataset_size( + frame, + dense, + selection=searched, + selection_target_weighting=weighting, + **_COMMON, + ) + assert reused.receipt["selection_target_weighting"]["rule"] == "grain_family_equal" + + +def test_held_out_rows_train_at_zero_weight() -> None: + frame = _clone_frame() + dense = _dense(frame) + weighting = _weighting(dense, held_out=(1,)) + sized = refit_uk_dataset_size( + frame, dense, refit_target_weighting=weighting, **_COMMON + ) + assert sized.result.target_loss_weights[1] == 0.0 + assert sized.result.target_loss_weights[0] > 0.0 + assert sized.receipt["refit_target_weighting"]["n_held_out"] == 1 + + +def test_a_full_pool_size_refuses_size_stage_settings() -> None: + frame = _clone_frame() + dense = _dense(frame) + full = dict(households=3, epochs=1, learning_rate=0.02, seed=7) + assert refit_uk_dataset_size(frame, dense, **full).result is dense + for extra in ( + {"refit_l2_lambda": 0.1}, + {"selection_l2_lambda": 0.1}, + {"refit_target_weighting": _weighting(dense)}, + ): + with pytest.raises(ValueError, match="full-pool size"): + refit_uk_dataset_size(frame, dense, **full, **extra) + + +def test_stored_draw_payload_round_trips() -> None: + frame = _clone_frame() + dense = _dense(frame) + selection = select_uk_dataset_size(frame, dense, **_COMMON) + draw = draw_uk_dataset_size( + frame, dense, selection=selection, households=2, seed=7, pi_hi=0.5 + ) + payload = json.dumps( + { + "problem_sha256": "abc", + "method": "exact_count", + "support": draw.support.tolist(), + "sampling": draw.sampling, + "inclusion_probabilities": draw.inclusion_probabilities.tolist(), + "feasibility": draw.feasibility, + "seed": draw.seed, + "pi_hi": draw.pi_hi, + "probabilities_sha256": draw.probabilities_sha256, + } + ) + restored = UKSizeDraw.from_payload(payload, problem_sha256="abc") + np.testing.assert_array_equal(restored.support, draw.support) + np.testing.assert_array_equal( + restored.inclusion_probabilities, draw.inclusion_probabilities + ) + assert restored.probabilities_sha256 == draw.probabilities_sha256 + with pytest.raises(ValueError, match="another ordered problem"): + UKSizeDraw.from_payload(payload, problem_sha256="other") + full = json.dumps({"problem_sha256": "abc", "method": "full_pool", "support": []}) + assert UKSizeDraw.from_payload(full, problem_sha256="abc") is None + with pytest.raises(ValueError, match="completed exact-count draw"): + UKSizeDraw.from_payload( + json.dumps({"problem_sha256": "abc", "method": "other"}), + problem_sha256="abc", + ) + + +def test_checkpoint_binds_the_selection_l2_and_refuses_weighted_searches( + tmp_path, +) -> None: + from types import SimpleNamespace + + from microcosm.build.uk_runtime.size_checkpoint import ( + load_uk_size_checkpoint, + uk_size_checkpoint_identity, + write_uk_size_checkpoint, + ) + + frame = _clone_frame() + dense = _dense(frame) + plain = select_uk_dataset_size(frame, dense, **_COMMON) + penalized = select_uk_dataset_size( + frame, dense, selection_l2_lambda=0.01, **_COMMON + ) + l2_identity = { + "run": "x", + "selection_l2": uk_size_l2("selection", l2_lambda=0.01).as_dict(), + } + write_uk_size_checkpoint( + tmp_path / "plain", + frame=frame, + dense=dense, + selection=plain, + identity={"run": "x"}, + ) + write_uk_size_checkpoint( + tmp_path / "l2", + frame=frame, + dense=dense, + selection=penalized, + identity=l2_identity, + ) + targets = TargetSet(list(dense.problem.targets)) + restored = load_uk_size_checkpoint( + tmp_path / "l2", frame=frame, target_set=targets, identity=l2_identity + ) + assert ( + uk_size_l2_from_options(restored.selection.selection.options)["lambda"] == 0.01 + ) + # An L2 run cannot import an unpenalized search, nor the reverse. + with pytest.raises(ValueError, match="selection_l2: absent in checkpoint"): + load_uk_size_checkpoint( + tmp_path / "plain", frame=frame, target_set=targets, identity=l2_identity + ) + with pytest.raises(ValueError, match="searched under L2"): + load_uk_size_checkpoint( + tmp_path / "l2", frame=frame, target_set=targets, identity={"run": "x"} + ) + # The writer refuses an identity that contradicts the search, and a search + # run under other loss weights than the dense solve's. + with pytest.raises(ValueError, match="disagrees with the L2 penalty"): + write_uk_size_checkpoint( + tmp_path / "bad", + frame=frame, + dense=dense, + selection=penalized, + identity={"run": "x"}, + ) + weighted = select_uk_dataset_size( + frame, dense, target_weighting=_weighting(dense), **_COMMON + ) + with pytest.raises(ValueError, match="one loss-weight vector"): + write_uk_size_checkpoint( + tmp_path / "weighted", + frame=frame, + dense=dense, + selection=weighted, + identity={"run": "x"}, + ) + # The identity gains the key only when the selection penalty is on. + assert "selection_l2" not in _identity( + uk_size_checkpoint_identity, SimpleNamespace() + ) + assert _identity( + uk_size_checkpoint_identity, SimpleNamespace(selection_l2_lambda=0.02) + )["selection_l2"] == {"lambda": 0.02, "anchor": "initial", "basis": "chi_square"} + + +def _identity(builder, extra): + from argparse import Namespace + + from microcosm.build.uk_runtime.rowwise_posture import UK_ROWWISE_DENSE_POSTURE + + args = Namespace( + _posture=UK_ROWWISE_DENSE_POSTURE, + ledger_facts_sha256="f", + ledger_manifest_sha256="m", + seed=7, + selection_seed=None, + n_clones=2, + dataset_households=2, + epochs=2, + learning_rate=0.02, + sample_fraction=1.0, + sample_seed=1, + source_lineage_modulus=None, + target_weight_rule="grain_equal", + engine_blocks=1, + measure_exclusions=None, + **vars(extra), + ) + return builder( + args, + pins={"dataset": {"sha256": "d"}, "ladder": {"sha256": "l"}}, + source_year=2024, + ) diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_size_l2_graph.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_size_l2_graph.py new file mode 100644 index 0000000000..a7d473dd95 --- /dev/null +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_size_l2_graph.py @@ -0,0 +1,71 @@ +"""Size-stage L2 penalties as graph configuration and node parameters (microcosm#1124).""" + +from __future__ import annotations + +import pytest + +from microcosm.build.uk_runtime.dataset_size import uk_size_l2 +from microcosm.build.uk_runtime.graph_calibration import ( + UKGraphCalibrationConfig, + uk_calibration_nodes, +) + +_COLUMNS = {("household", "marker"): "int64"} +_BASE_PARAMS = { + "epochs", + "learning_rate", + "seed", + "households", + "pi_hi", + "initial_lambda", +} + + +def _params(config): + nodes = uk_calibration_nodes( + base="pool", columns=_COLUMNS, problem_producer="pool", config=config + ).nodes + return {node.id.rsplit(".", 1)[-1]: dict(node.params) for node in nodes} + + +def test_default_size_nodes_keep_their_parameters() -> None: + params = _params(UKGraphCalibrationConfig(epochs=2, dataset_households=2)) + assert set(params["size_search"]) == _BASE_PARAMS + assert set(params["size_draw"]) == _BASE_PARAMS + assert set(params["size_refit"]) == _BASE_PARAMS | {"baseline_pi_floor"} + + +def test_selection_l2_joins_the_shared_size_params_and_refit_l2_the_refit_only() -> ( + None +): + params = _params( + UKGraphCalibrationConfig( + epochs=2, + dataset_households=2, + selection_l2=uk_size_l2("selection", l2_lambda=1e-3), + refit_l2=uk_size_l2("refit", l2_lambda=0.03, anchor="uniform"), + ) + ) + selection_keys = { + "selection_l2_lambda", + "selection_l2_anchor", + "selection_l2_basis", + } + refit_keys = {"refit_l2_lambda", "refit_l2_anchor", "refit_l2_basis"} + for node in ("size_search", "size_draw", "size_refit"): + assert selection_keys <= set(params[node]) + assert not refit_keys & set(params["size_search"]) + assert not refit_keys & set(params["size_draw"]) + assert params["size_refit"]["refit_l2_lambda"] == 0.03 + assert params["size_refit"]["refit_l2_anchor"] == "uniform" + + +def test_config_refuses_l2_without_a_size_or_on_the_wrong_stage() -> None: + with pytest.raises(ValueError, match="requires a dataset size"): + UKGraphCalibrationConfig(refit_l2=uk_size_l2("refit", l2_lambda=0.1)) + with pytest.raises(ValueError, match="selection-stage UKSizeL2"): + UKGraphCalibrationConfig( + dataset_households=2, selection_l2=uk_size_l2("refit", l2_lambda=0.1) + ) + with pytest.raises(ValueError, match="Unknown target_weight_rule"): + UKGraphCalibrationConfig(target_weight_rule="family_weird") diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_target_weight_wiring.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_target_weight_wiring.py new file mode 100644 index 0000000000..fe86c61ce0 --- /dev/null +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_target_weight_wiring.py @@ -0,0 +1,133 @@ +"""Row-metadata weighting rules reach the solve, the holdout and the vocabulary mirrors.""" + +from __future__ import annotations + +import typing + +import numpy as np +import pandas as pd +import pytest + +from microcosm.build.uk_runtime import ( + build_uk_rowwise_local_matrix, + rotated_uk_local_holdout, + uk_national_frame, +) +from microcosm.build.uk_runtime.local_doctrine import ( + UK_LOCAL_ROW_METADATA_RULES, + UK_LOCAL_TARGET_WEIGHT_RULES, + uk_local_target_loss_weights, +) +from microcosm.build.uk_runtime.local_rowwise import prepare_uk_full_solve +from microcosm.build.uk_runtime.rowwise_posture import UK_ROWWISE_DENSE_POSTURE +from microcosm.build.uk_runtime.target_weights import ( + uk_target_loss_weights_for_rows, +) +from microcosm.diagnostics.schema import UKMeasuredRotatedHoldout +from microcosm.frame import WeightKind + +_IDS = [101, 102, 103, 104, 105] +_AREAS = ["E001", "W001", "S001", "N001", "E002"] + + +def _frame(): + return uk_national_frame( + person=pd.DataFrame( + { + "person_id": [1, 2, 3, 4, 5], + "person_household_id": _IDS, + "person_benunit_id": [11, 12, 13, 14, 15], + } + ), + benunit=pd.DataFrame({"benunit_id": [11, 12, 13, 14, 15]}), + household=pd.DataFrame({"household_id": _IDS, "household_weight": [1.0] * 5}), + time_period="2023", + weight_kind=WeightKind.IMPORTANCE, + ) + + +def _problem(): + metrics = pd.DataFrame( + {"households": np.ones(5), "tenure/social_rent": np.ones(5)}, + index=_IDS, + ) + targets = pd.DataFrame( + {"code": _AREAS, "households": [2.0] * 5, "tenure/social_rent": [1.0] * 5} + ) + return build_uk_rowwise_local_matrix( + metrics, pd.Series(_AREAS, index=_IDS), targets + ) + + +_FAMILIES = ["census_households/constituency", "tenure/constituency"] + + +def test_prepared_solve_binds_the_rule_computed_from_its_own_rows() -> None: + problem = _problem() + legacy = prepare_uk_full_solve( + _frame(), problem, bound_families=_FAMILIES, target_weight_rule="grain_equal" + ) + assert legacy.target_rows is None + np.testing.assert_array_equal( + legacy.target_loss_weights, + uk_local_target_loss_weights( + problem.target_frame["area_type"].astype(str).tolist(), rule="grain_equal" + ), + ) + for rule in UK_LOCAL_ROW_METADATA_RULES: + prepared = prepare_uk_full_solve( + _frame(), problem, bound_families=_FAMILIES, target_weight_rule=rule + ) + rows = prepared.target_rows + assert rows is not None + assert rows.names == tuple( + target.row_name for target in prepared.target_set.targets + ) + assert set(rows.family) == {"census_households", "tenure"} + np.testing.assert_array_equal( + prepared.target_loss_weights, + uk_target_loss_weights_for_rows(rows, rule=rule), + ) + # One grain, two families: grain_family_equal halves the loss by family, + # where grain_equal is uniform over the ten rows. + family = prepare_uk_full_solve( + _frame(), + problem, + bound_families=_FAMILIES, + target_weight_rule="grain_family_equal", + ) + metrics = problem.target_frame["metric"].to_numpy() + assert family.target_loss_weights[metrics == "households"].sum() == pytest.approx( + 0.5 + ) + + +def test_holdout_weights_held_rows_through_the_carrier_and_is_schema_valid() -> None: + payload = rotated_uk_local_holdout( + _frame(), + _problem(), + target_weight_rule="grain_family_equal", + epochs=2, + solve_seed=17, + ) + assert payload["target_weight_rule"] == "grain_family_equal" + assert payload["n_folds"] == 5 + assert np.isfinite(payload["mean_holdout_loss"]) + UKMeasuredRotatedHoldout.model_validate(payload) + + +def test_diagnostics_schema_admits_exactly_the_local_vocabulary() -> None: + annotation = UKMeasuredRotatedHoldout.model_fields["target_weight_rule"].annotation + assert typing.get_args(annotation) == UK_LOCAL_TARGET_WEIGHT_RULES + + +def test_dense_posture_allows_the_row_rules_and_keeps_its_default() -> None: + assert ( + UK_ROWWISE_DENSE_POSTURE.target_weight_rule + == "nation_grain_family_equal_sqrt_count" + ) + assert UK_ROWWISE_DENSE_POSTURE.allowed_target_weight_rules == ( + "grain_equal", + "uniform", + *UK_LOCAL_ROW_METADATA_RULES, + ) diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_target_weights.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_target_weights.py new file mode 100644 index 0000000000..9cb90ff5a1 --- /dev/null +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_target_weights.py @@ -0,0 +1,393 @@ +"""Row-metadata target-weight rules for the UK local solve (microcosm#1124).""" + +from __future__ import annotations + +from types import SimpleNamespace + +import numpy as np +import pytest + +from microcosm.build.uk_runtime.local_doctrine import ( + UK_LOCAL_ROW_METADATA_RULES, + UK_LOCAL_SOLVE_DOCTRINE, + UK_LOCAL_TARGET_WEIGHT_RULES, + UKLocalSolveDoctrine, + uk_local_doctrine_with_overrides, + uk_local_target_loss_weights, +) +from microcosm.build.uk_runtime.target_weights import ( + NATION_REGION_GRAIN, + UKStageTargetWeighting, + uk_rule_loss_weights, + uk_target_loss_weight_receipt, + uk_target_loss_weights_for_rows, + uk_target_loss_weights_sha256, + uk_target_rows, + uk_target_rows_from_problem, + uk_target_rows_from_targets, + uk_target_value_basis, +) + + +def _local(name, area_type, code, metric, family, value): + return ( + name, + {"area_type": area_type, "area_code": code, "metric": metric, "family": family}, + value, + True, + ) + + +def _national(name, family, geography, unit, value): + return ( + name, + { + "family": family, + "ledger_geography_id": geography, + "ledger_measure_unit": unit, + }, + value, + False, + ) + + +def _rows(*rows): + names, metadata, values, local = zip(*rows, strict=True) + return uk_target_rows(names, metadata, values, local=local) + + +@pytest.fixture +def surface(): + return _rows( + _local( + "c1/households", + "constituency", + "E14000001", + "households", + "census_households", + 40_000.0, + ), + _local( + "c1/age", "constituency", "E14000001", "age/0_10", "age_structure", 9_000.0 + ), + _local("c2/age", "constituency", "N05000001", "age/0_10", "age_structure", 4.0), + _local( + "l1/tenure", "la", "W06000001", "tenure/private_rent", "tenure", 10_000.0 + ), + _local( + "l1/amount", + "la", + "W06000001", + "hmrc/employment_income/amount", + "hmrc_income_by_area", + 9e9, + ), + _local( + "l1/count", + "la", + "W06000001", + "hmrc/employment_income/count", + "hmrc_income_by_area", + 100.0, + ), + _national("uk/population", "ons_population", "K02000001", "count", 69e6), + _national( + "gb/households", "ons_household_composition", "K03000001", "count", 4e6 + ), + _national("wales/population", "ons_population", "W92000004", "count", 3e6), + _national("se/income", "hmrc_spi_region", "E12000008", "gbp", 2e11), + ) + + +def test_vocabulary_admits_the_row_metadata_rules_as_overrides() -> None: + assert UK_LOCAL_TARGET_WEIGHT_RULES == ( + "uniform", + "grain_equal", + "grain_family_equal", + "grain_family_equal_sqrt_count", + "nation_grain_family_equal", + "nation_grain_family_equal_sqrt_count", + ) + assert ( + UK_LOCAL_SOLVE_DOCTRINE.target_weight_rule + == "nation_grain_family_equal_sqrt_count" + ) + for rule in UK_LOCAL_ROW_METADATA_RULES: + assert UKLocalSolveDoctrine(target_weight_rule=rule).target_weight_rule == rule + doctrine, receipt = uk_local_doctrine_with_overrides( + UK_LOCAL_SOLVE_DOCTRINE, {"target_weight_rule": rule} + ) + default = UK_LOCAL_SOLVE_DOCTRINE.target_weight_rule + assert receipt == ( + {} + if rule == default + else {"target_weight_rule": {"default": default, "effective": rule}} + ) + with pytest.raises(ValueError, match="uk_target_loss_weights_for_rows"): + uk_local_target_loss_weights(["national"], rule=rule) + + +def test_legacy_rules_match_the_grain_label_function_bit_for_bit(surface) -> None: + expected = uk_local_target_loss_weights(list(surface.grain), rule="grain_equal") + assert np.array_equal(uk_rule_loss_weights(surface, rule="grain_equal"), expected) + assert np.array_equal( + uk_rule_loss_weights(surface, rule="uniform"), np.ones(len(surface)) + ) + with pytest.raises(ValueError, match="must be one of"): + uk_rule_loss_weights(surface, rule="family_equal") + + +def test_grain_family_equal_splits_grains_then_families_equally(surface) -> None: + weights = uk_target_loss_weights_for_rows(surface, rule="grain_family_equal") + assert weights.sum() == pytest.approx(1.0) + # constituency grain: census households (1 row), age (2 rows) + assert weights[0] == pytest.approx(1 / 3 * 1 / 2) + assert weights[1] == pytest.approx(1 / 3 * 1 / 2 * 1 / 2) + assert weights[2] == pytest.approx(1 / 3 * 1 / 2 * 1 / 2) + # la grain: tenure (1 row), hmrc (2 rows) + assert weights[3] == pytest.approx(1 / 6) + assert weights[4] == weights[5] == pytest.approx(1 / 12) + # national grain: three families, population has two rows + assert weights[6] == weights[8] == pytest.approx(1 / 3 * 1 / 3 * 1 / 2) + assert weights[7] == weights[9] == pytest.approx(1 / 9) + + +def test_sqrt_count_keeps_every_cell_budget_and_shares_counts_by_root(surface) -> None: + plain = uk_target_loss_weights_for_rows(surface, rule="grain_family_equal") + sqrt = uk_target_loss_weights_for_rows( + surface, rule="grain_family_equal_sqrt_count" + ) + assert sqrt.sum() == pytest.approx(1.0) + # the constituency age cell keeps its budget; its two count rows split it + # by the square roots of 9,000 and 4 + cell = [1, 2] + assert sqrt[cell].sum() == pytest.approx(plain[cell].sum()) + assert sqrt[1] / sqrt[2] == pytest.approx(np.sqrt(9_000.0) / np.sqrt(4.0)) + # the hmrc cell mixes an amount and a count row: the amount row keeps its + # equal share, the count row keeps the count budget alone + assert sqrt[4] == pytest.approx(plain[4]) + assert sqrt[5] == pytest.approx(plain[5]) + # national population: two count rows split by root of value + assert sqrt[6] / sqrt[8] == pytest.approx(np.sqrt(69e6) / np.sqrt(3e6)) + assert sqrt[[6, 8]].sum() == pytest.approx(plain[[6, 8]].sum()) + assert uk_target_value_basis(surface) == ( + "count", "count", "count", "count", "amount", "count", + "count", "count", "count", "amount", + ) # fmt: skip + + +def test_nation_grain_moves_sub_uk_national_rows_and_shares_four_ways(surface) -> None: + weights = uk_target_loss_weights_for_rows(surface, rule="nation_grain_family_equal") + assert weights.sum() == pytest.approx(1.0) + receipt = uk_target_loss_weight_receipt( + surface, weights, rule="nation_grain_family_equal" + ) + grains = {block["grain"]: block for block in receipt["grains"]} + assert set(grains) == {"constituency", "la", "national", NATION_REGION_GRAIN} + for block in grains.values(): + assert block["loss_share"] == pytest.approx(0.25) + # UK (K02) and GB (K03) stay national; Wales (W92) and a region (E12) move + assert grains["national"]["n_targets"] == 2 + assert grains[NATION_REGION_GRAIN]["n_targets"] == 2 + # with no sub-UK national row the rule is grain_family_equal + uk_only = surface.take([0, 1, 2, 3, 4, 5, 6, 7]) + assert np.allclose( + uk_target_loss_weights_for_rows(uk_only, rule="nation_grain_family_equal"), + uk_target_loss_weights_for_rows(uk_only, rule="grain_family_equal"), + rtol=0, + atol=1e-15, + ) + + +def test_an_undeclared_basis_is_refused_only_by_the_sqrt_rules() -> None: + rows = _rows( + _local( + "c/x", + "constituency", + "E14000001", + "mystery/metric", + "census_households", + 5.0, + ), + _national("uk/x", "obr", "K02000001", "count", 1.0), + ) + assert uk_target_loss_weights_for_rows(rows, rule="grain_family_equal").sum() == ( + pytest.approx(1.0) + ) + with pytest.raises(ValueError, match="no declared count or amount basis"): + uk_target_loss_weights_for_rows(rows, rule="grain_family_equal_sqrt_count") + + +def test_nation_grain_refuses_a_national_row_without_geography() -> None: + rows = _rows( + _local( + "c/h", "constituency", "E14000001", "households", "census_households", 5.0 + ), + ("uk/x", {"family": "obr", "ledger_measure_unit": "gbp"}, 1.0, False), + ) + with pytest.raises(ValueError, match="names no geography"): + uk_target_loss_weights_for_rows(rows, rule="nation_grain_family_equal") + + +def test_the_carrier_refuses_rows_whose_signals_disagree() -> None: + with pytest.raises(ValueError, match="area_type"): + _rows( + _local("c/h", "ward", "E05000001", "households", "census_households", 5.0) + ) + with pytest.raises(ValueError, match="local area_type"): + _rows(("n/x", {"family": "obr", "area_type": "la"}, 1.0, False)) + with pytest.raises(ValueError, match="no family"): + _rows(_national("n/x", "", "K02000001", "count", 1.0)) + with pytest.raises(ValueError, match="unique"): + _rows( + _national("n/x", "obr", "K02000001", "count", 1.0), + _national("n/x", "obr", "K02000001", "count", 2.0), + ) + # a local row without a family falls back to the census classifier + rows = _rows( + ( + "c/h", + {"area_type": "la", "area_code": "E06000001", "metric": "households"}, + 5.0, + True, + ) + ) + assert rows.family == ("census_households",) + + +def test_targets_and_stored_problem_build_the_same_carrier(surface) -> None: + local_targets = [ + SimpleNamespace(row_name=name, metadata=meta, value=value) + for name, meta, value, local in zip( + surface.names, + [ + { + "area_type": grain, + "area_code": code, + "metric": metric, + "family": family, + } + for grain, code, metric, family in zip( + surface.grain, + surface.geography, + surface.measure, + surface.family, + strict=True, + ) + ], + surface.values, + surface.local, + strict=True, + ) + if local + ] + national_specs = [ + SimpleNamespace( + to_target=lambda name=name: SimpleNamespace(row_name=name), + metadata={"ledger_geography_id": code, "ledger_measure_unit": unit}, + family=family, + value=value, + ) + for name, code, unit, family, value, local in zip( + surface.names, + surface.geography, + surface.measure, + surface.family, + surface.values, + surface.local, + strict=True, + ) + if not local + ] + from_targets = uk_target_rows_from_targets(local_targets, national_specs) + stored = SimpleNamespace( + problem=SimpleNamespace(names=surface.names, target_vector=surface.values), + target_metadata=tuple( + { + "area_type": grain, + "area_code": code, + "metric": metric, + "family": family, + "materialization": "uk_local_surface", + } + if local + else { + "family": family, + "ledger_geography_id": code, + "ledger_measure_unit": metric, + "materialization": "uk_national_measure", + } + for local, grain, code, metric, family in zip( + surface.local, + surface.grain, + surface.geography, + surface.measure, + surface.family, + strict=True, + ) + ), + ) + from_problem = uk_target_rows_from_problem(stored) + for rule in UK_LOCAL_TARGET_WEIGHT_RULES: + assert np.array_equal( + uk_rule_loss_weights(from_targets, rule=rule), + uk_rule_loss_weights(from_problem, rule=rule), + ) + broken = SimpleNamespace( + problem=stored.problem, + target_metadata=( + {**stored.target_metadata[0], "materialization": "uk_national_measure"}, + *stored.target_metadata[1:], + ), + ) + with pytest.raises(ValueError, match="disagrees with area_type"): + uk_target_rows_from_problem(broken) + + +def test_weights_digest_is_the_canonical_row_name_and_hex_form() -> None: + digest = uk_target_loss_weights_sha256(["a@2025", "b@2025"], np.array([0.25, 0.75])) + assert digest == "895ece4fa4b33fcffe5c2ff05d73333fc6466b592df58c482d2a70582220ac9f" + + +def test_receipt_shares_sum_to_one_by_every_partition(surface) -> None: + for rule in UK_LOCAL_ROW_METADATA_RULES: + weights = uk_target_loss_weights_for_rows(surface, rule=rule) + receipt = uk_target_loss_weight_receipt(surface, weights, rule=rule) + for key in ("grains", "cells", "nations"): + assert sum(block["loss_share"] for block in receipt[key]) == pytest.approx( + 1.0 + ) + assert receipt["weights_sha256"] == uk_target_loss_weights_sha256( + surface.names, weights + ) + if rule.endswith("_sqrt_count"): + assert all("n_count" in block for block in receipt["cells"]) + + +def test_stage_weighting_zeroes_held_rows_and_reapplies_the_rule(surface) -> None: + held = (1, 4) + stage = UKStageTargetWeighting("grain_family_equal", surface, held_out=held) + weights = stage.loss_weights(surface.names) + assert (weights[list(held)] == 0).all() + training = [i for i in range(len(surface)) if i not in held] + assert np.array_equal( + weights[training], + uk_rule_loss_weights(surface.take(training), rule="grain_family_equal"), + ) + assert np.array_equal( + stage.held_out_weights(), + uk_rule_loss_weights(surface.take(list(held)), rule="grain_family_equal"), + ) + receipt = stage.receipt(surface.names) + assert receipt["n_held_out"] == 2 and receipt["rule"] == "grain_family_equal" + with pytest.raises(ValueError, match="another row axis"): + stage.loss_weights(surface.names[::-1]) + with pytest.raises(ValueError, match="must be one of"): + UKStageTargetWeighting("family_equal", surface) + with pytest.raises(ValueError, match="at least one row"): + UKStageTargetWeighting("uniform", surface, held_out=tuple(range(len(surface)))) + # no held rows: the stage vector is the rule's own vector + assert np.array_equal( + UKStageTargetWeighting("grain_equal", surface).loss_weights(surface.names), + uk_rule_loss_weights(surface, rule="grain_equal"), + ) diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_terminal_gates.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_terminal_gates.py index 934ba203a5..ed4563fedf 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_terminal_gates.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_terminal_gates.py @@ -759,9 +759,16 @@ def test_concentrated_qrf_output_fails_by_name() -> None: def test_committed_degenerate_register_is_the_policy_of_record() -> None: register = uk_default_degenerate_reviewed_exclusions() assert set(register) == { + "household.atomic_area_basis", "household.source_year", "person.tax_free_childcare_spend_routed_share", } + basis = register["household.atomic_area_basis"] + assert basis.approved_by == "juaristi22" + assert basis.approved_on == "2026-10-13" + assert basis.expires_on == "2027-04-13" + # Constant by construction only on the local line's K-clone pool. + assert basis.release_lines == ("local",) record = register["household.source_year"] assert record.adjudication == "microcosm#630" assert record.approved_by == "juaristi22" diff --git a/packages/microcosm-build/tests/engine_free/uk/test_uk_weighted_integrity.py b/packages/microcosm-build/tests/engine_free/uk/test_uk_weighted_integrity.py index 61cd1fa0b8..7338654b32 100644 --- a/packages/microcosm-build/tests/engine_free/uk/test_uk_weighted_integrity.py +++ b/packages/microcosm-build/tests/engine_free/uk/test_uk_weighted_integrity.py @@ -35,6 +35,7 @@ UKQRFTailConcentrationPolicy, UKReviewedExclusion, coerce_input_mass_reference_registry, + coerce_reviewed_exclusions, load_uk_input_mass_reference, load_uk_local_area_support_exclusion_register, load_uk_reference_scoped_exclusion_register, @@ -464,7 +465,22 @@ def test_qrf_columns_check_every_declared_output_regardless_of_density() -> None assert "density_filter" in surface # Declared outputs the fixture does not carry are reported, not invented. assert "dividend_income" in surface["absent_columns"] - assert surface["declared_qrf_outputs"] >= 47 + # The 48 declared outputs, less the ten spi_benefit_coherence replaces on + # every SPI-channel row (microcosm#1100); they hold no forest draw. + assert surface["declared_qrf_outputs"] == 38 + assert surface["superseded_by_spi_benefit_coherence"] == [ + "income_support_reported", + "working_tax_credit_reported", + "child_tax_credit_reported", + "sda_reported", + "iidb_reported", + "afcs_reported", + "bsp_reported", + "jsa_contrib_reported", + "jsa_income_reported", + "esa_income_reported", + ] + assert not set(surface["superseded_by_spi_benefit_coherence"]) & set(values) def test_declared_absent_qrf_output_is_a_named_gate_failure() -> None: @@ -541,6 +557,24 @@ def test_qrf_surface_rejects_partially_omitted_declared_outputs() -> None: assert "QRF surface declarations must reconcile exactly" in gate.failures[0] +def test_the_superseded_columns_are_the_coherence_stage_declaration() -> None: + """The gate reads the stage's declaration, which names the same columns + the stage's transform zeroes and restores.""" + + from microcosm.build.uk_runtime.hmrc_source_contract import ( + uk_spi_coherence_superseded_columns, + ) + from microcosm.build.uk_runtime.spi_benefit_coherence import ( + SPI_RESTORED_REPORT_COLUMNS, + SPI_ZEROED_REPORT_COLUMNS, + ) + + assert uk_spi_coherence_superseded_columns() == ( + *SPI_ZEROED_REPORT_COLUMNS, + *SPI_RESTORED_REPORT_COLUMNS, + ) + + def test_concentrated_qrf_column_fails_by_name() -> None: n = 10 concentrated = np.ones(n) @@ -562,7 +596,7 @@ def test_concentrated_qrf_column_fails_by_name() -> None: assert gate.name == "qrf_tail_concentration" assert not gate.passed assert "self_employment_income" in gate.failures[0] - assert gate.details["surface"]["declared_qrf_outputs"] >= 47 + assert gate.details["surface"]["declared_qrf_outputs"] == 38 def test_thin_qrf_column_is_reported_not_checked() -> None: @@ -648,6 +682,29 @@ def test_qrf_policy_validation() -> None: ) +def test_release_lines_scope_a_receipt_without_moving_unscoped_payloads() -> None: + plain = coerce_reviewed_exclusions({"x": _entry("r")}, label="t")["x"] + # Unscoped receipts keep the pre-scope payload, so pinned digests hold. + assert set(plain.policy_payload()) == set(_entry("r")) + assert plain.applies_on("local") and plain.applies_on(None) + + scoped = coerce_reviewed_exclusions( + {"x": {**_entry("r"), "release_lines": ["national"]}}, label="t" + )["x"] + assert scoped.policy_payload()["release_lines"] == ["national"] + assert scoped.applies_on("national") and not scoped.applies_on("local") + # A scoped entry fails closed where no line is declared. + assert not scoped.applies_on(None) + + for bad in ([], ["regional"], ["local", "local"], "national"): + with pytest.raises(ValueError, match="release_lines"): + coerce_reviewed_exclusions( + {"x": {**_entry("r"), "release_lines": bad}}, label="t" + ) + with pytest.raises(ValueError, match="optionally with"): + coerce_reviewed_exclusions({"x": {**_entry("r"), "scope": "x"}}, label="t") + + def test_committed_exclusion_registers_load() -> None: input_mass = load_uk_reference_scoped_exclusion_register( None, @@ -682,18 +739,33 @@ def test_committed_exclusion_registers_load() -> None: "dfe_education_spending", "property_wealth", } - assert set(qrf_tail) == {"charitable_investment_gifts"} + assert set(qrf_tail) == { + "charitable_investment_gifts", + "hmrc_spi_other_income", + "hmrc_spi_taxable_termination_pay", + } + # All three were earned on the national line's evidence. + assert {record.release_lines for record in qrf_tail.values()} == {("national",)} for record in (*input_mass["efrs-post-calibration"].values(), *qrf_tail.values()): assert record.approved_by == "juaristi22" - assert record.approved_on in {"2026-10-02", "2026-10-04", "2026-10-11"} + assert record.approved_on in { + "2026-10-02", + "2026-10-04", + "2026-10-11", + "2026-10-14", + } assert record.expires_on in { "2027-04-02", "2027-01-04", "2027-04-04", "2027-04-11", + # Time-boxed until the tail gate counts independent draws. + "2027-01-14", } - assert "microcosm#1063" in record.adjudication or ( - "microcosm#1106" in record.adjudication + assert ( + "microcosm#1063" in record.adjudication + or "microcosm#1106" in record.adjudication + or "microcosm#1277" in record.adjudication ) assert uk_default_input_mass_reviewed_exclusions() is ( uk_default_input_mass_reviewed_exclusions() diff --git a/packages/microcosm-build/tests/engine_workflow/us/test_us_multispine_pool_tool.py b/packages/microcosm-build/tests/engine_workflow/us/test_us_multispine_pool_tool.py index 799d806d36..47e910c59b 100644 --- a/packages/microcosm-build/tests/engine_workflow/us/test_us_multispine_pool_tool.py +++ b/packages/microcosm-build/tests/engine_workflow/us/test_us_multispine_pool_tool.py @@ -338,7 +338,10 @@ def capture_equality(expected: object, actual: object) -> None: "country": "us", "schema_id": "country_spec", "schema_version": 1, - "spec_sha256": "3080fe938df62216dc4853c1163639817de3c537e0e932309642fbd3f7d8f266", + # The US bundle's semantic hash covers microcosm.calibrate.solve's + # bytes (its seed protocol); microcosm#1124's opt-in + # retain_best_objective moves it. + "spec_sha256": "4d08ff77d2118ec4f7d63fcf17983f774b9d1142b0b8308bfdc30767ece76f1f", }, } diff --git a/packages/microcosm-calibrate/src/microcosm/calibrate/geography_constants.py b/packages/microcosm-calibrate/src/microcosm/calibrate/geography_constants.py index e3e96ccb69..07a4b3aa04 100644 --- a/packages/microcosm-calibrate/src/microcosm/calibrate/geography_constants.py +++ b/packages/microcosm-calibrate/src/microcosm/calibrate/geography_constants.py @@ -15,6 +15,7 @@ "UK_LADDER_NATION_REGION_CODES", "UK_REGION_TIER", "UK_REGION_TIER_ENUM", + "UK_WIDE_GEOGRAPHY_IDS", "US_STATE_FIPS_TO_POSTAL", "US_STATE_NUMERIC_FIPS_TO_POSTAL", "US_STATE_POSTAL_TO_NUMERIC_FIPS", @@ -47,6 +48,14 @@ ) +#: The UK-wide aggregates among ``UK_GEOGRAPHY_ID_TO_LABEL`` (the K-prefixed +#: GSS codes: the United Kingdom, Great Britain, England and Wales). Every +#: other national reference names a nation or an English region; the nation +#: grain of the UK target-weight rules splits on this set (microcosm#1124). +UK_WIDE_GEOGRAPHY_IDS: frozenset[str] = frozenset( + code for code in UK_GEOGRAPHY_ID_TO_LABEL if code.startswith("K") +) + #: The twelve-area region tier of the national calibration surface, in the #: order the FRS ``gvtregno`` coding and the incumbent's regional rows use: #: the nine English regions at Chronicle's ``region`` level, then Wales, diff --git a/packages/microcosm-calibrate/src/microcosm/calibrate/solve.py b/packages/microcosm-calibrate/src/microcosm/calibrate/solve.py index 9051ee6f3a..7aa5557344 100644 --- a/packages/microcosm-calibrate/src/microcosm/calibrate/solve.py +++ b/packages/microcosm-calibrate/src/microcosm/calibrate/solve.py @@ -1032,6 +1032,7 @@ def _optimize( progress_context: Mapping[str, object] | None = None, return_gate_open_probabilities: bool = False, selection_receipt: dict[str, object] | None = None, + retain_best_objective: bool = False, ) -> tuple[np.ndarray, np.ndarray] | tuple[np.ndarray, np.ndarray, np.ndarray | None]: """Run the torch optimization and return weights plus its trajectory. @@ -1129,7 +1130,14 @@ def _optimize( w0_t = None anchor_share_t = None - retain_best = gates is None and not conserve_mass and l2_lambda == 0.0 + # Without an L2 penalty the solve keeps its best target-loss epoch. With + # one, ``retain_best_objective`` keeps the epoch with the lowest objective + # the optimizer minimizes (target loss plus the weighted penalty); without + # it the closing state is returned, as before. + track_objective = l2_lambda != 0.0 and retain_best_objective + retain_best = ( + gates is None and not conserve_mass and (l2_lambda == 0.0 or track_objective) + ) best_loss = float("inf") best_epoch = 0 best_log_w: torch.Tensor | None = None @@ -1166,8 +1174,13 @@ def _optimize( l2_penalty = _l2_penalty(torch.exp(log_w), w0_t, anchor_share_t, l2_basis) total_loss = loss + penalty + l2_lambda * l2_penalty trajectory[epoch] = float(loss.item()) - if retain_best and trajectory[epoch] < best_loss: - best_loss = trajectory[epoch] + tracked = ( + float((loss + l2_lambda * l2_penalty).item()) + if track_objective + else trajectory[epoch] + ) + if retain_best and tracked < best_loss: + best_loss = tracked best_epoch = epoch best_log_w = log_w.detach().clone() if progress_callback is not None: @@ -1233,6 +1246,10 @@ def _optimize( target_loss_scales, target_loss_cap, ) + if track_objective: + closing_loss = closing_loss + l2_lambda * _l2_penalty( + weights, w0_t, anchor_share_t, l2_basis + ) if float(closing_loss.item()) > best_loss: # retain_best requires free mass, so never under softmax. assert not softmax_mass, "best-iterate retention under softmax" @@ -1245,7 +1262,11 @@ def _optimize( if selection_receipt is not None: selection_receipt.update( { - "rule": "best_feasible_loss", + "rule": ( + "best_feasible_objective" + if track_objective + else "best_feasible_loss" + ), "selected_epoch": selected_epoch, "epochs_executed": epochs, "epoch_convention": "completed_optimizer_updates; zero is start", @@ -1986,6 +2007,7 @@ def calibrate( target_loss_cap: float = _DEFAULT_TARGET_LOSS_CAP, warm_start_weights: np.ndarray | None = None, progress_callback: Callable[[dict[str, object]], None] | None = None, + retain_best_objective: bool = False, ) -> CalibrationResult: """Calibrate ``weight_entity``'s weights to ``targets`` over ``frame``. @@ -2136,6 +2158,13 @@ def calibrate( "calibration_epoch", "epoch": 1, "epochs": 256, "loss": ...}``. Build drivers use this to publish staging telemetry; calibration results are unchanged. + retain_best_objective: With ``l2_lambda > 0`` on the plain Adam path + (free mass, no L0), return the epoch with the lowest objective the + optimizer minimizes, target loss plus ``l2_lambda`` times the + penalty, instead of the closing state. Recorded as + ``options["iterate_selection"] == "best_feasible_objective"``. + Without an L2 penalty the solve already keeps its best target-loss + epoch, and the flag changes nothing. Returns: A :class:`CalibrationResult` with the calibrated frame, per-target @@ -2470,6 +2499,7 @@ def calibrate( progress_callback=progress_callback, return_gate_open_probabilities=True, selection_receipt=iterate_selection_receipt, + retain_best_objective=retain_best_objective, **( {} if gate_initialization is None @@ -2543,13 +2573,15 @@ def calibrate( "epochs": epochs, "learning_rate": learning_rate, "iterate_selection": ( - "best_feasible_loss" - if method == "adam" - and mass == FREE_MASS - and l0_lambda == 0.0 - and l2_lambda == 0.0 - and target_records is None - else "closing_state" + "closing_state" + if method != "adam" + or mass != FREE_MASS + or l0_lambda != 0.0 + or target_records is not None + or (l2_lambda != 0.0 and not retain_best_objective) + else "best_feasible_objective" + if l2_lambda != 0.0 + else "best_feasible_loss" ), "iterate_selection_receipt": iterate_selection_receipt, "mass": mass, @@ -2837,6 +2869,7 @@ def refit_l0_selection( target_loss_scales: np.ndarray | None = None, target_loss_cap: float = _DEFAULT_TARGET_LOSS_CAP, progress_callback: Callable[[dict[str, object]], None] | None = None, + retain_best_objective: bool = False, ) -> L0RefitResult: """Refit ordinary calibration on a frozen support from an L0 solve. @@ -3022,6 +3055,7 @@ def refit_l0_selection( target_loss_scales=target_loss_scales, target_loss_cap=target_loss_cap, progress_callback=progress_callback, + retain_best_objective=retain_best_objective, ) return L0RefitResult( selection=selection, diff --git a/packages/microcosm-calibrate/tests/engine_free/shared/test_diagnostics.py b/packages/microcosm-calibrate/tests/engine_free/shared/test_diagnostics.py index 4d30fd2d7a..fff1c41741 100644 --- a/packages/microcosm-calibrate/tests/engine_free/shared/test_diagnostics.py +++ b/packages/microcosm-calibrate/tests/engine_free/shared/test_diagnostics.py @@ -41,6 +41,7 @@ from microcosm.calibrate._target_loss_attribution import target_loss_basis_hash from microcosm.calibrate.geography_constants import ( UK_GEOGRAPHY_ID_TO_LABEL, + UK_WIDE_GEOGRAPHY_IDS, US_STATE_FIPS_TO_POSTAL, US_STATE_NUMERIC_FIPS_TO_POSTAL, US_STATE_POSTAL_TO_NUMERIC_FIPS, @@ -79,6 +80,8 @@ def test_shared_geography_constant_views_are_consistent() -> None: "E12000008": "South East", "E12000009": "South West", } + assert UK_WIDE_GEOGRAPHY_IDS == {"K02000001", "K03000001", "K04000001"} + assert UK_WIDE_GEOGRAPHY_IDS <= set(UK_GEOGRAPHY_ID_TO_LABEL) assert len(US_STATE_FIPS_TO_POSTAL) == 51 assert US_STATE_FIPS_TO_POSTAL["01"] == "AL" assert US_STATE_FIPS_TO_POSTAL["11"] == "DC" diff --git a/packages/microcosm-calibrate/tests/engine_free/shared/test_l2_basis.py b/packages/microcosm-calibrate/tests/engine_free/shared/test_l2_basis.py index d4b160066d..0a2af329fc 100644 --- a/packages/microcosm-calibrate/tests/engine_free/shared/test_l2_basis.py +++ b/packages/microcosm-calibrate/tests/engine_free/shared/test_l2_basis.py @@ -326,6 +326,39 @@ def test_softmax_parametrization_conserves_mass_and_respects_the_cap( assert isinstance(exhausted, int) and 0 <= exhausted <= 300 +def test_retaining_the_best_objective_under_l2_never_returns_a_worse_epoch() -> None: + frame, targets, _design = _problem(3, n=60, k=4) + common = dict( + epochs=300, + learning_rate=0.3, + l2_lambda=0.05, + l2_anchor="uniform", + l2_basis=L2_BASIS_CHI_SQUARE, + seed=0, + ) + closing = calibrate(frame, targets, **common) + best = calibrate(frame, targets, retain_best_objective=True, **common) + assert closing.options["iterate_selection"] == "closing_state" + assert best.options["iterate_selection"] == "best_feasible_objective" + receipt = best.options["iterate_selection_receipt"] + assert receipt["rule"] == "best_feasible_objective" + assert receipt["selected_loss_float32"] <= receipt["closing_iterate_loss_float32"] + # One optimization path; only the returned epoch can differ. + np.testing.assert_array_equal(best.loss_trajectory, closing.loss_trajectory) + # Without an L2 penalty the solve already keeps its best epoch. + plain = calibrate(frame, targets, epochs=300, learning_rate=0.3, seed=0) + flagged = calibrate( + frame, + targets, + epochs=300, + learning_rate=0.3, + seed=0, + retain_best_objective=True, + ) + np.testing.assert_array_equal(plain.weights, flagged.weights) + assert flagged.options["iterate_selection"] == "best_feasible_loss" + + # --- 6: the regularization path ---------------------------------------------- #: Dense at large lambda, where the derived slack 2 * eps / (lam_b - lam_a) is diff --git a/packages/microcosm-data/src/microcosm/data/contract.py b/packages/microcosm-data/src/microcosm/data/contract.py index b22280c9ac..7b3e3fabf5 100644 --- a/packages/microcosm-data/src/microcosm/data/contract.py +++ b/packages/microcosm-data/src/microcosm/data/contract.py @@ -646,7 +646,7 @@ def compatibility_claim_declarer_error(declared_by: object) -> str | None: # is a vintage pin that moves with every reviewed register edit (the same # tripwire as the policy digest). _UK_GATE_BATTERY_DEGENERATE_EVIDENCE_SHA256 = ( - "6f0243bcda09dad26945376230c44ec3cf55d4e417c3a25e29bae8c59bc1a69d" + "6a179222c6b6058ad9caacfd347f7bdb8cb98a62237b50d1fc1136df09337251" ) diff --git a/packages/microcosm-data/tests/engine_free/shared/test_contract.py b/packages/microcosm-data/tests/engine_free/shared/test_contract.py index 1e17d62211..f4d901c31c 100644 --- a/packages/microcosm-data/tests/engine_free/shared/test_contract.py +++ b/packages/microcosm-data/tests/engine_free/shared/test_contract.py @@ -223,7 +223,7 @@ def _trusted_terminal_gate_signing_key(monkeypatch) -> None: "4560684d4f936f86766cf921f745a6971b2ec2bed2aaf4e7ab73a2081349c771" ) UK_GATE_BATTERY_DEGENERATE_EVIDENCE_SHA256 = ( - "6f0243bcda09dad26945376230c44ec3cf55d4e417c3a25e29bae8c59bc1a69d" + "6a179222c6b6058ad9caacfd347f7bdb8cb98a62237b50d1fc1136df09337251" ) UK_GATE_BATTERY_INPUT_MASS_EVIDENCE_SHA256 = ( "82a5fa0c7dec5ac5c169f0b8b2f4acc6b302e80637dc698d3247e75a6d133933" diff --git a/packages/microcosm-diagnostics/src/microcosm/diagnostics/schema.py b/packages/microcosm-diagnostics/src/microcosm/diagnostics/schema.py index e5052a334a..358360bd21 100644 --- a/packages/microcosm-diagnostics/src/microcosm/diagnostics/schema.py +++ b/packages/microcosm-diagnostics/src/microcosm/diagnostics/schema.py @@ -472,7 +472,16 @@ class UKMeasuredRotatedHoldout(DiagnosticsModel): method: Literal["rotated_folds"] target_loss_cap: PositiveFiniteFloat loss_weight_scale: Literal["held_local_grains_only"] - target_weight_rule: Literal["uniform", "grain_equal"] + # The UK local target-weight vocabulary (microcosm#1124); a microcosm-build + # test pins it to ``UK_LOCAL_TARGET_WEIGHT_RULES``. + target_weight_rule: Literal[ + "uniform", + "grain_equal", + "grain_family_equal", + "grain_family_equal_sqrt_count", + "nation_grain_family_equal", + "nation_grain_family_equal_sqrt_count", + ] population: Literal["held_out_local_targets"] grains: list[Literal["constituency", "local_authority", "la"]] = Field(min_length=1) n_folds: Annotated[int, Field(ge=2)] diff --git a/packages/microcosm-graph/tests/fixtures/parity/kernels/calibrate/pins.json b/packages/microcosm-graph/tests/fixtures/parity/kernels/calibrate/pins.json index c446db360a..a514be17ca 100644 --- a/packages/microcosm-graph/tests/fixtures/parity/kernels/calibrate/pins.json +++ b/packages/microcosm-graph/tests/fixtures/parity/kernels/calibrate/pins.json @@ -1 +1 @@ -{"dependencies":{"numpy":"2.4.6","pandas":"3.0.3","scipy":"1.17.1","torch":"2.12.0"},"implementation_hash":"b34011e4aa9ca6d13881ef921d31406cdc91da7ac3faca25bc0f481a1e802cc3","kernel":"calibrate.adam@1","node":"calibrate","node_key":"8a0a7c39a1a1c041102e1506b50561e3b7d28b28ebc7853a56d16735151e5856","numeric":"bitwise","platform":"arm64/darwin/py3.14","platforms":{"arm64/darwin/py3.14":{"direct":"direct.csv","node_key":"8a0a7c39a1a1c041102e1506b50561e3b7d28b28ebc7853a56d16735151e5856"}},"seed":0} +{"dependencies":{"numpy":"2.4.6","pandas":"3.0.3","scipy":"1.17.1","torch":"2.12.0"},"implementation_hash":"f429c85bad8689f4e453dbabe279068993380154395029673a5471277930f3c8","kernel":"calibrate.adam@1","node":"calibrate","node_key":"e19eb4a6f75638d1f06ea43d9080150942bd190b14eb2a95fadfe37c5a256c7f","numeric":"bitwise","platform":"arm64/darwin/py3.14","platforms":{"arm64/darwin/py3.14":{"direct":"direct.csv","node_key":"e19eb4a6f75638d1f06ea43d9080150942bd190b14eb2a95fadfe37c5a256c7f"}},"seed":0} diff --git a/test_support/microcosm_build/uk_full_build_cli.py b/test_support/microcosm_build/uk_full_build_cli.py index 4e9ba2293a..2d13aaf3dc 100644 --- a/test_support/microcosm_build/uk_full_build_cli.py +++ b/test_support/microcosm_build/uk_full_build_cli.py @@ -34,7 +34,10 @@ uk_full_gate_manifest, ) from microcosm.build.uk_runtime.graph_build import UKFullBuildConfig, UKFullGraph -from microcosm.build.uk_runtime.graph_calibration import UKCalibrationNodes +from microcosm.build.uk_runtime.graph_calibration import ( + SIZE_RECEIPT_TYPE, + UKCalibrationNodes, +) from microcosm.build.uk_runtime.graph_population import ( GEOGRAPHY_GATE_TYPE, POPULATION_RECEIPT_TYPE, @@ -589,6 +592,28 @@ def run(self, context): ) +class SizeReceipt(Evidence): + """A sized build's refit receipt: requested and realized household counts.""" + + ref = "uk.test.cli-size@1" + + def run(self, context): + receipt = size_receipt(int(context.params["households"])) + return KernelResult(artifacts={"size": canonical_json(receipt)}) + + +def size_receipt(households: int) -> dict: + """The stand-in size receipt of a build that realized its request.""" + + return { + "method": "exact_k_informed_l0", + "requested_households": households, + "realized_households": households, + "selection_pi_hi": 0.5, + "baseline_pi_floor": 0.5, + } + + class Targets(Evidence): """The compiled surface and the ordered problem the projection reads.""" @@ -695,8 +720,18 @@ def run_dense_main( monkeypatch.delenv("POPULACE_LOGBOOK_PREV_ROW_DIGEST", raising=False) patch_certification(monkeypatch) args = arguments(tmp_path, *extra, staging=staging) + households = ( + int(extra[extra.index("--dataset-households") + 1]) + if "--dataset-households" in extra + else None + ) build = ( - prepared(tmp_path, failed, release_candidate="--release-candidate" in extra) + prepared( + tmp_path, + failed, + release_candidate="--release-candidate" in extra, + size_households=households, + ) if build is None else build ) @@ -723,9 +758,10 @@ def graph_dense_bundle(tmp_path, monkeypatch, *extra, staging="--no-staging") -> return out -def prepared(tmp_path, failed=None, *, release_candidate=False): +def prepared(tmp_path, failed=None, *, release_candidate=False, size_households=None): """The synthetic prepared build; its gate nodes evaluate in the request's - posture (``release_candidate``), as the real gate nodes do.""" + posture (``release_candidate``), as the real gate nodes do. A sized + request (``size_households``) adds a stand-in size refit receipt.""" # Only a release-candidate request adds the parameter, so every other # synthetic build keeps its node keys. posture = {"release_candidate": True} if release_candidate else {} @@ -836,6 +872,16 @@ def prepared(tmp_path, failed=None, *, release_candidate=False): ), ), ] + if size_households is not None: + nodes.append( + Node( + "uk.full.size_refit", + SizeReceipt.ref, + population=root.id, + params={"households": int(size_households)}, + artifact_outputs=(ArtifactOutput("size", SIZE_RECEIPT_TYPE),), + ) + ) nodes = [ replace( node, @@ -902,6 +948,7 @@ def prepared(tmp_path, failed=None, *, release_candidate=False): Holdout(), Targets(), Population(), + SizeReceipt(), Certification(), ): kernels.register(kernel) diff --git a/tools/preflight_uk_local_release_candidate.py b/tools/preflight_uk_local_release_candidate.py index a5edef24dc..9c1a078749 100644 --- a/tools/preflight_uk_local_release_candidate.py +++ b/tools/preflight_uk_local_release_candidate.py @@ -8,7 +8,10 @@ ``--candidate-dir`` after the run: the manifest and the signed gate report say what a publishable release candidate must say — release posture attested, shippable, every release-blocking gate passed, - single-block engine, doctrine values, the A15 census household + an exact engine population (one block, or one block per clone + whose identical copies are scaled to the pool), doctrine values + (and, for a sized run, the size doctrine: 100,000 households under + the doctrine's draw, floor and refit penalty), the A15 census household uprating and A17 tenure application and the measure exclusions receipted, holdout measured, Logbook row present, artifact digest recorded and matching. @@ -99,7 +102,7 @@ def check_env( UK_LOCAL_SOLVE_EPOCHS, ) - ruled = (10.0, "grain_equal", 15, 1500) + ruled = (10.0, "nation_grain_family_equal_sqrt_count", 15, 2000) found = ( float(UK_LOCAL_MAX_WEIGHT_RATIO), UK_LOCAL_SOLVE_DOCTRINE.target_weight_rule, @@ -113,6 +116,62 @@ def check_env( return failures +#: The local size doctrine (microcosm#1124, ruling of 2026-10-13): a sized +#: release candidate ships this many households, drawn and refit under these +#: settings, with no selection L2 penalty (refused for every run above). +#: Restated here, like the solve doctrine, so the pre-flight checks the run +#: against the ruling rather than against the code that ran it. +SIZE_DOCTRINE = { + "dataset_households": 100_000, + "selection_pi_hi": 0.5, + "baseline_pi_floor": 0.5, +} +REFIT_L2_DOCTRINE = { + "lambda": 0.0215748772773509, + "anchor": "uniform", + "basis": "chi_square", +} + + +def _size_failures(manifest: dict, parameters: dict) -> list[str]: + """A sized run must carry the size doctrine; an unsized one, no refit L2.""" + + requested = parameters.get("dataset_households") + if requested is None: + if parameters.get("refit_l2") is not None: + return ["a refit L2 penalty was applied without a size request."] + return [] + failures = [ + f"parameters.{key} = {parameters.get(key)!r}, expected the size " + f"doctrine's {value!r}." + for key, value in SIZE_DOCTRINE.items() + if parameters.get(key) != value + ] + if parameters.get("refit_l2") != REFIT_L2_DOCTRINE: + failures.append( + f"parameters.refit_l2 = {parameters.get('refit_l2')!r}, expected the " + f"size doctrine's {REFIT_L2_DOCTRINE!r}." + ) + if parameters.get("resume_size_checkpoint") is not None: + failures.append( + "the selection was resumed from a size checkpoint; a release draws its own." + ) + size = manifest.get("solve", {}).get("dataset_size") + if not isinstance(size, dict): + failures.append("solve.dataset_size is missing from a sized run.") + elif not ( + size.get("requested_households") == size.get("realized_households") == requested + ): + failures.append( + "solve.dataset_size requested " + f"{size.get('requested_households')!r} / realized " + f"{size.get('realized_households')!r}, expected {requested!r}." + ) + if manifest.get("release_posture", {}).get("size_doctrine") is not True: + failures.append("release_posture.size_doctrine is not true.") + return failures + + def check_candidate_dir(candidate_dir: Path, *, today: date | None = None) -> list[str]: failures: list[str] = [] today = today or date.today() @@ -138,8 +197,8 @@ def check_candidate_dir(candidate_dir: Path, *, today: date | None = None) -> li doctrine = parameters.get("doctrine", {}) expected = { "max_weight_ratio": 10.0, - "target_weight_rule": "grain_equal", - "solve_epochs": 1500, + "target_weight_rule": "nation_grain_family_equal_sqrt_count", + "solve_epochs": 2000, "clone_count": 15, } for key, value in expected.items(): @@ -147,17 +206,35 @@ def check_candidate_dir(candidate_dir: Path, *, today: date | None = None) -> li failures.append( f"doctrine.{key} = {doctrine.get(key)!r}, expected {value!r}." ) - if parameters.get("epochs") != 1500 or parameters.get("n_clones") != 15: + if parameters.get("epochs") != 2000 or parameters.get("n_clones") != 15: failures.append( - f"run parameters epochs={parameters.get('epochs')} n_clones={parameters.get('n_clones')} are not the doctrine's 1500 / 15." + f"run parameters epochs={parameters.get('epochs')} n_clones={parameters.get('n_clones')} are not the doctrine's 2000 / 15." ) if parameters.get("skip_holdout"): failures.append("holdout was skipped.") + if parameters.get("selection_l2") is not None: + failures.append("a selection L2 penalty was applied (candidate-only).") + failures.extend(_size_failures(manifest, parameters)) solve = manifest.get("solve", {}) resolution = solve.get("measure_resolution", {}) - if resolution.get("blocks") != 1: + blocks = resolution.get("blocks") + representation = resolution.get("engine_population_representation") or {} + # One block per clone measures the pool exactly when the blocks are + # identical copies scaled to the pool (rowwise_cli.release_verdict); the + # build records that check, and nothing else stands in for it. + per_clone_exact = ( + type(blocks) is int + and blocks > 1 + and blocks == parameters.get("n_clones") + and representation.get("blocks") == blocks + and representation.get("mode") == "block_weights_scaled_to_pool" + and representation.get("exact") is True + ) + if blocks != 1 and not per_clone_exact: failures.append( - f"engine resolved in {resolution.get('blocks')} blocks; release posture is 1." + f"engine resolved in {blocks} blocks without an exact pool " + "representation; release posture is one block, or one block per " + "clone recorded exact." ) if solve.get("target_weight_rule_override"): failures.append("a target-weight-rule override was applied.") diff --git a/tools/score_uk_local_candidate.py b/tools/score_uk_local_candidate.py index 778fe3c699..fb9fddd76e 100644 --- a/tools/score_uk_local_candidate.py +++ b/tools/score_uk_local_candidate.py @@ -19,7 +19,10 @@ import numpy as np import pandas as pd -from microcosm.build.uk_runtime.local_doctrine import UK_LOCAL_TARGET_LOSS_CAP +from microcosm.build.uk_runtime.local_doctrine import ( + UK_LOCAL_TARGET_LOSS_CAP, + UK_LOCAL_TARGET_WEIGHT_RULES, +) from microcosm.calibrate import ( CALIBRATION_DIAGNOSTICS_SCHEMA_VERSION, TargetRegistry, @@ -27,7 +30,12 @@ relative_error_loss, ) -UK_LOCAL_ACTIVE_REFERENCE_COUNT = 20_885 +#: The compiled local reference count the score covers. microcosm#1123 +#: regenerated the UK target surface on the 942a1fa feed and moved it from +#: 20,885 to 21,804 (docs/evidence/uk-target-surface-1123/surface-diff.json, +#: ``register_completeness.compiled_local_reference_count``). +UK_LOCAL_ACTIVE_REFERENCE_COUNT = 21_804 +_LOCAL_GRAINS = ("constituency", "local_authority", "la") UK_LOCAL_SCORE_TARGET_PERIOD = 2025 #: The incumbent is scored from published weights, never re-solved, so no #: incumbent holdout exists to place beside the candidate's rotation. @@ -153,7 +161,7 @@ def _candidate_holdout(diagnostics: Mapping[str, object]) -> dict[str, object]: f"at {float(UK_LOCAL_TARGET_LOSS_CAP)!r}; re-measure the candidate " "rather than reporting the two on different scales." ) - if holdout.get("target_weight_rule") not in ("uniform", "grain_equal"): + if holdout.get("target_weight_rule") not in UK_LOCAL_TARGET_WEIGHT_RULES: raise ValueError("candidate holdout target_weight_rule is missing or invalid.") if holdout.get("loss_weight_scale") != "held_local_grains_only": raise ValueError( @@ -549,6 +557,35 @@ def _read_json(path: Path) -> dict[str, object]: return payload +def load_scored_registry(path: Path) -> TargetRegistry: + """The registry the score covers, from a registry artifact or a graph build. + + A graph-built candidate records its selected registry inside its target + selection (the manifest's ``outputs.target_registry``), national and local + rows together; the local rows are the surface this scorer covers. + """ + + payload = _read_json(path) + if not {"registry", "receipt"} <= set(payload): + return TargetRegistry.from_json(path) + from microcosm.build.uk_runtime.graph_targets import registry_from_payload + + selected = registry_from_payload(payload["registry"]) + return TargetRegistry( + [ + spec + for spec in selected.specs + if str( + spec.metadata.get( + "ledger_geography_level", spec.metadata.get("geography_level", "") + ) + ) + in _LOCAL_GRAINS + ], + country=selected.country, + ) + + def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--candidate-diagnostics-json", required=True, type=Path) @@ -557,7 +594,15 @@ def main(argv: list[str] | None = None) -> int: parser.add_argument("--incumbent-weights-sha256", required=True) parser.add_argument("--incumbent-household-metrics-csv", required=True, type=Path) parser.add_argument("--incumbent-household-metrics-sha256", required=True) - parser.add_argument("--registry-json", required=True, type=Path) + parser.add_argument( + "--registry-json", + required=True, + type=Path, + help=( + "the frozen local target registry artifact, or a graph-built " + "candidate's target selection (manifest outputs.target_registry)" + ), + ) parser.add_argument("--registry-sha256", required=True) parser.add_argument("--output-json", required=True, type=Path) args = parser.parse_args(argv) @@ -584,7 +629,7 @@ def main(argv: list[str] | None = None) -> int: candidate_diagnostics=_read_json(args.candidate_diagnostics_json), incumbent_weights=pd.read_csv(args.incumbent_weights_csv), incumbent_metrics=pd.read_csv(args.incumbent_household_metrics_csv), - target_registry=TargetRegistry.from_json(args.registry_json), + target_registry=load_scored_registry(args.registry_json), ) score["artifacts"] = artifacts args.output_json.write_text(