Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion changelog.d/1124-uk-qrf-tail-surface.changed.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
The UK QRF tail-concentration gate no longer checks the report columns `spi_benefit_coherence` (microcosm#1100) replaces on every SPI-channel row. The stage zeroes the income-tested legacy reports (contributory and income-based JSA, Income Support, Working and Child Tax Credit, income-related ESA, Severe Disablement Allowance) and restores IIDB, AFCS and bereavement support from the FRS twin. After it these columns hold only FRS respondents' own reports, so the donor-tail broadcast the gate looks for (#462) cannot occur in them; their concentration is that of a small population of genuine reporters and their clone copies. The surface reads the superseded columns from the stage's declared operations and lists them in its details as `superseded_by_spi_benefit_coherence`.

`hmrc_spi_other_income` and `hmrc_spi_taxable_termination_pay` stay on the surface: they are real stage-1 draws. They failed national release-cut certification as genuinely concentrated instruments, rare on the SPI tape, whose top 100 records carry nearly their whole mass whether counted by row or by independent record; the CGT clones and SPI copies also take them past the carrier floor. They are deferred behind national-line reviewed exclusions, time-boxed to 2027-01-14, until the gate is re-armed on independent draws (microcosm#1277).
`hmrc_spi_other_income` and `hmrc_spi_taxable_termination_pay` stay on the surface: they are real stage-1 draws. They failed national release-cut certification as genuinely concentrated instruments, rare on the SPI tape, whose top 100 records carry nearly their whole mass whether counted by row or by independent record; the CGT clones and SPI copies also take them past the carrier floor. They were deferred behind national-line reviewed exclusions until microcosm#1281 removed the gate and its exclusion register; the check that replaces it is tracked in microcosm#1276.
1 change: 1 addition & 0 deletions changelog.d/1276-remove-uk-qrf-tail-gate.removed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The UK national release-cut battery no longer runs the QRF tail-concentration gate (`uk_qrf_tail_concentration`), and its exclusion register (`qrf_tail_reviewed_exclusions.json`) is gone. On the UK surface the gate's top-100 share could only bite on a column with fewer than 100 independent draws whose CGT-clone copies took it past the 104-row floor, so its verdict measured how little mass those copies carry rather than whether an imputation broadcast a donor tail; and at the armed 0.99947 threshold the #462 incident it was built for (89% of a column's mass in 100 records) would have passed. The data contract drops the gate's pins, detail schema and validation, and re-pins the release-cut part and full-battery digests. `uk_qrf_tail_concentration_columns` and the weighted-integrity baselines tool stay as the measurement surface for the replacement microcosm#1276 asks for: a source-fidelity check of each imputed column against its donor.
6 changes: 3 additions & 3 deletions docs/gate-battery-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,6 @@ Canonical artifact keys:
| `parity_evidence` | the UK national build | `export_surface`, `target_surface`, `target_fit` |
| `input_mass_reference` | the UK national build, when the licensed sidecar is supplied | `input_mass_parity` (UK override) |
| `reviewed_input_mass_exclusions` (optional override) | the UK national build | `input_mass_parity` (UK override) |
| `reviewed_qrf_tail_exclusions` (optional override) | the UK national build | `tail_concentration` (UK override) |
| `uk_area_support_summary`, `exclusions_evaluated_on` | the UK local rowwise build | `area_support` (UK binding; reviewed exclusions carry measured shortfalls and stale entries fail) |
| `cgt_projection` | the UK calibration seam: read from the installed engine's growth path and exempt amounts, or stated from the manifest pins when no engine is installed (the receipt's `projection_engine` names the source, and the release certifier refuses a seam part whose fence read the pins) | `cgt_projection_entrants` (UK binding; the count is a cumulative stock by year, the bound is the vendored HMRC thin-band taxpayer count as a plausibility ceiling, and the growth path and exempt amount are pinned in the manifest and drift-checked, microcosm#970) |

Expand All @@ -139,8 +138,9 @@ reference implementation of a country registry. Its bindings adapt the
reviewed policy objects from the frozen declared parameters, hold
runtime-supplied references to the spec-declared pins, declare the
parameter vocabulary each evaluator can route (so a stray key is refused
at arm time, not discovered mid-battery), and re-mint exactly
two UK-flavored result names onto the shared vocabulary. Every verdict
at arm time, not discovered mid-battery), and re-mint the
UK-flavored result names of the legacy evaluators onto the shared
vocabulary. Every verdict
is computed by the existing UK gate functions — a binding adapts
evidence; it never re-implements a comparison. The differential test
(`tests/test_uk_battery_bindings.py`) holds the battery's verdicts equal
Expand Down
20 changes: 20 additions & 0 deletions docs/uk-national-release-assembly-runbook-806.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,26 @@ With the default paths, this writes
`microcosm_uk_2024_25.release_certification.json` next to the candidate. Continue
only when the certification says `shippable: true`.

**Interim check until microcosm#1276 lands.** microcosm#1281 removed the QRF
tail-concentration gate, and nothing release-blocking yet checks the SPI
stage-1 columns (`hmrc_spi_*` and the other stage-1 outputs). Before assembly,
measure the candidate and the currently promoted cut together:

```bash
uv run --no-sync python tools/measure_uk_weighted_integrity_baselines.py \
--h5 candidate=<candidate-dir>/microcosm_uk_2024_25.h5 \
--h5 previous=<promoted-cut-h5> \
--output <candidate-dir>/stage1_comparison.json
```

Compare each stage-1 column's weighted total (`input_mass_totals`) and carrier
count (`qrf_tail`) between the two, and explain any column that moves by more
than half in either direction before continuing. microcosm#1256 cut
`hmrc_spi_incapacity_benefit_income` by about six-sevenths, and only this kind
of comparison would have shown it. The file is licensed output: keep it
local, out of the release directory and out of any public text. The tool
suppresses carrier counts below 10.

## 3. Assemble the release directory

Assembly verifies the complete hash join before writing, mints the calibration
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -242,11 +242,6 @@
"kind": "legacy_json",
"schema_id": "legacy_json"
},
{
"path": "qrf_tail_reviewed_exclusions.json",
"kind": "legacy_json",
"schema_id": "legacy_json"
},
{
"path": "release_input_coverage_manifest.json",
"kind": "legacy_json",
Expand Down
13 changes: 0 additions & 13 deletions packages/microcosm-build/src/microcosm/build/uk/gates.json
Original file line number Diff line number Diff line change
Expand Up @@ -1247,19 +1247,6 @@
},
"notes": "Weighted input-mass parity against the active named reference in the registry, with reviewed exclusions scoped per reference so a channel-blind reference exclusion does not suppress comparison against a future channel-aware reference. The tolerance is the worst surviving measured drift at the reviewed floor, exact, with no headroom; the reviewed per-reference exclusions (charitable_investment_gifts, signed at +15,168.8% on the post-#710 surface; owned_land, signed for seed variance) were retired on 2026-09-22 when the first release-cut certification measured both inside the fence (-99.0% and -73.5% against the enhanced FRS reference, microcosm#967); the register is empty and the gate's current failures are the two genuine breaches it names. The 0.0 floor excludes only arithmetically meaningless zero-mass denominators. Totals stay uncommitted under the UKDS licence, supplied per-run and pinned by totals_sha256; missing sidecar evidence records evidence_absent. Totals keys are flat frame column names since the #611 A4 dedup; that digest re-freeze was keys-only against the #610 values (identity unchanged, values byte-identical). Sensitivity floor, recorded rather than silent (microcosm#706 review): the tolerance is the worst surviving measured drift, and that edge is currently set by a column whose deviation is explained (vintage plus SPI composition, microcosm#703's head-to-head receipt), so at this value the gate is a gross mass-loss/explosion fence rather than a regression detector — the compared population sits at median 0.24 and p90 0.63. Tightening runs through this register: each explained top drift that earns a receipt lets the tolerance re-mint at the worst unexplained survivor, with the full distribution and the exclusions-versus-tolerance tradeoff in experiments/630-uk-gate-adjudication-receipts.md."
},
{
"id": "uk_qrf_tail_concentration",
"gate": "tail_concentration",
"phase": "terminal",
"criticality": "release_blocking",
"parameters": {
"reviewed_exclusions_resource": "qrf_tail_reviewed_exclusions.json",
"top_k": 100,
"max_top_share": 0.9994670564654868,
"min_nonzero_records": 104
},
"notes": "Weighted tail-concentration audit of the QRF-imputed output columns derived from the HMRC source manifest. Armed from the #686 L3 baselines per #757 B4, each threshold recording the run that measured it: the 47-column qrf_tail grids of uk_weighted_integrity_baselines_686.json (sha256 8b3f6c4e9c522445bf3e05d3451ee8557f4e14d25524ceac3af8bfe05401a146, licensed acceptance dir 686-spine-swap), measured on spine-a.h5 (sha256 a65f2132736d1dcecb91709a513a0d54a5887635ea03760629cc888d188ee306, 113,649 person rows, design weights) for the #609/#578 threshold adjudication. top_k 100 is the measurement grid anchor [10, 100, 500, 1000]; max_top_share is the exact measured maximum over the checked surface (hmrc_spi_other_social_security_income, 104 carriers), no headroom; min_nonzero_records is the thinnest measured column above the grid anchor (the same column) - the policy domain requires min_nonzero_records > top_k, so the three saturated sub-anchor columns (hmrc_spi_taxable_termination_pay 12, charitable_investment_gifts 22, sda_reported 24 carriers, each top-100 share 1.0) sit below it and go thin visibly in the signed details on every run, never a silent pass. The baselines are design-weight measurements and the terminal gate runs on the calibrated release frame; a calibrated-weight breach names its column and is a finding, not noise. The 12 household_qrf_tail grids (was_wealth surface) are measured in the same baselines file and not yet armed - a household-surface gate entry is a declared follow-up. Reviewed exclusions live in the named register resource of this package."
},
{
"id": "uk_local_geography_ladder_post_calibration",
"gate": "spine_agreement",
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -577,14 +577,12 @@
from microcosm.build.uk_runtime.weighted_integrity import (
UKInputMassParityPolicy,
UKInputMassReference,
UKQRFTailConcentrationPolicy,
load_uk_input_mass_reference,
load_uk_local_area_support_exclusion_register,
load_uk_reviewed_exclusion_register,
uk_input_mass_parity_gate,
uk_input_mass_totals,
uk_qrf_tail_concentration_columns,
uk_qrf_tail_concentration_gate,
)

__all__ = [
Expand Down Expand Up @@ -1025,7 +1023,6 @@
"UK_EDUCATION_GRANT_CAPACITY_PREDICTORS",
"UK_LEGACY_PROXY_PREDICTORS",
"UK_YEAR_RULES",
"UKQRFTailConcentrationPolicy",
"UKZeroWeightStratumDeclaration",
"load_uk_frs_release",
"load_uk_input_mass_reference",
Expand All @@ -1036,7 +1033,6 @@
"uk_input_mass_parity_gate",
"uk_input_mass_totals",
"uk_qrf_tail_concentration_columns",
"uk_qrf_tail_concentration_gate",
"uk_target_fit_gate",
"uk_target_surface_gate",
"uk_weight_ess_gate",
Expand Down
Loading
Loading