Skip to content

Certify Microcosm UK national cut a133f37a for the Autumn Budget - #715

Draft
MaxGhenis wants to merge 9 commits into
budget-local-data-manifestfrom
certify-uk-a133f37a-national
Draft

MaxGhenis wants to merge 9 commits into
budget-local-data-manifestfrom
certify-uk-a133f37a-national

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #714

Certifies Saturday's Microcosm UK national cut for the Autumn Budget (28 Oct): microcosm-uk-2024-25-national-20261023T075039Z-a133f37a on policyengine-uk==2.147.2 and core 3.33.3, national-only (--drop-regional-release).

  • It is stacked on Support bundle-pinned local data release manifests #704 (budget-local-data-manifest, 463c3a953), under the same-line re-cut exception that docs/engineering/skills/data-certification.md records (the Budget coordinator's 23 October decision under d1682 and d1344).
  • Constituency and local-authority outputs are absent until Monday's cut.
  • María Juaristi (juaristi22) can override the routing decision.

Draft until the 2×2 attribution has run and its result is reviewed. The Budget coordinator asked for this PR to open now, so the review of the rest of the evidence and CI on the test fixes can start. The 2×2 runs in Fleet ops' reservation, from the end of the PolicyBench booking (Sat 22:00Z at the latest) to Sun 10:00Z.

Impact: this changes the certified UK default dataset to a new cut of the same Microcosm line, and the UK model pin from 2.132.6 to 2.147.2. Area regions drop out of the default registry until Monday's cut. The new pin also brings in policyengine-uk#2396's default capital gains realisation response, a retention-rate elasticity of 1 (first released in 2.134.0, so 2.132.6 lacks it). On this candidate the reform contract finds that default on every route. The country routes read the elasticity as 1.0, and the wrapper routes' realised gains and CGT changes match an elasticity of 1. So CGT reforms costed through policyengine.py respond by default once this lands. #602 tracks what follows. Its step 6 makes the contract's default-response cases gate (they report as pending until then). Its step 7 updates the app's methodology page with policyengine-api's bump. The 2×2 attribution, still to come, will separate the model and data effects on the Budget headline figures for FY2026-27 to FY2030-31.

Mechanical handoff; wrapper qualification pending unless reviewed receipts cover this exact candidate pair, commit and bundle. Microcosm's 19 producer release-cut gates and passing pytest do not establish wrapper qualification.

  • Regional route: --drop-regional-release
  • Compatibility basis: built_with_model_package
  • Made by tools/certify_uk_cut_in_policyengine_py.py in PolicyEngine/microcosm at 0788c00c7.

Evidence package (same-line re-cut exception)

docs/engineering/skills/data-certification.md ("Same-line re-cut exception") lists what this exception needs. The status of each item at e02d41b:

Item Status Receipt
Microcosm release-cut gates pass: 19 of 19 producer-gates.json (producer evidence only)
UC floor pass: benefit units 0.925 of 6.7m (DWP), spend 0.958 of £79.3bn (OBR), GB, 2025 certification.uc_floor.json
Household calculator snapshots pass: 11 passed certify log
Strict check (bundle.py check --published-spm --include-tros --strict-tros) pass with one warning: the US release's core claims name 3.32.5, but the bundle pins 3.33.3. The certificate names no core, so the US follows the core pin. This is US-only and not changed here. certify log
#574 reform contract on the exact cut pass: Contract 2.1.0, 905 checks pass, 0 fail, 0 error. The 34 pending checks belong to the two default capital gains response cases. The contract reports them without gating until policyengine.py pins a policyengine-uk release with the #2396 default (a retention-rate elasticity of 1, made the default by policyengine-uk#2419 on 15 Oct and first released in 2.134.0). This candidate pins 2.147.2, which has it, and all 34 hold (holds: true; for example, the default elasticity reads 1.0 against an expected 1.0). The 120 unsupported are route and case pairs the contract excludes by design. The counts match the 21 Oct qualification run on the enhanced FRS (job 119239815076). Ran 05:48Z to 09:10Z on 24 Oct (3 h 22 min). reform-contract.json
Budget-inputs gates on the exact cut needs review, accepted for this cut by the Budget coordinator's explicit decision (comment): 13 newly missing inputs, none a data loss (see below). The condition: the dashboard holds any measure whose figures depend on one of them as a data limit, until a cut carries the input or a named person at the Treasury check (G3) releases it with the default stated. Supported years pass (2024–2032). budget-inputs.json
Same-line 2×2 attribution vs 6cba5d71 on pe-uk 2.132.6, FY2026-27 to FY2030-31 ran; reviewed mappings attribute 144 of 1,092 flagged moves; the other 948 keep their unattributed label (712 poverty, 236 money). The run passed at 02:19Z on 25 Oct (540 s), with all four cells computable. The model's effect on the government balance in 2026-27 is −£53.9bn on the old cut and −£46.0bn on this one. VAT's whole-head change is 87.66% and 102.71% of that move: policyengine-uk#2187 moved VAT's grossing divisor from 0.38 to 0.469, and VAT under 2.147.2 is 0.810235 of VAT under 2.132.6 in every year on both cuts (0.38 / 0.469 = 0.810235). Other model mappings: SDLT (#2235, #2241, #2477, #2446), Universal Credit (#1989 and four others; their historical effects overlap and aren't added up), State Pension (#2336), childcare (#2570, #2465), War Widow's Pension (#2382) and, in 2029–30, NI (#1863). The data's effect on the balance is +£0.3bn on the old model and +£8.3bn on the new. Its largest head is DfT subsidy, −£16.5bn, mapped to Microcosm #1329 (the ORR rail-support row held out of calibration) and #1332 (the rail and bus imputation draws), whose shares aren't separated; SDLT, BSP, War Widow's Pension and childcare are also mapped. Income tax (−£1.2bn in 2026-27), the data's UC, DfE and council-tax effects, and every poverty move stay unattributed: no common-baseline counterfactual separates them. An independent review (GPT-6.1 Sol) requested changes to the evidence and wording; they were adopted, and its delta review approved. Small cells are withheld per head: Disabled Students' Allowance and Statutory Paternity Pay (each under ten cases) are suppressed, and every other head and the balance are reported. attribution.json (rebuilt from the run's own cells with the reviewed mappings, no rerun; sha256 e636bc7f…)
Reform responsiveness on the exact cut (the #685 coverage) needs review, accepted for this cut by the Budget coordinator's explicit decision (comment). Finished at 04:23Z on 25 Oct (7,447 s). The qualification's reform_responsiveness gate (data/qualification/uk.json): the pinned metrics tool costs its ten standard reforms, 2025–2031, and each must move its target in the stated direction by at least its minimum in every year. All ten ran, and the tool's responsive check passes for all ten. Nine are published in every year. The tenth, raise the additional rate by 3pp, is withheld in 2027-28 to 2030-31 under the tool's small-cell rule (reform_withheld: a reform's costing is withheld in a year it would come from a small cell). There the gate can't recompute the tool's verdict, so it returns needs_review, not pass. In the published years its income tax change is +£3.9bn to +£5.5bn, against a £0.55bn minimum. Recording which small-cell condition withheld a year is a non-blocking follow-up: policyengine-uk#2654. reform-responsiveness.json
Scorecard replay unavailable: the Microcosm replay fix hasn't landed. Unavailable is not a pass. replay.json
Full test suite The local run was interrupted (its tests stage keeps return code 2). The 6 routing failures are fixed in e02d41b, and the 82 tests the run never reached pass when run alone (logs). The full-suite evidence is this PR's CI on the exact head: the evidence index records it in final-head-ci.json and refuses to index unless every check, including the four Test jobs, has passed at the candidate commit. CI on e02d41b passed all 12 checks (run, 24 Oct). The four Test jobs each collected 4,315 tests, with no failures: 3.11 passed 4,275 and skipped 40; 3.12 and 3.14 passed 4,296 and skipped 19; 3.13 passed 4,301 and skipped 14. In each, the qualification-fallback test passed, and test_uk_budget_reform_contract.py ran 139 tests, all passing. certify log; CI run 40071835088; final-head-ci.json (to come)

The evidence receipts hold aggregates and digests only; per-record logs stay outside Git. Two harness fixes were needed during the run, and neither touches the repository:

  • checked_data in the budget-inputs and reform-contract steps resolved data/old.h5 and data/new.h5 to Hugging Face cache blobs, whose names have no .h5 suffix, and policyengine-uk refused that path.
  • The attribution script resolved them the same way. Each step now keeps the link's own path, and the SHA-256 pin checks are unchanged.

After the first review, three more harness changes, none in the repository: a reform-responsiveness step; an evidence index that requires its receipt and binds the final-head CI; and an atomic write of attribution.json, so an interrupted write can't leave a truncated receipt. After the 2×2's first pass (02:01Z), a fourth: that pass's whole-year suppression hid every money total, because two heads have fewer than ten cases. The rerun suppresses only those two heads, and the first pass's receipts are kept.

Test failures in the certification run, classified

The certification suite ran with four workers (-n 4) for 4 h 05 m. Its progress line reached 98% after about 2 h 10 m and stayed there for about two hours. The UK hub then interrupted pytest (SIGINT, 05:32Z, 24 Oct), as the Budget coordinator had ruled at 04:50Z.

6 failed, 4,211 passed, 14 skipped; 82 never ran. The session collected 4,313 tests.

  • All 82 that never ran are in tests/test_uk_budget_reform_contract.py. Worker gw0 had run that file's first 38 tests in order.
  • After test_the_capital_gains_cases_hold_on_every_in_process_route passed, gw0 reported nothing more.
  • One worker process stayed busy (73–95% CPU when checked at 05:16Z and 05:32Z) while the other three sat idle.
  • No worker dump is visible in the captured log. The controller's dumps show it waiting in xdist. The stall's cause remains unknown.
  • The hub then ran the 82 at e02d41b in one process, in file order and after the capital-gains test, as gw0 would have run them (-p no:xdist -o faulthandler_timeout=300). All 83 passed in 3 m 54 s (05:41–05:45Z; log), so the stall did not reproduce.
  • The full suite runs again in this PR's CI.

All six failures come from the national-only route (--drop-regional-release). This cut's manifest sets regional_release_kind to none, and its region_datasets hold only national. So area_regions_routed_to_local_release() is false, the default registry holds no constituency or local-authority regions, and uk_area_dataset_hint returns an empty string. All six tests assumed the previous bundle's local_area routing:

Test Assertion that failed Why
test_uk_regions.py::…children_are_countries_and_areas {'country': 4} == {'country': 4, 'constituency': 650, 'local_authority': 361} no area regions are routed
test_uk_regions.py::…national_and_area_regions_have_datasets 1 == 1 + 1011 the same
test_uk_regions.py::…countries_and_areas_filter 4 == 4 + 1011 the same
test_uk_regions.py::…has_1016_regions 5 == 5 + 1011 the same
test_release_manifests.py::…area_regions_route_to_local_release None == 'local_area' the manifest has no regional release kind
test_uk_geography_disclosure.py::test_a_missing_area_column_names_the_dataset_that_carries_it 'microcosm_uk_2024_25_dense' in '' no dataset is routed, so the hint is empty

e02d41b makes each test assert the bundle's own routing, as the coordinator ruled, rather than skipping it:

  • No local_area routing:
    • the registry has only the national region and the four countries;
    • constituency/E14001063 and local_authority/N09000011 return None;
    • the manifest's kind is None, its regions are {"national": "microcosm_uk_2024_25.h5"}, and it bundles no dense dataset;
    • the hint is empty, and the error doesn't claim a routed dataset.
  • local_area routing: the earlier assertions stand unchanged.
  • New test: test_a_routed_area_names_its_dataset_in_the_error patches in a routed manifest. It checks that the error still names the area's dataset, whatever the packaged routing.

At e02d41b, the changed tests pass single-process: 75 passed, and 119 across all three files (logs). ruff format and ruff check are clean on them.

No other failures. test_uk_qualification_pipeline.py::test_the_certification_environment_falls_back_to_the_packaged_certificate passed in this run. The log holds one FAILED line for it, from an earlier attempt that started at 21:29Z on 23 Oct. That attempt was stopped before pytest printed its summary, so the line has no traceback. Run on its own, the test's subprocess takes about 14 s of its 60 s budget. Per the coordinator's ruling, it blocks this PR if CI fails it.

Budget-inputs gate: needs review

budget_inputs returns needs_review: the new pair has 13 model inputs that the data doesn't carry and the previous pair didn't flag. None of them is data the earlier cut had and this one lost:

  • None was supplied by the previous data (previous_input_status: in_data false for all 13). The receipt does not reconstruct cuts before that one; its limitations say so.
  • Eleven are new variables in policyengine-uk 2.147.2. They aren't in 2.132.6 at all: defined_benefit_pension_share, directly_held_shares, early_years_school_start_deferred, is_full_year_category_a_director, non_residential_property_secured_debt, other_residential_property_secured_debt, pension_annual_allowance_taxable_lump_sum_death_benefits, pension_contributions_via_salary_sacrifice_pre_2015, pip_dl_points, pip_m_points and unit_and_investment_trusts.
  • Two became inputs. local_bus_uncapped_single_fare and salary_sacrifice_returned_to_income were in 2.132.6, but not as inputs.
  • No critical input is missing or degenerate (unusable_critical is empty). The cut supports 2024–2032, which covers the required years 2025–2030 and the Budget horizon 2026–2031 (supported_years: pass).

Each of the 13 takes its model default on this cut. What that means differs by input (policyengine-uk 2.147.2):

  • local_bus_uncapped_single_fare: a bus fare cap change is priced from the parameter table's three uncapped-fare bands, gov.dft.bus.uncapped_single_fares (person_bus_fare_spending.py:89–98), instead of each person's own fare.
  • pip_dl_points and pip_m_points: the cut stores PIP categories, and the loader copies them into every later year (data/economic_assumptions.py:216–242). pip_category (utils/pip.py:18–72) runs only when no category is supplied, so a reform to the points thresholds has no effect and raises no error. Reforms to PIP rates are unaffected.
  • salary_sacrifice_returned_to_income: a legacy input that defaults to zero. The salary-sacrifice cap works through salary_sacrifice_pension_ni_excess, which doesn't read it.
  • pension_contributions_via_salary_sacrifice_pre_2015: read only by pension_annual_allowance_threshold_income (:29), so it affects only the tapered annual allowance.

Receipts (from the certification tool)

  • UC floor (weighted national totals, scripts/check_uk_uc_floor.py): passed; measured/benchmark ratios: benefit_units=0.925245, spend=0.957813
  • Tests: === 6 failed, 4211 passed, 14 skipped, 5688 warnings in 14695.24s (4:04:55) ====
  • Certified-constant files rewritten: none
How it was made (the tool's commands)
uv sync --frozen --extra uk --extra dev --python 3.13
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python scripts/bundle.py certify-data --country uk --data-producer populace --manifest-uri hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-national-20261023T075039Z-a133f37a/releases/microcosm-uk-2024-25-national/release_manifest.json --drop-regional-release --model-version 2.147.2 --core-version 3.33.3
uv lock --no-config --no-sources --default-index https://pypi.org/simple --python /Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python
uv sync --frozen --extra uk --extra dev --python 3.13
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python scripts/release_lock.py
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python scripts/bundle.py generate --include-tros
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python -c 'import json, sys; from pathlib import Path; from policyengine.provenance.uk_qualification import certified_constant_replacements as r, rewrite_certified_constants as w; old, new = json.loads(sys.argv[1]), json.loads(sys.argv[2]); print(json.dumps(w(Path('"'"'.'"'"'), r(old, new))))' '{"build_id": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "bundle_id": "uk-6.10.1", "certification": {"built_with_model_version": "2.132.6", "certified_by": "policyengine.py bundle certification", "certified_for_model_version": "2.132.6", "compatibility_basis": "built_with_model_package", "core_binding": "bundle_pin", "data_build_id": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71"}, "certified_data_artifact": {"build_id": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "data_package": {"name": "microcosm-data", "version": "0.1.0"}, "dataset": "microcosm_uk_2024_25", "sha256": "810ba980005a8b907cb51464279fa5e0125464b7af9a4871b7f6744c30951ece", "uri": "hf://policyengine/populace-uk-private/microcosm_uk_2024_25.h5@microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71"}, "country_id": "uk", "data_package": {"name": "microcosm-data", "release_manifest_path": "releases/microcosm-uk-2024-25-national/release_manifest.json", "release_manifest_revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "repo_id": "policyengine/populace-uk-private", "repo_type": "dataset", "version": "0.1.0"}, "data_producer": "populace", "datasets": {"calibration_diagnostics": {"path": "releases/microcosm-uk-2024-25-national/calibration_diagnostics.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "sha256": "34978e2bfde3f2641769c9c3a8f60455c59ecae5a50524b359d74388d7edc888"}, "microcosm_uk_2024_25": {"path": "microcosm_uk_2024_25.h5", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "sha256": "810ba980005a8b907cb51464279fa5e0125464b7af9a4871b7f6744c30951ece"}, "microcosm_uk_2024_25_calibration": {"path": "microcosm_uk_2024_25_calibration.npz", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "sha256": "104a8253bf1aebc91147c9bbb40e2603d1025317ba8c6b991383d5e4d91852c2"}, "microcosm_uk_2024_25_dense": {"path": "microcosm_uk_2024_25_dense.h5", "repo_id": "policyengine/populace-uk-private", "repo_type": "dataset", "revision": "microcosm-uk-2024-25-dense-20261019T000432Z-ee5c2dbe", "sha256": "8e10e130d0260e0b52e2e29902f0502ebdff76a14a3e57d2c876bf4cd8db5872"}, "release_certification": {"path": "releases/microcosm-uk-2024-25-national/release_certification.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "sha256": "6a68cc93595fc5a7ad451d75a902523c25659de27014f50431dd5e19af5bd959"}, "release_cut_gates": {"path": "releases/microcosm-uk-2024-25-national/release_cut_gates.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "sha256": "c91c1364d9c3be98493039d4e1dd8c2256881a434c1f124dd2ed0701bfaf7751"}, "score_vs_enhanced_frs": {"path": "releases/microcosm-uk-2024-25-national/score_vs_enhanced_frs.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "sha256": "ba8a675f36e7c836bf5521e5933e21bb689442734b7340e05cabd3e0cc55323d"}, "spine_gates": {"path": "releases/microcosm-uk-2024-25-national/spine_gates.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "sha256": "6dbba976dd1304c07da531382cecb11c9cb69512de0ee65126b71f032a0f37d6"}, "terminal_gates": {"path": "releases/microcosm-uk-2024-25-national/terminal_gates.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "sha256": "e29439d5ce0ea098d32579284ac06f26f086feba2fb8d12d1e88ac9a54840808"}}, "default_dataset": "microcosm_uk_2024_25", "default_dataset_uri": "hf://policyengine/populace-uk-private/microcosm_uk_2024_25.h5@microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71", "model_package": {"name": "policyengine-uk", "sha256": "cc54025f2a3bc54be4f8d024d1e6f11ed29fed75ca513b214f2d2963c5bf4354", "version": "2.132.6", "wheel_url": "https://files.pythonhosted.org/packages/11/f2/c003cd7a95cb499737a5b8af0d619ba4e759c8bc7e55a2bb5c7efa46abd1/policyengine_uk-2.132.6-py3-none-any.whl"}, "policyengine_version": "6.10.1", "region_datasets": {"constituency": {"path_template": "microcosm_uk_2024_25_dense.h5"}, "local_authority": {"path_template": "microcosm_uk_2024_25_dense.h5"}, "national": {"path_template": "microcosm_uk_2024_25.h5"}}, "regional_release_kind": "local_area", "regional_release_manifest_uri": "https://huggingface.co/datasets/policyengine/populace-uk-private/resolve/microcosm-uk-2024-25-dense-20261019T000432Z-ee5c2dbe/releases/microcosm-uk-2024-25-dense/release_manifest.json", "regional_source_manifest_uri": "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-dense-20261019T000432Z-ee5c2dbe/releases/microcosm-uk-2024-25-dense/release_manifest.json", "release_manifest_uri": "https://huggingface.co/datasets/policyengine/populace-uk-private/resolve/microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71/releases/microcosm-uk-2024-25-national/release_manifest.json", "schema_version": 1, "source_manifest_uri": "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71/releases/microcosm-uk-2024-25-national/release_manifest.json", "version": "microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71"}' '{"build_id": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "bundle_id": "uk-6.10.1", "certification": {"built_with_model_version": "2.147.2", "certified_by": "policyengine.py bundle certification", "certified_for_core_version": "3.33.3", "certified_for_model_version": "2.147.2", "compatibility_basis": "built_with_model_package", "core_binding": "certified", "data_build_id": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a"}, "certified_data_artifact": {"build_id": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "data_package": {"name": "microcosm-data", "version": "0.1.0"}, "dataset": "microcosm_uk_2024_25", "sha256": "64f83900ca3a1242fb017585234844ce89f743d32e41cd2b6558057777f914aa", "uri": "hf://policyengine/populace-uk-private/microcosm_uk_2024_25.h5@microcosm-uk-2024-25-national-20261023T075039Z-a133f37a"}, "country_id": "uk", "data_package": {"name": "microcosm-data", "release_manifest_path": "releases/microcosm-uk-2024-25-national/release_manifest.json", "release_manifest_revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "repo_id": "policyengine/populace-uk-private", "repo_type": "dataset", "version": "0.1.0"}, "data_producer": "populace", "datasets": {"calibration_diagnostics": {"path": "releases/microcosm-uk-2024-25-national/calibration_diagnostics.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "sha256": "0c975ad04a2d1c0207c3be480d68f645c3a1b24f786c3eef76d6e984dfea0637"}, "microcosm_uk_2024_25": {"path": "microcosm_uk_2024_25.h5", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "sha256": "64f83900ca3a1242fb017585234844ce89f743d32e41cd2b6558057777f914aa"}, "microcosm_uk_2024_25_calibration": {"path": "microcosm_uk_2024_25_calibration.npz", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "sha256": "074872730ad5678cdb4e483d0141ece912be65d6bc032d01bf756f2c3868f2a7"}, "release_certification": {"path": "releases/microcosm-uk-2024-25-national/release_certification.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "sha256": "633661c40492645236c0d286576c6206d1fe87609ac7ce90234f625397b9260c"}, "release_cut_gates": {"path": "releases/microcosm-uk-2024-25-national/release_cut_gates.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "sha256": "19f9b8bde4cafcfd179607541c5c8fa4247c97858a4ace7a1838bbacd223bc17"}, "score_vs_enhanced_frs": {"path": "releases/microcosm-uk-2024-25-national/score_vs_enhanced_frs.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "sha256": "cb4dcde88bf016e0646cdc46a9ab9adf9973c77b16a0cb9e1a25987d57b212ab"}, "spine_gates": {"path": "releases/microcosm-uk-2024-25-national/spine_gates.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "sha256": "7959f2a19b6535e67f8d9cb2c37ee68f027892ca10d9d8422be18a8261798f34"}, "terminal_gates": {"path": "releases/microcosm-uk-2024-25-national/terminal_gates.json", "repo_id": "policyengine/populace-uk-private", "revision": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "sha256": "dac50ba94464ca4e2b494fd8ffcd04e47da6dd6e49e9f53bdb7384de7983097f"}}, "default_dataset": "microcosm_uk_2024_25", "default_dataset_uri": "hf://policyengine/populace-uk-private/microcosm_uk_2024_25.h5@microcosm-uk-2024-25-national-20261023T075039Z-a133f37a", "model_package": {"name": "policyengine-uk", "sha256": "4b48ab1008b00610abc38192927ac9d4bb22037b6bbd9578926c2899e639f972", "version": "2.147.2", "wheel_url": "https://files.pythonhosted.org/packages/96/dd/54ddc9226756877e630a3f9fcda4dbf2ae142635e96a564cab3a062ced62/policyengine_uk-2.147.2-py3-none-any.whl"}, "policyengine_version": "6.10.1", "region_datasets": {"national": {"path_template": "microcosm_uk_2024_25.h5"}}, "release_manifest_uri": "https://huggingface.co/datasets/policyengine/populace-uk-private/resolve/microcosm-uk-2024-25-national-20261023T075039Z-a133f37a/releases/microcosm-uk-2024-25-national/release_manifest.json", "schema_version": 1, "source_manifest_uri": "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-national-20261023T075039Z-a133f37a/releases/microcosm-uk-2024-25-national/release_manifest.json", "version": "microcosm-uk-2024-25-national-20261023T075039Z-a133f37a"}'
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python -c 'import faulthandler, os, sys
diagnostic_stream = os.fdopen(os.dup(2), '"'"'w'"'"')
faulthandler.dump_traceback_later(300.0, repeat=True, file=diagnostic_stream)
try:
    import pytest
    raise SystemExit(pytest.main(sys.argv[1:]))
finally:
    faulthandler.cancel_dump_traceback_later()
    diagnostic_stream.close()
' tests/test_household_calculator_snapshot.py -q
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python scripts/bundle.py check --published-spm --include-tros --strict-tros
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python scripts/check_uk_uc_floor.py --dataset microcosm_uk_2024_25 --year 2025 --output /Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification.uc_floor.json
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python -c 'import xdist'
uv pip install --python /Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python pytest-xdist
/Users/maxghenis/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification/.venv/bin/python -c 'import faulthandler, os, sys
diagnostic_stream = os.fdopen(os.dup(2), '"'"'w'"'"')
faulthandler.dump_traceback_later(300.0, repeat=True, file=diagnostic_stream)
try:
    import pytest
    raise SystemExit(pytest.main(sys.argv[1:]))
finally:
    faulthandler.cancel_dump_traceback_later()
    diagnostic_stream.close()
' -n 4 -vv --durations=20 -o faulthandler_timeout=300 -p no:cacheprovider

Before this leaves draft

  • The 2×2 attribution and a review of its result. It passed at 02:19Z on 25 Oct. The reviewed mappings attribute 144 of 1,092 moves and label the rest unattributed, and the independent review approved them after its changes were adopted.
  • Reform responsiveness on the exact cut (reform-responsiveness.json), reviewed, with any non-pass resolved by an explicit decision. It returns needs_review (the additional-rate reform's withheld years, above); the Budget coordinator accepted it for this cut (comment). Follow-up: policyengine-uk#2654.
  • The reform contract's result in the table above, reviewed, with any non-pass resolved: it passes, and its 34 pending checks all hold at the #2396 default this candidate pins (24 Oct; the independent review covers it with the other receipts).
  • CI green on this head, including the qualification-fallback test and the 82 reform-contract tests the local run never reached (run 40071835088, 24 Oct).
  • The evidence index over the exact pair: it needs attribution.json, reform-responsiveness.json and green final-head CI (final-head-ci.json), and it is reviewed. It was written at 05:03Z on 25 Oct (evidence-index.json: 35 files; status "evidence collected; review required; replay unavailable"). It indexes the mapped attribution.json (e636bc7f…), the reform-responsiveness receipt and Certify Microcosm UK national cut a133f37a for the Autumn Budget #715's CI at e02d41b (run 40071835088). Its gates: budget_inputs needs_review (accepted), supported_years pass, reform_contract pass, reform_responsiveness needs_review. The independent review of the index is running.
  • The budget-inputs needs_review: accepted for this cut, with the dashboard data-limit condition (coordinator decision). The receipt stays needs_review.
  • The coordinator's condition, before adoption: all 13 inputs registered as dashboard data limits, or a link to the work that registers them, with adoption held until then.
  • Wrapper qualification (from the tool): exact-pair receipts for each required gate; scorecard replay stays unavailable under the exception. This script waives no gate.
  • One writer: the UK hub and María know this PR exists.

🤖 Generated with Claude Code

MaxGhenis and others added 9 commits October 23, 2026 13:18
Fleet ops lets budget-certify start once Route A's reservation releases
(expected 19:30-20:30Z on 23 Oct), not at a fixed 00:30Z. The
reservation check in guard() is now the only start gate; load and disk
checks are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#704 dropped its job hand-off note (463c3a9). This merge takes that, and
stops tracking the preparation tooling (.job/, pr-body.md, run-heavy.sh)
so the certification commit built on this preparation carries only the
data-certification skill's re-cut exception and the generated files. The
tooling stays in the workspace, untracked; run-heavy.sh reads it from
disk. A durable copy is in ~/reviews/uk-hub/jobs/budget-certify-a133/prep/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…9Z-a133f37a

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This cut is national-only (--drop-regional-release), so the manifest
routes no constituency or local-authority regions. The region, manifest
and geography-disclosure tests now assert that routing: with no
local_area release the area regions are absent and their lookups return
nothing; with one, the earlier assertions stand. A new test checks that
a routed area's error still names its dataset, whatever the packaged
routing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

MaxGhenis commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor Author

Coordinator decision on the budget-inputs gate (same-line re-cut exception, under d1682 and d1344). This is the explicit decision the exception requires for the 13 newly missing inputs.

Accepted for this cut. None is a regression: no Microcosm cut has ever carried any of them. Eleven are new variables in policyengine-uk 2.147.2, and two became inputs there. No critical input is unusable, and the supported years pass.

The condition: they must not reach published figures silently. A reform reading one of these inputs gets its default value, not data. On the dashboard, any measure whose figures depend on one of the 13 is therefore held as a data limit until a cut carries the input. The hold follows the measure's own variables, as the existing data-limit checks do. Before the release is adopted, the dashboard registers the 13 as a data limit. The ones most likely to matter this Budget:

  • pip_dl_points and pip_m_points: the cut stores PIP categories, and the loader copies the data year into every later year (extend_single_year_dataset). So pip_category, which runs only when no category is supplied, never runs. A change to gov.dwp.pip.*.points_thresholds therefore costs nothing, silently. Supplying points wouldn't fix that while the categories still load. Changes to PIP rates are unaffected.
  • local_bus_uncapped_single_fare: a change to the bus fare cap is priced from the parameter table's three uncapped-fare bands (gov.dft.bus.uncapped_single_fares), not from each person's own fare (person_bus_fare_spending.py). G3 can release it, with that stated.
  • salary_sacrifice_returned_to_income and pension_contributions_via_salary_sacrifice_pre_2015: the salary sacrifice cap and anything near it.
  • non_residential_property_secured_debt and other_residential_property_secured_debt: landlord finance-cost relief.
  • directly_held_shares and unit_and_investment_trusts: wealth and capital measures.
  • defined_benefit_pension_share, pension_annual_allowance_taxable_lump_sum_death_benefits, early_years_school_start_deferred and is_full_year_category_a_director: the same rule applies.

Each held measure can be released by a named person at the Treasury check (G3), with the default stated in its public text, or by a later cut that carries the input.

This doesn't change the rest of the evidence package. The 2×2 is still required before merge.

Corrected 24 Oct ~07:10Z: the PIP and bus lines. The earlier bus line was wrong: a cap change does produce figures, from the fare table. The PIP line understated the problem. Both now follow the UK hub's reading of the policyengine-uk 2.147.2 code, which I checked.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

UK hub: the local reruns behind this PR's test claims, at e02d41b. The review asked for their output. Each ran single-process (-p no:xdist).

  • The changed routing tests: 75 passed. These are tests/test_uk_regions.py, tests/test_uk_geography_disclosure.py and test_release_manifests.py::…area_regions_follow_the_regional_release, run 24 Oct, 07:11Z. All three files together: 119 passed.
  • The 82 reform-contract tests the certification run never reached: 83 passed in 3 m 54 s. The 83 include the capital-gains test first, in gw0's order, with -o faulthandler_timeout=300. Run 24 Oct, 05:41–05:45Z.

The final-head CI still has to pass. It is the full-suite evidence the evidence index requires.

routing-changed-tests.log
2026-10-24T07:11:06Z
head e02d41bceab57756373b2dc3bcf78635b856ec81
cmd: .venv/bin/python -m pytest -p no:xdist -p no:cacheprovider -v tests/test_uk_regions.py tests/test_uk_geography_disclosure.py tests/test_release_manifests.py::TestReleaseManifests::test__given_uk_manifest__then_area_regions_follow_the_regional_release
============================= test session starts ==============================
platform darwin -- Python 3.13.9, pytest-9.1.1, pluggy-1.6.0 -- ./.venv/bin/python
hypothesis profile 'default'
rootdir: ~/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification
configfile: pyproject.toml
plugins: cov-7.1.0, xdist-3.8.0, asyncio-1.4.0, hypothesis-6.168.1
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 75 items
tests/test_uk_regions.py::TestUKCountries::test__given_uk_countries__then_has_four_entries PASSED [  1%]
tests/test_uk_regions.py::TestUKCountries::test__given_uk_countries__then_all_countries_present PASSED [  2%]
tests/test_uk_regions.py::TestUKCountries::test__given_uk_countries__then_labels_capitalized PASSED [  4%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_country_id_is_uk PASSED [  5%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_has_national_region PASSED [  6%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_has_four_country_regions PASSED [  8%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_england_region__then_filters_from_national PASSED [  9%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_country_regions__then_filter_stored_regions PASSED [ 10%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_scotland_region__then_filters_from_national PASSED [ 12%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_wales_region__then_filters_from_national PASSED [ 13%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_northern_ireland_region__then_filters_from_national PASSED [ 14%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_national__then_children_are_countries_and_routed_areas PASSED [ 16%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_national_and_area_regions_have_datasets PASSED [ 17%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_countries_and_areas_filter PASSED [ 18%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_default_registry__then_counts_the_routed_regions PASSED [ 20%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_default_registry__then_area_lookups_follow_the_routing PASSED [ 21%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_builder_without_optional_regions__then_returns_5_regions PASSED [ 22%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_builder__then_accepts_include_constituencies_flag PASSED [ 24%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_builder__then_accepts_include_local_authorities_flag PASSED [ 25%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_constituencies_included__then_filters_on_dataset_geography PASSED [ 26%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_local_authorities_included__then_filters_on_dataset_geography PASSED [ 28%]
tests/test_uk_regions.py::TestUKAreaRegionRouting::test__given_unrouted_bundle__then_area_regions_filter_the_national_dataset PASSED [ 29%]
tests/test_uk_regions.py::TestUKAreaRegionRouting::test__given_national_first_bundle__then_area_regions_load_the_efrs PASSED [ 30%]
tests/test_uk_regions.py::TestUKAreaRegionRouting::test__given_local_area_bundle__then_default_registry_adds_area_regions PASSED [ 32%]
tests/test_uk_regions.py::TestUKAreaRegionRouting::test__given_local_area_bundle__then_explicit_exclusion_wins PASSED [ 33%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[str-england] PASSED [ 34%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[str-scotland] PASSED [ 36%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[str-wales] PASSED [ 37%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[str-northern_ireland] PASSED [ 38%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[bytes-england] PASSED [ 40%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[bytes-scotland] PASSED [ 41%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[bytes-wales] PASSED [ 42%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[bytes-northern_ireland] PASSED [ 44%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-england] PASSED [ 45%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-scotland] PASSED [ 46%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-wales] PASSED [ 48%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-northern_ireland] PASSED [ 49%]
tests/test_uk_regions.py::test_england_scoping_allows_unrepresented_english_regions PASSED [ 50%]
tests/test_uk_regions.py::test_country_scoping_json_round_trip[england] PASSED [ 52%]
tests/test_uk_regions.py::test_country_scoping_json_round_trip[scotland] PASSED [ 53%]
tests/test_uk_regions.py::test_country_scoping_json_round_trip[wales] PASSED [ 54%]
tests/test_uk_regions.py::test_country_scoping_json_round_trip[northern_ireland] PASSED [ 56%]
tests/test_uk_regions.py::test_country_filters_can_be_combined_without_duplicate_households PASSED [ 57%]
tests/test_uk_geography_disclosure.py::test_regions_are_published_with_intervals_and_pass_the_validator PASSED [ 58%]
tests/test_uk_geography_disclosure.py::test_public_rows_round_trip_through_json PASSED [ 60%]
tests/test_uk_geography_disclosure.py::test_the_region_table_maps_every_stored_region_to_its_gss_code PASSED [ 61%]
tests/test_uk_geography_disclosure.py::test_regions_are_keyed_on_gss_codes_whatever_the_dataset_stores PASSED [ 62%]
tests/test_uk_geography_disclosure.py::test_region_codes_are_preferred_over_the_stored_region PASSED [ 64%]
tests/test_uk_geography_disclosure.py::test_an_unknown_region_is_in_the_nation_but_in_no_region PASSED [ 65%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region-London] PASSED [ 66%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region-ENGLAND] PASSED [ 68%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region-Eastern] PASSED [ 69%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region_code_oa-E12000010] PASSED [ 70%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region_code_oa-E92000001] PASSED [ 72%]
tests/test_uk_geography_disclosure.py::test_a_regional_release_is_a_prior_for_seats_from_another_dataset PASSED [ 73%]
tests/test_uk_geography_disclosure.py::test_a_prior_cell_at_a_level_that_is_not_a_uk_level_raises PASSED [ 74%]
tests/test_uk_geography_disclosure.py::test_an_empty_region_code_column_falls_back_to_the_stored_region[] PASSED [ 76%]
tests/test_uk_geography_disclosure.py::test_an_empty_region_code_column_falls_back_to_the_stored_region[  ] PASSED [ 77%]
tests/test_uk_geography_disclosure.py::test_an_empty_region_code_column_falls_back_to_the_stored_region[None] PASSED [ 78%]
tests/test_uk_geography_disclosure.py::test_a_level_whose_column_gives_no_household_an_area_raises PASSED [ 80%]
tests/test_uk_geography_disclosure.py::test_a_dataset_without_a_source_id_is_refused_unless_ids_are_passed PASSED [ 81%]
tests/test_uk_geography_disclosure.py::test_passed_ids_count_clones_once PASSED [ 82%]
tests/test_uk_geography_disclosure.py::test_a_thin_seat_blocks_publication_until_it_is_resolved PASSED [ 84%]
tests/test_uk_geography_disclosure.py::test_frames_must_hold_the_same_households_in_the_same_order PASSED [ 85%]
tests/test_uk_geography_disclosure.py::test_an_unknown_or_missing_level_raises PASSED [ 86%]
tests/test_uk_geography_disclosure.py::test_seats_and_local_authorities_together_need_an_acknowledgement PASSED [ 88%]
tests/test_uk_geography_disclosure.py::test_noise_needs_a_key_and_the_key_never_reaches_the_results PASSED [ 89%]
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes PASSED [ 90%]
tests/test_uk_geography_disclosure.py::test_the_audit_goes_to_a_private_directory_outside_repositories PASSED [ 92%]
tests/test_uk_geography_disclosure.py::test_the_release_is_available_only_after_a_run PASSED [ 93%]
tests/test_uk_geography_disclosure.py::test_the_adapter_is_exported_from_outputs PASSED [ 94%]
tests/test_uk_geography_disclosure.py::test_a_missing_area_column_names_the_dataset_that_carries_it PASSED [ 96%]
tests/test_uk_geography_disclosure.py::test_a_routed_area_names_its_dataset_in_the_error PASSED [ 97%]
tests/test_uk_geography_disclosure.py::test_no_hint_when_nothing_routes_the_area PASSED [ 98%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_uk_manifest__then_area_regions_follow_the_regional_release PASSED [100%]
=============================== warnings summary ===============================
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-england]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-scotland]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-wales]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-northern_ireland]
  ./.venv/lib/python3.13/site-packages/microdf/microseries.py:808: Pandas4Warning: The copy keyword is deprecated and will be removed in a future version. Copy-on-Write is active in pandas since 3.0 which utilizes a lazy copy mechanism that defers copies until necessary. Use .copy() to make an eager copy if necessary.
    converted_series = super().astype(dtype, copy=copy, errors=errors)
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
  ./src/policyengine/disclosure/areas.py:205: RuntimeWarning: divide by zero encountered in matmul
    return np.asarray(self._matrix @ scaled, dtype=float)
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
  ./src/policyengine/disclosure/areas.py:205: RuntimeWarning: overflow encountered in matmul
    return np.asarray(self._matrix @ scaled, dtype=float)
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
  ./src/policyengine/disclosure/areas.py:205: RuntimeWarning: invalid value encountered in matmul
    return np.asarray(self._matrix @ scaled, dtype=float)
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:102
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:102: PytestWarning: (rm_rf) unknown function <built-in function scandir> when removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d86893ad-2067-4648-abf3-4e0f4b6635e0/test_locked_file_untouched0/root:
  <class 'PermissionError'>: [Errno 1] Operation not permitted: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d86893ad-2067-4648-abf3-4e0f4b6635e0/test_locked_file_untouched0/root'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d86893ad-2067-4648-abf3-4e0f4b6635e0/test_locked_file_untouched0/root
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d86893ad-2067-4648-abf3-4e0f4b6635e0/test_locked_file_untouched0/root'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d86893ad-2067-4648-abf3-4e0f4b6635e0/test_locked_file_untouched0
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d86893ad-2067-4648-abf3-4e0f4b6635e0/test_locked_file_untouched0'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d86893ad-2067-4648-abf3-4e0f4b6635e0
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d86893ad-2067-4648-abf3-4e0f4b6635e0'
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================= 75 passed, 17 warnings in 1.91s ========================
routing-three-files.log
2026-10-24T07:10:24Z
head e02d41bceab57756373b2dc3bcf78635b856ec81
cmd: .venv/bin/python -m pytest -p no:xdist -p no:cacheprovider tests/test_uk_regions.py tests/test_uk_geography_disclosure.py tests/test_release_manifests.py
============================= test session starts ==============================
platform darwin -- Python 3.13.9, pytest-9.1.1, pluggy-1.6.0 -- ./.venv/bin/python
hypothesis profile 'default'
rootdir: ~/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification
configfile: pyproject.toml
plugins: cov-7.1.0, xdist-3.8.0, asyncio-1.4.0, hypothesis-6.168.1
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 119 items
tests/test_uk_regions.py::TestUKCountries::test__given_uk_countries__then_has_four_entries PASSED [  0%]
tests/test_uk_regions.py::TestUKCountries::test__given_uk_countries__then_all_countries_present PASSED [  1%]
tests/test_uk_regions.py::TestUKCountries::test__given_uk_countries__then_labels_capitalized PASSED [  2%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_country_id_is_uk PASSED [  3%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_has_national_region PASSED [  4%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_has_four_country_regions PASSED [  5%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_england_region__then_filters_from_national PASSED [  5%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_country_regions__then_filter_stored_regions PASSED [  6%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_scotland_region__then_filters_from_national PASSED [  7%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_wales_region__then_filters_from_national PASSED [  8%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_northern_ireland_region__then_filters_from_national PASSED [  9%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_national__then_children_are_countries_and_routed_areas PASSED [ 10%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_national_and_area_regions_have_datasets PASSED [ 10%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_uk_registry__then_countries_and_areas_filter PASSED [ 11%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_default_registry__then_counts_the_routed_regions PASSED [ 12%]
tests/test_uk_regions.py::TestUKRegionRegistry::test__given_default_registry__then_area_lookups_follow_the_routing PASSED [ 13%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_builder_without_optional_regions__then_returns_5_regions PASSED [ 14%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_builder__then_accepts_include_constituencies_flag PASSED [ 15%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_builder__then_accepts_include_local_authorities_flag PASSED [ 15%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_constituencies_included__then_filters_on_dataset_geography PASSED [ 16%]
tests/test_uk_regions.py::TestUKRegionRegistryBuilder::test__given_local_authorities_included__then_filters_on_dataset_geography PASSED [ 17%]
tests/test_uk_regions.py::TestUKAreaRegionRouting::test__given_unrouted_bundle__then_area_regions_filter_the_national_dataset PASSED [ 18%]
tests/test_uk_regions.py::TestUKAreaRegionRouting::test__given_national_first_bundle__then_area_regions_load_the_efrs PASSED [ 19%]
tests/test_uk_regions.py::TestUKAreaRegionRouting::test__given_local_area_bundle__then_default_registry_adds_area_regions PASSED [ 20%]
tests/test_uk_regions.py::TestUKAreaRegionRouting::test__given_local_area_bundle__then_explicit_exclusion_wins PASSED [ 21%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[str-england] PASSED [ 21%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[str-scotland] PASSED [ 22%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[str-wales] PASSED [ 23%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[str-northern_ireland] PASSED [ 24%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[bytes-england] PASSED [ 25%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[bytes-scotland] PASSED [ 26%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[bytes-wales] PASSED [ 26%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[bytes-northern_ireland] PASSED [ 27%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-england] PASSED [ 28%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-scotland] PASSED [ 29%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-wales] PASSED [ 30%]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-northern_ireland] PASSED [ 31%]
tests/test_uk_regions.py::test_england_scoping_allows_unrepresented_english_regions PASSED [ 31%]
tests/test_uk_regions.py::test_country_scoping_json_round_trip[england] PASSED [ 32%]
tests/test_uk_regions.py::test_country_scoping_json_round_trip[scotland] PASSED [ 33%]
tests/test_uk_regions.py::test_country_scoping_json_round_trip[wales] PASSED [ 34%]
tests/test_uk_regions.py::test_country_scoping_json_round_trip[northern_ireland] PASSED [ 35%]
tests/test_uk_regions.py::test_country_filters_can_be_combined_without_duplicate_households PASSED [ 36%]
tests/test_uk_geography_disclosure.py::test_regions_are_published_with_intervals_and_pass_the_validator PASSED [ 36%]
tests/test_uk_geography_disclosure.py::test_public_rows_round_trip_through_json PASSED [ 37%]
tests/test_uk_geography_disclosure.py::test_the_region_table_maps_every_stored_region_to_its_gss_code PASSED [ 38%]
tests/test_uk_geography_disclosure.py::test_regions_are_keyed_on_gss_codes_whatever_the_dataset_stores PASSED [ 39%]
tests/test_uk_geography_disclosure.py::test_region_codes_are_preferred_over_the_stored_region PASSED [ 40%]
tests/test_uk_geography_disclosure.py::test_an_unknown_region_is_in_the_nation_but_in_no_region PASSED [ 41%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region-London] PASSED [ 42%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region-ENGLAND] PASSED [ 42%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region-Eastern] PASSED [ 43%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region_code_oa-E12000010] PASSED [ 44%]
tests/test_uk_geography_disclosure.py::test_a_value_that_names_no_region_raises[region_code_oa-E92000001] PASSED [ 45%]
tests/test_uk_geography_disclosure.py::test_a_regional_release_is_a_prior_for_seats_from_another_dataset PASSED [ 46%]
tests/test_uk_geography_disclosure.py::test_a_prior_cell_at_a_level_that_is_not_a_uk_level_raises PASSED [ 47%]
tests/test_uk_geography_disclosure.py::test_an_empty_region_code_column_falls_back_to_the_stored_region[] PASSED [ 47%]
tests/test_uk_geography_disclosure.py::test_an_empty_region_code_column_falls_back_to_the_stored_region[  ] PASSED [ 48%]
tests/test_uk_geography_disclosure.py::test_an_empty_region_code_column_falls_back_to_the_stored_region[None] PASSED [ 49%]
tests/test_uk_geography_disclosure.py::test_a_level_whose_column_gives_no_household_an_area_raises PASSED [ 50%]
tests/test_uk_geography_disclosure.py::test_a_dataset_without_a_source_id_is_refused_unless_ids_are_passed PASSED [ 51%]
tests/test_uk_geography_disclosure.py::test_passed_ids_count_clones_once PASSED [ 52%]
tests/test_uk_geography_disclosure.py::test_a_thin_seat_blocks_publication_until_it_is_resolved PASSED [ 52%]
tests/test_uk_geography_disclosure.py::test_frames_must_hold_the_same_households_in_the_same_order PASSED [ 53%]
tests/test_uk_geography_disclosure.py::test_an_unknown_or_missing_level_raises PASSED [ 54%]
tests/test_uk_geography_disclosure.py::test_seats_and_local_authorities_together_need_an_acknowledgement PASSED [ 55%]
tests/test_uk_geography_disclosure.py::test_noise_needs_a_key_and_the_key_never_reaches_the_results PASSED [ 56%]
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes PASSED [ 57%]
tests/test_uk_geography_disclosure.py::test_the_audit_goes_to_a_private_directory_outside_repositories PASSED [ 57%]
tests/test_uk_geography_disclosure.py::test_the_release_is_available_only_after_a_run PASSED [ 58%]
tests/test_uk_geography_disclosure.py::test_the_adapter_is_exported_from_outputs PASSED [ 59%]
tests/test_uk_geography_disclosure.py::test_a_missing_area_column_names_the_dataset_that_carries_it PASSED [ 60%]
tests/test_uk_geography_disclosure.py::test_a_routed_area_names_its_dataset_in_the_error PASSED [ 61%]
tests/test_uk_geography_disclosure.py::test_no_hint_when_nothing_routes_the_area PASSED [ 62%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_us_manifest__then_has_pinned_model_and_data_packages PASSED [ 63%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_uk_manifest__then_has_pinned_model_and_data_packages PASSED [ 63%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_us_dataset_name__then_resolves_to_versioned_hf_url PASSED [ 64%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_dataset_explicit_revision__then_resolves_to_that_revision PASSED [ 65%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_uk_dataset_name__then_resolves_to_versioned_hf_url PASSED [ 66%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_uk_manifest__then_area_regions_follow_the_regional_release PASSED [ 67%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_uk_legacy_dataset_names__then_resolves_bundled_aliases PASSED [ 68%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_explicit_url__then_resolution_is_noop PASSED [ 68%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_existing_local_path__then_resolution_is_noop PASSED [ 69%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_default_dataset__then_prefers_certified_data_artifact_uri PASSED [ 70%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_no_dataset__then_managed_resolution_uses_certified_default PASSED [ 71%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_local_area_dataset__then_national_default_is_unchanged PASSED [ 72%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_local_area_name__then_resolves_to_its_immutable_tag PASSED [ 73%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_local_area_dataset__then_registered_with_sha_but_not_default PASSED [ 73%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_us_manifest__then_local_regions_resolve_to_acs_local PASSED [ 74%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_us_ensure_datasets_without_dataset__then_uses_certified_default PASSED [ 75%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_explicit_uri__then_managed_resolution_requires_opt_in PASSED [ 76%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_local_path__then_managed_resolution_requires_opt_in PASSED [ 77%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_local_file_named_like_logical_dataset__then_manifest_wins PASSED [ 78%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_versioned_dataset_url__then_logical_name_drops_version PASSED [ 78%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_country__then_can_fetch_data_release_manifest PASSED [ 79%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_explicit_manifest_revision__then_builds_manifest_url PASSED [ 80%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_release_manifest_artifact_uses_version_tag__then_rewrites_to_commit PASSED [ 81%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_missing_data_release_manifest__then_fetch_raises_unavailable PASSED [ 82%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_range_specifier__then_certification_accepts_compatible_version PASSED [ 83%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_matching_fingerprint__then_certification_allows_reuse PASSED [ 84%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_release_with_cut_tag__then_cut_tag_is_the_data_build_id PASSED [ 84%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_private_manifest_unavailable__then_bundled_certification_is_used PASSED [ 85%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_manifest_request_timeout__then_bundled_certification_is_used PASSED [ 86%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_private_manifest_unavailable_and_fingerprint_mismatch__then_fails PASSED [ 87%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_legacy_compatible_certification__then_offline_fingerprint_mismatch_is_allowed PASSED [ 88%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_manifest_fetch_failure_and_version_mismatch__then_fallback_is_unverified PASSED [ 89%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_offline_hf__then_us_import_uses_bundled_certification PASSED [ 89%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_mismatched_version_and_fingerprint__then_certification_fails PASSED [ 90%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_manifest_certification__then_release_bundle_exposes_it PASSED [ 91%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_runtime_certification__then_release_bundle_prefers_runtime_value PASSED [ 92%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_us_managed_microsimulation__then_passes_certified_dataset_and_bundle PASSED [ 93%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_us_unmanaged_dataset_uri__then_source_is_not_rewritten PASSED [ 94%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_us_unmanaged_local_path__then_dataset_reaches_microsimulation PASSED [ 94%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_uk_managed_dataset_name__then_resolves_within_bundle PASSED [ 95%]
tests/test_release_manifests.py::TestReleaseManifests::test__given_uk_unmanaged_dataset_uri__then_source_is_not_rewritten PASSED [ 96%]
tests/test_release_manifests.py::TestDatasetOverlays::test__given_recertified_release__then_overlay_survives PASSED [ 97%]
tests/test_release_manifests.py::TestDatasetOverlays::test__given_no_overlays__then_payload_is_returned_unchanged PASSED [ 98%]
tests/test_release_manifests.py::TestDatasetOverlays::test__given_overlay_shadowing_default__then_raises PASSED [ 99%]
tests/test_release_manifests.py::TestDatasetOverlays::test__given_overlay_colliding_with_certified_dataset__then_raises PASSED [100%]
=============================== warnings summary ===============================
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-england]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-scotland]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-wales]
tests/test_uk_regions.py::test_country_scoping_uses_raw_regions_and_preserves_entities[category-northern_ireland]
  ./.venv/lib/python3.13/site-packages/microdf/microseries.py:808: Pandas4Warning: The copy keyword is deprecated and will be removed in a future version. Copy-on-Write is active in pandas since 3.0 which utilizes a lazy copy mechanism that defers copies until necessary. Use .copy() to make an eager copy if necessary.
    converted_series = super().astype(dtype, copy=copy, errors=errors)
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
  ./src/policyengine/disclosure/areas.py:205: RuntimeWarning: divide by zero encountered in matmul
    return np.asarray(self._matrix @ scaled, dtype=float)
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
  ./src/policyengine/disclosure/areas.py:205: RuntimeWarning: overflow encountered in matmul
    return np.asarray(self._matrix @ scaled, dtype=float)
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
tests/test_uk_geography_disclosure.py::test_a_weight_matrix_can_replace_the_area_codes
  ./src/policyengine/disclosure/areas.py:205: RuntimeWarning: invalid value encountered in matmul
    return np.asarray(self._matrix @ scaled, dtype=float)
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:102
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:102: PytestWarning: (rm_rf) unknown function <built-in function scandir> when removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-84fe8bbd-9bf0-4dfa-94f9-15e3eb8b7a0f/test_locked_file_untouched0/root:
  <class 'PermissionError'>: [Errno 1] Operation not permitted: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-84fe8bbd-9bf0-4dfa-94f9-15e3eb8b7a0f/test_locked_file_untouched0/root'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-84fe8bbd-9bf0-4dfa-94f9-15e3eb8b7a0f/test_locked_file_untouched0/root
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-84fe8bbd-9bf0-4dfa-94f9-15e3eb8b7a0f/test_locked_file_untouched0/root'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-84fe8bbd-9bf0-4dfa-94f9-15e3eb8b7a0f/test_locked_file_untouched0
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-84fe8bbd-9bf0-4dfa-94f9-15e3eb8b7a0f/test_locked_file_untouched0'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-84fe8bbd-9bf0-4dfa-94f9-15e3eb8b7a0f
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-84fe8bbd-9bf0-4dfa-94f9-15e3eb8b7a0f'
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================== 119 passed, 17 warnings in 12.94s =======================
reform-contract-83.log
============================= test session starts ==============================
platform darwin -- Python 3.13.9, pytest-9.1.1, pluggy-1.6.0 -- ./.venv/bin/python
hypothesis profile 'default'
rootdir: ~/.subfleet/worktrees/20261023-110044-budget-certify-a133-prep/.job/certification
configfile: pyproject.toml
plugins: cov-7.1.0, xdist-3.8.0, asyncio-1.4.0, hypothesis-6.168.1
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 83 items
tests/test_uk_budget_reform_contract.py::test_the_capital_gains_cases_hold_on_every_in_process_route PASSED [  1%]
tests/test_uk_budget_reform_contract.py::test_the_default_response_is_reported_without_gating PASSED [  2%]
tests/test_uk_budget_reform_contract.py::test_a_parameter_modifier_writes_the_annualised_parameters PASSED [  3%]
tests/test_uk_budget_reform_contract.py::test_modifier_cases_differ_only_in_order_and_skip_the_household_route PASSED [  4%]
tests/test_uk_budget_reform_contract.py::test_population_errors_emit_only_the_declared_diagnostic_after_progress[early] PASSED [  6%]
tests/test_uk_budget_reform_contract.py::test_population_errors_emit_only_the_declared_diagnostic_after_progress[partial] PASSED [  7%]
tests/test_uk_budget_reform_contract.py::test_a_report_with_the_population_error_diagnostic_does_not_hold PASSED [  8%]
tests/test_uk_budget_reform_contract.py::test_population_coverage_deduplicates_only_country_baseline_rows[routes0] PASSED [  9%]
tests/test_uk_budget_reform_contract.py::test_population_coverage_deduplicates_only_country_baseline_rows[routes1] PASSED [ 10%]
tests/test_uk_budget_reform_contract.py::test_population_coverage_deduplicates_only_country_baseline_rows[routes2] PASSED [ 12%]
tests/test_uk_budget_reform_contract.py::test_the_subsample_is_deterministic_and_keeps_whole_households PASSED [ 13%]
tests/test_uk_budget_reform_contract.py::test_population_checks_hold_on_a_tiny_dataset PASSED [ 14%]
tests/test_uk_budget_reform_contract.py::test_responsiveness_checks_fail_with_no_award_to_move PASSED [ 15%]
tests/test_uk_budget_reform_contract.py::test_expected_annual_for_a_blended_change_that_started_earlier PASSED [ 16%]
tests/test_uk_budget_reform_contract.py::test_the_subsample_reads_a_file_without_an_h5_extension PASSED [ 18%]
tests/test_uk_budget_reform_contract.py::test_household_result_totals_read_each_variable_from_its_own_entity PASSED [ 19%]
tests/test_uk_budget_reform_contract.py::test_a_capped_award_whose_deductions_rise_with_the_allowance_holds PASSED [ 20%]
tests/test_uk_budget_reform_contract.py::test_how_much_deductions_rise_is_not_bounded[2.0] PASSED [ 21%]
tests/test_uk_budget_reform_contract.py::test_how_much_deductions_rise_is_not_bounded[60.0] PASSED [ 22%]
tests/test_uk_budget_reform_contract.py::test_how_much_deductions_rise_is_not_bounded[357.0] PASSED [ 24%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_on_each_unlawful_movement[award-falls-with-its-payment] PASSED [ 25%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_on_each_unlawful_movement[deductions-take-the-whole-award] PASSED [ 26%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_on_each_unlawful_movement[deductions-not-finite] PASSED [ 27%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_on_each_unlawful_movement[cap-reduction-not-finite] PASSED [ 28%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_on_each_unlawful_movement[payment-below-award-less-deductions] PASSED [ 30%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_on_each_unlawful_movement[payment-above-award-less-deductions] PASSED [ 31%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_on_each_unlawful_movement[deductions-released-but-payment-unchanged] PASSED [ 32%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_on_each_unlawful_movement[not-scaled] PASSED [ 33%]
tests/test_uk_budget_reform_contract.py::test_float32_rounding_within_a_penny_holds[award-within-a-penny] PASSED [ 34%]
tests/test_uk_budget_reform_contract.py::test_float32_rounding_within_a_penny_holds[payment-a-little-low] PASSED [ 36%]
tests/test_uk_budget_reform_contract.py::test_float32_rounding_within_a_penny_holds[payment-a-little-high] PASSED [ 37%]
tests/test_uk_budget_reform_contract.py::test_the_uc_allowance_check_fails_when_no_payment_rises PASSED [ 38%]
tests/test_uk_budget_reform_contract.py::test_errors_that_cancel_across_units_still_fail PASSED [ 39%]
tests/test_uk_budget_reform_contract.py::test_a_gain_must_exceed_a_penny[0.009-False] PASSED [ 40%]
tests/test_uk_budget_reform_contract.py::test_a_gain_must_exceed_a_penny[0.011-True] PASSED [ 42%]
tests/test_uk_budget_reform_contract.py::test_the_allowance_must_scale_by_ten_percent[5e-07-True] PASSED [ 43%]
tests/test_uk_budget_reform_contract.py::test_the_allowance_must_scale_by_ten_percent[2e-06-False] PASSED [ 44%]
tests/test_uk_budget_reform_contract.py::test_deductions_may_leave_exactly_the_payable_floor PASSED [ 45%]
tests/test_uk_budget_reform_contract.py::test_extraction_errors_and_unsupported_rows_carry_distinct_ids[country.reform_dict] PASSED [ 46%]
tests/test_uk_budget_reform_contract.py::test_extraction_errors_and_unsupported_rows_carry_distinct_ids[wrapper.policy] PASSED [ 48%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-uc_standard_allowance-base] PASSED [ 49%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-uc_standard_allowance-reformed] PASSED [ 50%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-universal_credit-base] PASSED [ 51%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-universal_credit-reformed] PASSED [ 53%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-universal_credit_pre_benefit_cap-base] PASSED [ 54%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-universal_credit_pre_benefit_cap-reformed] PASSED [ 55%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-benefit_cap_reduction-base] PASSED [ 56%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-benefit_cap_reduction-reformed] PASSED [ 57%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-uc_deductions-base] PASSED [ 59%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[nan-uc_deductions-reformed] PASSED [ 60%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-uc_standard_allowance-base] PASSED [ 61%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-uc_standard_allowance-reformed] PASSED [ 62%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-universal_credit-base] PASSED [ 63%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-universal_credit-reformed] PASSED [ 65%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-universal_credit_pre_benefit_cap-base] PASSED [ 66%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-universal_credit_pre_benefit_cap-reformed] PASSED [ 67%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-benefit_cap_reduction-base] PASSED [ 68%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-benefit_cap_reduction-reformed] PASSED [ 69%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-uc_deductions-base] PASSED [ 71%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[inf-uc_deductions-reformed] PASSED [ 72%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-uc_standard_allowance-base] PASSED [ 73%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-uc_standard_allowance-reformed] PASSED [ 74%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-universal_credit-base] PASSED [ 75%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-universal_credit-reformed] PASSED [ 77%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-universal_credit_pre_benefit_cap-base] PASSED [ 78%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-universal_credit_pre_benefit_cap-reformed] PASSED [ 79%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-benefit_cap_reduction-base] PASSED [ 80%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-benefit_cap_reduction-reformed] PASSED [ 81%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-uc_deductions-base] PASSED [ 83%]
tests/test_uk_budget_reform_contract.py::test_any_non_finite_value_in_either_run_fails[-inf-uc_deductions-reformed] PASSED [ 84%]
tests/test_uk_budget_reform_contract.py::test_the_payable_floor_holds_in_both_runs[0.12-True-base] PASSED [ 85%]
tests/test_uk_budget_reform_contract.py::test_the_payable_floor_holds_in_both_runs[0.12-True-reformed] PASSED [ 86%]
tests/test_uk_budget_reform_contract.py::test_the_payable_floor_holds_in_both_runs[0.115-True-base] PASSED [ 87%]
tests/test_uk_budget_reform_contract.py::test_the_payable_floor_holds_in_both_runs[0.115-True-reformed] PASSED [ 89%]
tests/test_uk_budget_reform_contract.py::test_the_payable_floor_holds_in_both_runs[0.109-False-base] PASSED [ 90%]
tests/test_uk_budget_reform_contract.py::test_the_payable_floor_holds_in_both_runs[0.109-False-reformed] PASSED [ 91%]
tests/test_uk_budget_reform_contract.py::test_an_award_below_the_floor_is_paid_in_full PASSED [ 92%]
tests/test_uk_budget_reform_contract.py::test_each_route_binds_the_reform_it_consumes PASSED [ 93%]
tests/test_uk_budget_reform_contract.py::test_an_unbindable_reform_has_no_identity_and_says_why[april_rate_change-country.reform_from_dict-no representation] PASSED [ 95%]
tests/test_uk_budget_reform_contract.py::test_an_unbindable_reform_has_no_identity_and_says_why[april_rate_change-api.sim_api-in-process] PASSED [ 96%]
tests/test_uk_budget_reform_contract.py::test_an_unbindable_reform_has_no_identity_and_says_why[modifier_then_parameter_changes-country.reform_dict-closure] PASSED [ 97%]
tests/test_uk_budget_reform_contract.py::test_an_unbindable_reform_has_no_identity_and_says_why[parameter_changes_then_modifier-wrapper.policy-closure] PASSED [ 98%]
tests/test_uk_budget_reform_contract.py::test_a_value_receipts_refuse_gives_no_identity_rather_than_an_error PASSED [100%]
=============================== warnings summary ===============================
tests/test_uk_budget_reform_contract.py: 30 warnings
  ./.venv/lib/python3.13/site-packages/policyengine_uk/variables/household/income/household_income_decile.py:25: UserWarning: Accessing .values on a MicroSeries returns a plain numpy array without weights. Operations like .mean() on the result will be unweighted. Use MicroSeries methods directly for weighted calculations (e.g., ms.mean() instead of ms.values.mean()).
    return weighted_income.decile_rank().values
tests/test_uk_budget_reform_contract.py: 30 warnings
  ./.venv/lib/python3.13/site-packages/policyengine_uk/variables/household/wealth/household_wealth_decile.py:16: UserWarning: Accessing .values on a MicroSeries returns a plain numpy array without weights. Operations like .mean() on the result will be unweighted. Use MicroSeries methods directly for weighted calculations (e.g., ms.mean() instead of ms.values.mean()).
    decile = weighted_wealth.decile_rank().values
tests/test_uk_budget_reform_contract.py: 921 warnings
  ./src/policyengine/tax_benefit_models/uk/model.py:278: UserWarning: Accessing .values on a MicroSeries returns a plain numpy array without weights. Operations like .mean() on the result will be unweighted. Use MicroSeries methods directly for weighted calculations (e.g., ms.mean() instead of ms.values.mean()).
    ).values
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:102
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:102: PytestWarning: (rm_rf) unknown function <built-in function scandir> when removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d1627b2f-a967-4600-8abf-726b7deea1b6/test_locked_file_untouched0/root:
  <class 'PermissionError'>: [Errno 1] Operation not permitted: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d1627b2f-a967-4600-8abf-726b7deea1b6/test_locked_file_untouched0/root'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d1627b2f-a967-4600-8abf-726b7deea1b6/test_locked_file_untouched0/root
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d1627b2f-a967-4600-8abf-726b7deea1b6/test_locked_file_untouched0/root'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d1627b2f-a967-4600-8abf-726b7deea1b6/test_locked_file_untouched0
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d1627b2f-a967-4600-8abf-726b7deea1b6/test_locked_file_untouched0'
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95
  ./.venv/lib/python3.13/site-packages/_pytest/pathlib.py:95: PytestWarning: (rm_rf) error removing /private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d1627b2f-a967-4600-8abf-726b7deea1b6
  <class 'OSError'>: [Errno 66] Directory not empty: '/private/var/folders/9l/_wztzgbx7mgc7l1r0416cy7m0000gn/T/pytest-of-maxghenis/garbage-d1627b2f-a967-4600-8abf-726b7deea1b6'
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================= slowest 10 durations =============================
132.81s call     tests/test_uk_budget_reform_contract.py::test_the_capital_gains_cases_hold_on_every_in_process_route
58.02s call     tests/test_uk_budget_reform_contract.py::test_population_checks_hold_on_a_tiny_dataset
24.50s call     tests/test_uk_budget_reform_contract.py::test_responsiveness_checks_fail_with_no_award_to_move
17.90s call     tests/test_uk_budget_reform_contract.py::test_the_default_response_is_reported_without_gating
1.21s call     tests/test_uk_budget_reform_contract.py::test_a_parameter_modifier_writes_the_annualised_parameters
0.26s call     tests/test_uk_budget_reform_contract.py::test_the_subsample_is_deterministic_and_keeps_whole_households
0.05s call     tests/test_uk_budget_reform_contract.py::test_the_subsample_reads_a_file_without_an_h5_extension
(3 durations < 0.005s hidden.  Use -vv to show these durations.)
================= 83 passed, 985 warnings in 234.92s (0:03:54) =================

🤖 Generated with Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Coordinator decision on the reform-responsiveness gate (under d1682 and d1344). This resolves its needs_review for this cut.

Accepted. The additional-rate reform responds in every year, including the four that are withheld. The gate returns needs_review because it can't recompute the tool's verdict for 2027-28 to 2030-31, where the tool withholds the figure. Its docstring names this as the gate's one review path (provenance/uk_candidate.py reform_responsiveness, at e02d41b). I checked that the tool's own verdict covers those years:

  • Withholding comes after the verdict. The tool checks responsive:<id> once per year, using the raw change in the reform's target. Withholding a year changes only what is published, plus the wording of the detail if that year fails (measure_reforms in aggregate_impact/measure.py, policyengine-uk 1191ac5a).
  • A failure in any year fails the check. A failing or missing year marks the merged check as failed and lists that year (Recorder.check; responsiveness fails a NaN change).
  • The receipt shows the verdict passing. responsive:additional_rate_plus_3pp is pass with no failing years. So are the other nine reforms' checks and determinism:rebuilt_baseline.
  • The gate agrees with the tool wherever it can recompute: all ten reforms in every published year, 66 reform-years in all. The additional-rate reform's published changes are +£3.9bn in 2025-26, +£4.2bn in 2026-27 and +£5.5bn in 2031-32, each at least 7 times its £0.55bn minimum.

The standard reforms are the qualification's test of the cut, not Budget measures. This decision says nothing about any dashboard figure.

Follow-up for the aggregate-impact tool (policyengine-uk#2335), not blocking: the receipt doesn't say which of reform_withheld's three conditions withheld 2027-30. The conditions are: a suppressed component changed; a component was left with 1 to 9 records; or the change fell on only 1 to 9 households or records. The baseline suppresses the same two heads in all seven years (Disabled Students' Allowance and Statutory Paternity Pay), so the receipt alone can't explain why only those four years are withheld. Recording which condition fired, with no counts or amounts, would make the next review quicker.

With this decision, the evidence index's gates stand as follows:

  • budget_inputs: accepted, with the data-limit condition;
  • supported_years: pass;
  • reform_contract: pass;
  • reform_responsiveness: accepted.

The budget-inputs condition still applies before adoption, and the independent review of the index is still running.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant