Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
58b90a6
Declare policyengine-uk's whole-population formulas and share the eng…
juaristi22 Nov 7, 2026
6901fd9
Carry the other spells by the engine's growth parameters (#931)
juaristi22 Nov 7, 2026
3d5ad69
Correct each engine block to the pool's share denominator and judge t…
juaristi22 Nov 7, 2026
935f706
Batch spine engine calls by clone with the pool's share denominator (…
juaristi22 Nov 7, 2026
ff8b855
Rank private-school attendance over the pool from household-local eng…
juaristi22 Nov 7, 2026
48d12e5
Describe corrected engine blocks in the dense runbook and the build-g…
juaristi22 Nov 7, 2026
369791d
Read batch records from the frame and hold each batch's tree to the r…
juaristi22 Nov 7, 2026
3229048
Read the survey's net income in both WAS stages (#931)
juaristi22 Nov 7, 2026
9b3d8c7
Describe the WAS income predictor as the engine's closest concept (#931)
juaristi22 Nov 7, 2026
cefe9df
Hold the corporate and total-economy land rows as measured diagnostic…
juaristi22 Nov 7, 2026
016797e
Test that the two land diagnostics leave the solve but stay measured …
juaristi22 Nov 7, 2026
b5ce1ba
Follow up the review of the land diagnostics (#931)
juaristi22 Nov 7, 2026
22cde4b
Mint UK copy ids from fixed radices that do not depend on K (#931)
juaristi22 Nov 7, 2026
52da13c
Follow up the review of the fixed copy-id radices (#931)
juaristi22 Nov 8, 2026
c1723a7
Decode UK copy ids at the declared radices outside the stages (#931)
juaristi22 Nov 8, 2026
b07e5b3
Follow up the review of the copy-id decoders (#931)
juaristi22 Nov 8, 2026
8ec385c
Count CGT support copies on the source-scaled weight (#931)
juaristi22 Nov 8, 2026
bf9173a
Follow up the review of the source-scaled CGT copy count (#931)
juaristi22 Nov 8, 2026
bad1f17
Tidy the re-reviews of the copy-id decoders and the CGT copy count (#…
juaristi22 Nov 8, 2026
3903903
Say where the held land rows are measured (#931)
juaristi22 Nov 8, 2026
06b1aa3
Retire the unused legacy UK national sampler (#931)
juaristi22 Nov 8, 2026
ebd0f1a
Bind every value-computing helper into the keys of the stages that ca…
juaristi22 Nov 8, 2026
5965036
Declare the CGT copy-count snap and hold the K = 1 recount to the tol…
juaristi22 Nov 8, 2026
4ba334a
Fold the high-value copies onto their roots and check E5 where the up…
juaristi22 Nov 8, 2026
254bbe4
Follow up the review of the sampler, binding, fallback and tool commi…
juaristi22 Nov 8, 2026
c64c2b3
Draw the SPI support channel as a keyed Bernoulli share (#931)
juaristi22 Nov 8, 2026
8137a1b
Seat the SPI income band donors within each geographic clone (#931)
juaristi22 Nov 9, 2026
07e9747
Follow up the review of the SPI share and band-donor commits (#931)
juaristi22 Nov 9, 2026
14849e6
Correct the CGT and high-value support totals on whole households fir…
juaristi22 Nov 9, 2026
c359955
Hold the K = 1 support-copy count exactly where a family's members ca…
juaristi22 Nov 9, 2026
b6e0278
Bind the first-party helpers outside microcosm.build into the stages …
juaristi22 Nov 9, 2026
2addcb6
Hold the other UK graph kernels to the binding rule (#931)
juaristi22 Nov 9, 2026
7607383
Follow up the review of the 9 Nov correction, count-check and binding…
juaristi22 Nov 9, 2026
c014d9a
Name stage 42's one-copy check and the fallback's limit in its declar…
juaristi22 Nov 9, 2026
6e42fa1
Hold the support-copy count on the members with untouched clone halve…
juaristi22 Nov 9, 2026
a4bb4a2
Bind what the injected engine computes through into the engine stages…
juaristi22 Nov 9, 2026
a509ae2
Key each gate battery phase on what its own gates compute through (#931)
juaristi22 Nov 9, 2026
c7874fc
Hold the UK target and problem kernels to the derived binding rule (#…
juaristi22 Nov 9, 2026
9f5469c
Follow up the review of the 9 Nov ruling commits (#931)
juaristi22 Nov 9, 2026
63f825d
Key the CGT redraw's walk offsets and plan jitters, and BADR's band o…
juaristi22 Nov 9, 2026
cca9a16
Break ties at the six id-ordered walks on identity-keyed uniforms (#931)
juaristi22 Nov 10, 2026
610123e
Give tied ISA balances one weighted mid-rank (#931)
juaristi22 Nov 10, 2026
f59bd51
Follow up the review of the keyed offsets, tie-break and ISA rank com…
juaristi22 Nov 10, 2026
b4868fd
Extract the derived binding rule into a shared test module (#931)
juaristi22 Nov 10, 2026
1ed4ada
Count a store into a container a function creates as local in the ref…
juaristi22 Nov 10, 2026
c45f6ed
Hold every transport kernel to the derived binding rule (#931)
juaristi22 Nov 10, 2026
358cc04
Close two gaps in the refusal check's local containers and control th…
juaristi22 Nov 10, 2026
0b888f4
Hold the engine hand-off to the built registry and correct two transp…
juaristi22 Nov 10, 2026
099bd16
Move the gate phase codes into a refusal-only module and refuse a sta…
juaristi22 Nov 10, 2026
fce2f86
Take frs_spine and frs_relationships off the target kernels' keys (#931)
juaristi22 Nov 10, 2026
1fbb41f
Count module-level aliases and wrapper internals in the hold's premis…
juaristi22 Nov 10, 2026
d3085ce
Follow up the review of the phase-code, target-key and premises-test …
juaristi22 Nov 10, 2026
a3a978e
Check every import of the annuity column's name, and say where the pr…
juaristi22 Nov 10, 2026
701ae51
Say the Lifetime ISA rank blends no income rank, and that tied balanc…
juaristi22 Nov 10, 2026
76909e5
Widen the Child Benefit gate's per-age tolerance to three standard er…
juaristi22 Nov 10, 2026
289cac9
Scope E8's band-donor rerun to the frame stage 17 saw (#931)
juaristi22 Nov 10, 2026
0982bfa
Fold geographic clones onto their lowest-id record, check record ids …
juaristi22 Nov 10, 2026
49f87c5
Add the geographic expand and identity transforms, and make three UK …
juaristi22 Nov 11, 2026
a7586b5
Clone the UK spine after its source stages: move stages 14 and 15 up …
juaristi22 Nov 11, 2026
1c54405
Receipt the geographic clones: fold variety, stage 16 and 17 forms, a…
juaristi22 Nov 11, 2026
330557f
Take the clone count on the spine CLI and hand it to the engine (#931)
juaristi22 Nov 11, 2026
9ace5e2
Keep stage 16's refusal of a clone stratum without an SPI copy (#931)
juaristi22 Nov 11, 2026
4c341b1
Add a shared fallback-level chooser for conditioned area draws (#931)
juaristi22 Nov 11, 2026
e897513
Follow up the shared fallback-level chooser's review (#931)
juaristi22 Nov 11, 2026
8191972
Add the spine's area, local-authority, BRMA, fuel-relief, SPI-copy ar…
juaristi22 Nov 11, 2026
ba5e016
Count the take-up anchors and the apprentice target within each geogr…
juaristi22 Nov 11, 2026
725b9a5
Run the clone-first H2 test after merge only (#931)
juaristi22 Nov 11, 2026
5034f9b
Give each engine simulation rank ids where policyengine-core's 32-bit…
juaristi22 Nov 12, 2026
26ba2a2
Refuse a dense build on a spine that does not assign areas (#931)
juaristi22 Nov 12, 2026
51eee49
Retire the dense line's property, BRMA, private-rent and fuel-relief …
juaristi22 Nov 12, 2026
151ca60
Build the dense line on the bound spine's clones, areas and sampling …
juaristi22 Nov 12, 2026
d6be8df
Hold dense engine blocks, release candidates and size evaluation to t…
juaristi22 Nov 12, 2026
31fffe7
Add the WAS home-value localiser, stage 25's factor helper and the ge…
juaristi22 Nov 12, 2026
d741fb7
Measure the spine's own area draw in the atomic-assignment tool (#931)
juaristi22 Nov 12, 2026
a4c793e
Assign each clone's area in the spine, fold frs_brma into area_brma, …
juaristi22 Nov 13, 2026
a391831
Localise home values inside the WAS chain, retire regional_home_value…
juaristi22 Nov 13, 2026
3c41277
Export the national line from a one-clone spine that carries areas (#…
juaristi22 Nov 13, 2026
eeba2e3
Vendor 2024-25 Band D council tax for Great Britain's authorities, wi…
juaristi22 Nov 14, 2026
86174e3
Add shared helpers for a conditioned draw cell's expected value and f…
juaristi22 Nov 14, 2026
aea6e53
Add the council-tax bill provenance and the reduction-within-bill che…
juaristi22 Nov 14, 2026
494ca16
Add G6, council-tax localisation by the expected Band D of each house…
juaristi22 Nov 14, 2026
7e83997
Localise council tax by each clone's local authority before the WAS c…
juaristi22 Nov 14, 2026
b064a65
Gate council tax reduction within the bill, and hold later copies to …
juaristi22 Nov 14, 2026
e83c20f
Add the housing shell's council tax at each SPI copy's reference Band…
juaristi22 Nov 15, 2026
bf87cbc
Restore main's two spine stages to the stage counts that the rebase o…
juaristi22 Nov 15, 2026
5670b6e
Count the carer-link share's base in true source households on the cl…
juaristi22 Nov 15, 2026
b776798
Restate this branch's export and line tests to main's reference build…
juaristi22 Nov 15, 2026
b67c660
Teach main's CI plan the push-only engine-uk-after-merge job, and mak…
juaristi22 Nov 15, 2026
784e419
Hand main's one-command cut runner the conditioned atomic-area suppor…
juaristi22 Nov 15, 2026
96efc27
Build each per-clone engine call through the read-year window, and pi…
juaristi22 Nov 15, 2026
704191f
Bind main's newer stages and gate code under the derived binding rule…
juaristi22 Nov 15, 2026
1aab875
Re-derive the UK gate battery's contract pins from the merged gates.j…
juaristi22 Nov 15, 2026
035cc0e
Say which Chronicle commit the England and Wales Band D resource is r…
juaristi22 Nov 15, 2026
6567ce1
Price Great Britain SPI copies' council tax by the statute after the …
juaristi22 Nov 15, 2026
aa57265
Leave a dated measure's year after the resolver's out of an engine bl…
juaristi22 Nov 15, 2026
a28c250
Regenerate the H2 fixture and the coverage manifest on #1519's new he…
juaristi22 Nov 15, 2026
a1a9480
Re-pin the seed protocol and the specs that embed it, now that main a…
juaristi22 Nov 15, 2026
fcb65a3
Hold main's three income-evidence kernels to the transport binding ru…
juaristi22 Nov 15, 2026
f53d8dd
Count main's pip_points among the stages after the support split that…
juaristi22 Nov 15, 2026
4607e6e
Hold the apprentice walk's clone tests and E4 to main's full-time edu…
juaristi22 Nov 16, 2026
02f289a
Answer main's LCWRA element in the declaration test's Universal Credi…
juaristi22 Nov 16, 2026
7b7a59c
Re-review the UK engine-population declaration on main's policyengine…
juaristi22 Nov 16, 2026
dbad990
Refuse SPI training clones that share a key after the fold, and test …
juaristi22 Nov 16, 2026
f22cad9
Say that no gate can serve a stale count under a renamed source colum…
juaristi22 Nov 16, 2026
0f1a1c4
Restate four fragments' key and binding counts as their own commits g…
juaristi22 Nov 16, 2026
21421e2
Count main's three income-evidence kernels in the transport fragments…
juaristi22 Nov 16, 2026
45ceca0
Restate five more fragments' kernel, node and stage counts with main'…
juaristi22 Nov 16, 2026
2b0a792
Keep the H2 acceptance test on the public API: drop its rank-rule rec…
juaristi22 Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
30 changes: 28 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,10 @@ defaults:
# job's slowest successful leg on 2026-11-09, rounded up to a multiple of 5
# minutes: 1.5 times it for engine-free (109 minutes) and engine-us (73), twice
# for engine-uk (45) and integration-uk (8), and for each shorter job at least
# 5 minutes and at least twice its slowest run.
# 5 minutes and at least twice its slowest run. The engine-uk-after-merge cap
# is about 1.6x its expected leg: microcosm#931's clone-first H2 test runs
# about 2.6 times as long as the H2 parity test (10 minutes in engine-uk), so
# about 27 minutes, plus the sync.

# Every job names its runner image. `ubuntu-latest` is whichever image GitHub
# assigns: on 2026-11-10 it gave one engine-free job ubuntu-26.04 and another
Expand Down Expand Up @@ -148,6 +151,27 @@ jobs:
- name: Run UK engine tests
run: bash tools/run_engine_uk_tests.sh

engine-uk-after-merge:
# UK engine tests too slow for every pull request run on pushes to main
# only, after merge (microcosm#931). ci needs this job: on a push it must
# pass, and on a pull request its skip passes.
if: github.event_name == 'push'
runs-on: ubuntu-24.04
timeout-minutes: 50
strategy:
fail-fast: false
matrix:
python-version: ["3.13", "3.14"]
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
python-version: ${{ matrix.python-version }}
- name: Sync workspace with the UK engine
run: uv sync --all-packages --locked --extra uk
- name: Run UK after-merge engine tests
run: bash tools/run_engine_uk_tests.sh engine-uk-after-merge

integration-uk:
needs: select-countries
if: needs['select-countries'].outputs.run_uk == 'true'
Expand Down Expand Up @@ -204,7 +228,8 @@ jobs:
ci:
# ci gives the main ruleset one check that covers the country jobs. It
# fails unless every other job passed, or skipped because select-countries
# succeeded and reported `false` for its country. A skipped matrix job
# succeeded and reported `false` for its country, or skipped because it
# runs after merge and the run is not a push. A skipped matrix job
# reports under its unexpanded name, so a ruleset cannot require engine-us
# or engine-uk by name. GitHub reports a skipped required check as passing,
# so ci runs with `if: always()`. verify-workflow refuses any other shape
Expand All @@ -216,6 +241,7 @@ jobs:
- engine-free
- engine-us
- engine-uk
- engine-uk-after-merge
- integration-uk
- wheels
- installed-emitter
Expand Down
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ out/
*.h5
# Licensed run outputs: pool tables and audit directories (UKDS CD137 cl. 8).
*.npz
# Except the H2 parity fixture's three toy conditioned atomic-area supports,
# which tools/graph_uk_spine_fixture.py writes from test support's invented
# table over public area codes, as toy_<source id>.npz: a real support keeps
# its own name and stays ignored (microcosm#931).
!packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/toy_uk_ew_output_area_2021_conditioned_support.npz
!packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/toy_uk_scotland_output_area_2022_conditioned_support.npz
!packages/microcosm-graph/tests/fixtures/parity/uk_spine/sources/toy_uk_ni_data_zone_2021_conditioned_support.npz
audit-*/
!packages/*/build/
# microcosm.build is a real namespace package, not a build-artifact dir
Expand All @@ -27,3 +34,8 @@ tools/route_a/route_a.env
# file holds licensed-data review values and is never committed
PRIVATE_review.csv
signoff-draft/

# Spine stage profiles (--stage-profile-dir, microcosm#931): a record keeps the
# message of a failure its node raised, so profiles stay private
stage_profile.json
.stage_profile.json.lock
2 changes: 1 addition & 1 deletion changelog.d/1112-national-home-values.added.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
The national release now carries Land Registry home values (microcosm#1112). A new spine stage, `regional_home_values`, runs after the high-value home support copies and gives every England-and-Wales owner household with a value the one its region's uprated Land Registry distribution holds at the household's weighted mid-rank among the owner households of its FRS region, at the spine's weights. It ranks as the dense line's value node does (ties broken by a keyed draw; the support copies spread over the top stratum by their own draws), reads the region level of the same knots resource (`uk/high_value_home_value_knots.json`; Wales is `W92000004`), and moves `property_wealth` by the value's change. Scotland, Northern Ireland, renters and owners without a value keep theirs, and the council-tax band is untouched.
The national release now carries Land Registry home values (microcosm#1112). A new spine stage, `regional_home_values`, runs after the high-value home support copies and gives every England-and-Wales owner household with a value the one its region's uprated Land Registry distribution holds at the household's weighted mid-rank among the owner households of its FRS region, at the spine's weights. It ranks as the dense line's value node does (ties broken by a keyed draw; the support copies spread over the top stratum by their own draws), reads the region level of the same knots resource (`uk/high_value_home_value_knots.json`; Wales is `W92000004`), and moves `property_wealth` by the value's change. Scotland, Northern Ireland, renters and owners without a value keep theirs, and the council-tax band is untouched. (Later in #931 the stage retires: the WAS chain reads each England and Wales owner's main residence off its assigned constituency instead, `931-spine-localises-home-values`.)

Before it, the national line solved the bound spine with the WAS-imputed values after the regional uprating, not the regions' Land Registry values. The dense line is unchanged: the stage keeps the value it replaced in `main_residence_value_rank_basis`, the dense value node receives that column through its slice and ranks on it (refusing if it is missing), and the release boundary drops it on both lines.

Expand Down
2 changes: 1 addition & 1 deletion changelog.d/1262-uk-brma-from-area.added.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
A placed household's Broad Rental Market Area now follows its assigned area (microcosm#1262, B1). `tools/build_uk_brma_area_layer.py` places every 2021 LSOA of England and Wales and every 2022 output area of Scotland in one BRMA from Chronicle's raw registrations (PolicyEngine/chronicle#314): England and Wales by population-weighted centroid in the VOA May 2020 and Rent Officers Wales May 2012 polygons with uk-data#516's six overrides, Scotland by centroid in the Scottish Government's polygons corrected by Rent Service Scotland's postcode lookup with microcosm#1092's two overrides. The layer, `uk/brma_area_layer.json` (192 BRMAs, OGL v3.0), reproduces the region-by-BRMA support of uk-data#516's census table and microcosm#1092's 63 Scottish BRMA-by-council pairs exactly; its provenance records both checks. The post-geography node `uk.full.geography.brma` (`--brma-localisation area`, the default on the atomic assignment; `none` for measurement builds, refused by a release candidate) reads each placed Great Britain clone's BRMA off its LSOA or output area, so its LHA rate describes the same place as its local authority and constituency. Northern Ireland and the national line keep the spine's `frs_brma` draw. A terminal release-blocking gate, `uk_local_brma_enum_domain`, rechecks the rewritten column against the engine's enum; the local gate battery has seven entries. The dense and full-battery contract digests, the declared gate count and the certification exclusions move with it, so a national or dense release certified before this change fails `validate_release` after it.
A placed household's Broad Rental Market Area now follows its assigned area (microcosm#1262, B1). `tools/build_uk_brma_area_layer.py` places every 2021 LSOA of England and Wales and every 2022 output area of Scotland in one BRMA from Chronicle's raw registrations (PolicyEngine/chronicle#314): England and Wales by population-weighted centroid in the VOA May 2020 and Rent Officers Wales May 2012 polygons with uk-data#516's six overrides, Scotland by centroid in the Scottish Government's polygons corrected by Rent Service Scotland's postcode lookup with microcosm#1092's two overrides. The layer, `uk/brma_area_layer.json` (192 BRMAs, OGL v3.0), reproduces the region-by-BRMA support of uk-data#516's census table and microcosm#1092's 63 Scottish BRMA-by-council pairs exactly; its provenance records both checks. The post-geography node `uk.full.geography.brma` (`--brma-localisation area`, the default on the atomic assignment; `none` for measurement builds, refused by a release candidate) reads each placed Great Britain clone's BRMA off its LSOA or output area, so its LHA rate describes the same place as its local authority and constituency. Northern Ireland and the national line keep the spine's `frs_brma` draw. A terminal release-blocking gate, `uk_local_brma_enum_domain`, rechecks the rewritten column against the engine's enum; the local gate battery has seven entries. (Later in #931 the dense line takes its BRMAs from the spine, `uk_local_brma_enum_domain` retires, the local battery has six entries, and `uk_brma_enum_domain` checks the column at the spine's transferred boundary: `931-spine-localises-home-values`.) The dense and full-battery contract digests, the declared gate count and the certification exclusions move with it, so a national or dense release certified before this change fails `validate_release` after it.
2 changes: 1 addition & 1 deletion changelog.d/1308-uk-new-stage-health-gates.added.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
The three stages #1308 left ungated have stage-health gates (María's ruling of 2026-10-17, taken up with the next policyengine-uk bump), each release-blocking and blocking on an absent receipt. `uk_stage_property_purchases_fit` holds each purchase kind to HMRC's Stamp Tax transactions within 5% and to its value less a recorded shortfall of at most one fifth, the floor the stage itself refuses below. `uk_stage_regional_home_values_ranking` requires rows re-ranked in every English region and Wales on the 2024 knots, at the weighted midpoint within each FRS region, with no source at the value cap. `uk_stage_uc_legacy_closure_take_up_contract` checks what the Move to UC receipt prints, since it withholds what a thin combination realizes: the design-weight Great Britain basis, a probability and a declared source for every combination's rate, a printed support band, and a well-formed unreachable flag. The property purchases and regional home-values gates check the frame against published facts, so a synthetic smoke build records their failures without blocking on them (`population_fact_check`), as it does for the other fact gates; the legacy closure gate blocks every posture. The mirrored battery digests move.
The three stages #1308 left ungated have stage-health gates (María's ruling of 2026-10-17, taken up with the next policyengine-uk bump), each release-blocking and blocking on an absent receipt. `uk_stage_property_purchases_fit` holds each purchase kind to HMRC's Stamp Tax transactions within 5% and to its value less a recorded shortfall of at most one fifth, the floor the stage itself refuses below. `uk_stage_regional_home_values_ranking` requires rows re-ranked in every English region and Wales on the 2024 knots, at the weighted midpoint within each FRS region, with no source at the value cap. (Later in #931 it becomes `uk_stage_area_home_values_ranking`, on the WAS stage's area home values at the constituency level, with its maximum re-derived for the constituency knots: `931-spine-localises-home-values`.) `uk_stage_uc_legacy_closure_take_up_contract` checks what the Move to UC receipt prints, since it withholds what a thin combination realizes: the design-weight Great Britain basis, a probability and a declared source for every combination's rate, a printed support band, and a well-formed unreachable flag. The property purchases and regional home-values gates check the frame against published facts, so a synthetic smoke build records their failures without blocking on them (`population_fact_check`), as it does for the other fact gates; the legacy closure gate blocks every posture. The mirrored battery digests move.
Loading