Fix local UC target alignment by area code

[MaxGhenis]

Fixes #468.

Local UC targets followed Stat-Xplore workbook order while calibration rows followed unrelated area-code orders. Birmingham therefore received 5,828 households instead of 175,630, and constituency counts already on 2024 boundaries were also passed through the 2010→2024 mapping.

The parser now matches every area to a canonical code, preferring supplied codes and otherwise normalising names, including Welsh bilingual labels. Missing, duplicate, unknown or ambiguous matches raise before rescaling (policyengine_uk_data/utils/uc_data.py:20). Getters return code-indexed targets (policyengine_uk_data/targets/sources/local_uc.py:73, :118).

Both constituency paths add UC totals and children columns after boundary mapping, then align by code (policyengine_uk_data/datasets/local_areas/constituencies/loss.py:130; policyengine_uk_data/calibration/matrix_builder.py:234). Both LA paths also select UC by their actual target-row codes (policyengine_uk_data/datasets/local_areas/local_authorities/loss.py:117; policyengine_uk_data/calibration/matrix_builder.py:267). The children split takes country from each matched code and uses integer apportionment so its four buckets sum exactly to the area's total (policyengine_uk_data/targets/sources/local_uc.py:99). Database ETL now keys all 1,010 local UC targets by the parser's matched codes and canonical names (policyengine_uk_data/db/etl.py:432).

Code alignment also exposed five unchanged Scottish constituencies represented by 2010 codes in the mapping source (policyengine_uk_data/datasets/local_areas/constituencies/boundary_changes/mapping_matrix.py:8). Those codes are canonicalised, and mapped targets are explicitly reordered to the requested 2024 rows before the country mask is applied. On main only 593 of 650 mapped positions agree with the 2024 code file: every one of the 57 Scottish rows carries another Scottish seat's income, age and housing targets. This PR corrects them as a pure reorder; Scotland's totals are unchanged. SPI income source files and their source ordering are unchanged. The OA LA integration test also exposed imports of removed ONS uprating constants; that path now uses the same year-specific get_ons_income_uprating_factors API as the dense LA builder (policyengine_uk_data/calibration/matrix_builder.py:271).

The Scottish non-UC target moves from main to this PR are:

Target (57 Scottish constituencies)	Rows moving >5%	Rows moving >25%	Maximum move
Age population	44	11	269.7%
Housing anchor	44	11	269.7%
Employment income amount	53	26	245%
Self-employment income amount	53	37	799%

Scotland's totals are unchanged in every row above. These are target-level changes; recalibrated outcomes remain for queue row 17.

The regression tests state these invariants:

• Every UC target matches the independently parsed committed workbook for all 650 constituencies and 360 LAs, in both dense loss builders and the OA geography builder, including shuffled requested rows.
• Every country's totals equal its workbook totals after the existing national payment-distribution rescale and per-area rounding; the rescale itself is preserved.
• Hypothesis permutations of GB workbook rows and NI workbook columns leave code-indexed targets unchanged.
• Name normalisation handles case, punctuation, &/and, accents and either Welsh bilingual-label order. Explicit codes take precedence; incomplete, duplicate, unknown or ambiguous geography raises.
• Every children's row sums exactly to its area total and each integer bucket is within one household of its matched country's share. Dense country masks and feature columns follow the returned target-row order; every mapped constituency age and employment-income amount/count row agrees by code with an independent, name-resolved 2010→2024 boundary aggregation in both dense and OA builders, and each LA age row retains its code under an OA row permutation. Income and age source code order is now checked before positional assignment.
• ETL inserts exactly 1,010 targets with matched codes, canonical names and household counts.

Public target-only impact against daad71cd26c1 is below. Correlation is Pearson's r across all areas; movement means abs(new / old - 1), with strict thresholds. The old constituency vector includes its erroneous boundary mapping and the old row placement.

Area type	Areas	Old/new correlation	>5%	>25%	>100%	Old total → new total
2024 constituencies	650	0.110184	607	427	69	6,095,661 → 6,095,661
Local authorities	360	−0.019566	333	263	86	6,095,645 → 6,095,645

The national payment-distribution total remains 6,095,654. Existing independent integer rounding leaves the same +7 constituency and −9 LA residuals; this PR preserves that behaviour.

Examples are public Stat-Xplore household counts: Birmingham 5,828 → 175,630; Hackney 70 → 38,243; Glasgow City 6,728 → 84,123. Large constituency moves include Havant 28,570 → 8,609, Sheffield Hallam 20,179.5 → 3,016 and Hazel Grove 23,637.7 → 6,492. Fractional old constituency counts result from the erroneous mapping.

Correcting country shares changes the children-column national sums, while preserving the household-count totals above:

Children	Old bucket total	New bucket total
0	3,019,463	2,996,541
1	1,255,614	1,264,398
2	1,086,549	1,094,649
3+	734,039	740,073

Validation uses a locked Python 3.13.9 venv (UV_LINK_MODE=clone uv sync --frozen --all-extras), with BLAS/OpenMP threads capped at four. The dev-only pytest-xdist commit 6c81e891 is reverted in a separate commit. git diff daad71cd -- uv.lock pyproject.toml is empty. pytest-xdist 3.8.0 is supplied only to the validation environment to accept the requested serial -n 0; neither dependency file changes.

UV_LINK_MODE=clone uv sync --frozen --all-extras --python 3.13.9
uv pip install --python .venv/bin/python --no-deps pytest-xdist==3.8.0 execnet==2.1.2
.venv/bin/python -m pytest -n 0 \
  policyengine_uk_data/tests/test_local_uc_alignment.py \
  policyengine_uk_data/tests/test_local_uc_etl.py \
  policyengine_uk_data/tests/test_devolved_housing_targets.py \
  policyengine_uk_data/tests/test_la_loss_missing_sources.py \
  policyengine_uk_data/tests/test_matrix_builder.py \
  policyengine_uk_data/tests/test_uc_payment_distribution_targets.py \
  policyengine_uk_data/tests/test_uc_by_children.py::test_scaled_country_children_buckets_match_latest_gb_totals
uvx ruff format --check .

Results: 45 passed in the foreground (six NumPy matrix-product runtime warnings; value and country-total assertions pass); Ruff: 266 files already formatted (ruff 0.17.1). At the independently reviewed head, all 19 new regression cases fail against daad71cd26c1, with the following breakdown. Nine detect the bug on unmodified main; the other ten fail for missing API or an unrelated import:

Failure reason	Cases	Detects the alignment bug on main?
AttributeError for the new matcher API	9	No
Code index type mismatch	5	Yes
Builder value mismatch	3	Yes
ETL code set mismatch	1	Yes
Pre-existing OA-LA ImportError	1	No; fails on values after the unrelated uprating import is fixed

Round 2 strengthens both constituency builder cases to check mapped age and employment-income rows before UC values, using an independent boundary aggregation that resolves the Scottish historical destination codes by name. This includes S14000107 (2010 S14000006) and renumbered S14000021. Both strengthened constituency cases fail on values against the seven production modules from daad71cd26c1 (2 expected failures; no API/import failures or skips). Both also reject the review's isolated mutant: restore main's mapping_matrix.py, relabel mapped rows positionally with set_axis(const_2024["code"]), and then select requested rows for the shuffled OA path. Each mutant case reports 570 of 6,500 mapped age/income cells mismatched. All source bytes were restored afterward and git diff HEAD is empty. These negative checks ran serially in the foreground; their XML/logs and reproduction script are in .job/.

No dataset was built or recalibrated, and no licensed survey records were read. Recalibrated UC caseload/spending, local target fit and the #452 administrative caseload gap will be measured in the next hub batch build. Proposed rebuild-queue row 17 is prepared in .job/QUEUE_ROW.md for the hub to copy; it has no GO. Merge remains subject to Max's batch go (d833).

[MaxGhenis]

Hand-off at 248688c53b1060b1e32d46331c72313494fe24dd (PR #589). Commits: 6c81e891 locks the dev test runner; 248688c5 fixes code matching, post-mapping constituency UC, matched-country children splits, LA/OA alignment and the database importer. Income source files are unchanged.

Validation: 45 targeted tests passed in the foreground with -n 0, thread caps ≤4; uvx ruff format --check . passed (266 files). All 19 new cases fail against the original daad71cd26c1 production modules (no skips/setup errors). No dataset build, recalibration, licensed-record read or data-file push.

Target-only impact: constituency Pearson r 0.110184; LA r −0.019566. Areas moving >5%/>25%/>100%: 607/427/69 of 650 constituencies and 333/263/86 of 360 LAs. UC household totals remain 6,095,661 and 6,095,645, preserving existing rounding. Birmingham 5,828→175,630; Hackney 70→38,243; Glasgow City 6,728→84,123. Correcting country shares changes child bucket totals; each area now sums exactly.

Open items for the hub: independent review at this head; copy proposed queue row 17 from .job/QUEUE_ROW.md; measure recalibrated UC caseload/spending, local fit and #452 in the next batch. No build GO and no merge; landing remains subject to Max's d833 batch go. .job/HANDOFF.md, .job/handoffs.jsonl, .job/IMPACT.md and a Git bundle are prepared in the assigned workspace for hub collection.

[MaxGhenis]

Independent review with execution (UK hub, GPT-6.1 Sol, -s workspace-write): APPROVE WITH NITS at 248688c53.

The UC fix is correct.

• The reviewer's own oracle (openpyxl parse, header search, independent name normaliser) shows every UC row matches its code's workbook value in both dense builders and the OA builder, for both area types and for sorted, shuffled and reversed orders.
• Per-country totals match, and every country-mask row carries its own code's country.
• Every number in IMPACT.md reproduces. Constituency r = 0.110184 and LA r = −0.019566; Birmingham 5,828 → 175,630.

To fix before the batch (round 2):

1. Medium. The PR also corrects a second misalignment. On main, the 57 Scottish constituency rows carried another Scottish seat's income, age and housing targets (only 593 of 650 positions agree after the 2010→2024 map).
   • The fix is a pure reorder, but no test covers it. Reverting mapping_matrix.py and relabelling by position still passes every new test.
   • Add a builder test with at least one income variable that asserts mapped age and income rows per code, for example pinning S14000107 ← 2010 S14000006.
   • Cover it in the changelog and the PR body, with the size table (age and housing: 44 rows over 5%; employment income 53; self-employment 53, max 799%).
2. Low. Drop 6c81e891 (dev-only pytest-xdist). Production pins don't move: policyengine-uk 2.122.2, core 3.32.13 and numpy, pandas and torch are all unchanged. But the commit adds a one-line uv.lock conflict with Calibrate DWP and OBR welfare targets on the countries their source covers #542, Build council_tax before council tax reduction; stop imputing reported CTR on SPI rows #499, Split pensions, shares, trusts, ISAs and secured debt out of WAS wealth #501, Derive property_wealth from its uprated components instead of saving the WAS total #543 and Fill benunit_reported_capital from FRS benefit-unit capital, so UC uses it too #552, and CI never runs -n.
3. Low. "19 new cases fail on main" includes 9 AttributeErrors, where the new API is missing, and 1 pre-existing ImportError. Say how many detect the bug.
4. Nits:
   • add an equality check that income rows share the age code order (matrix_builder.py:202-219, both loss.py);
   • reword the Welsh-label comment (uc_data.py:15-16).

Scope additions ruled:

• The OA LA uprating change is accepted. The ImportError is real on main, and the replacement equals the removed constants for 2023–2030, so targets are unchanged.
• pytest-xdist is acceptable but better dropped (item 2).

This PR joins the next uk-data batch, which merges on Max's go.

[MaxGhenis]

Round 2 hand-off at a2eedce44e3a1fd5f16919f28ae022ad61820984. Pushed once to fix/468-local-uc-alignment; no history rewrite or merge.

• ff643bec reverts dev-only 6c81e891; git diff daad71cd -- uv.lock pyproject.toml is empty.
• 26e51a25 retains employment income in both constituency builder cases and checks every mapped age/income row against an independent name-resolved boundary aggregation. Changelog and PR body now disclose all 57 Scottish non-UC row corrections, the full size table, and unchanged Scottish non-UC totals.
• a2eedce4 checks income/age code order in all three loaders and updates the Welsh-label comment and fixtures.

Validation: 45 passed with -n 0 in the foreground (six existing NumPy matmul warnings). Both strengthened cases fail on main on values. Both reject the Scottish positional-relabel mutant on values: 570/6,500 mapped cells in each builder. Sources restored; working tree matches the committed head. uvx ruff format --check .: 266 files already formatted. PR body gives the review's 19-case breakdown and identifies the nine direct bug detections on unchanged main.

Impact remains target-level. No rebuild, recalibration, licensed-record read or data-file push. Recalibration stays in queue row 17, pending hub GO. Hub should dispatch independent review at this head; batch merge awaits Max's GO.

.job/HANDOFF.md, .job/handoffs.jsonl, logs/XML and verified .job/round2.bundle are available for collection. Sandbox-local Git metadata is .job/git; the original linked-worktree HEAD is unchanged, so collect the pushed head or bundle. Caller checkout untouched.