Split private renters across BRMAs to cut BRMA draw noise

[MaxGhenis]

Draft: the code is reviewed, the re-measurement on current main is pending. Merging is a data release, so this lands with a batched uk-data release on Max's go (d833), after a decision on the level shift described below.

Problem

Each FRS household gets one Broad Rental Market Area (BRMA), drawn at random within its region from census private-rented-household counts (#516). In PolicyEngine UK, brma is read only by utils/lha.py:benunit_lha, which feeds the LHA rates and caps for UC and HB.

One draw per household makes the published aggregates depend on the seed. On 10/2, changing only the BRMA seed of #516's build (0 → 1, old weights) moved 2026 UK UC by −£294m, AHC poverty by +165k and England net income by −£1.0bn.

Change

datasets/brma.py:

• household_brma_probabilities gives each household's exact single-draw distribution: the mean of its benefit units' census cells.

• lha_rate_keys gives PolicyEngine UK's LHA rate in every BRMA slot. It is used only to order the walk.

• brmas_at_quantiles and spaced_quantiles are the inverse-CDF draw at given quantiles and the spaced quantiles themselves.

• split_private_renters_across_brmas turns each private-renting household into k records:

  • each record has weight/k and the BRMA at quantile (u + i/k) mod 1, walking the household's BRMAs from the lowest LHA rate up;
  • each record gets brma_split_group and source_household_id;
  • id foreign keys are kept.

  It checks that the household and benefit-unit tables are in PolicyEngine UK's population order, reads tenure through the simulation, and refuses a split without tenure_type, with duplicate ids, or with benefit units that have no people.

create_datasets.py: the split runs after the imputations and the Pension Credit take-up step, and before the OA clone step. PE_UK_DATA_BRMA_SPLIT sets k. The default is 4; k = 1 is today's single draw, bitwise (checked on all 51 aggregates).

utils/calibrate.py: RecordGroups calibrates each split household as one household:

• target rows are its records' mean rows;
• its prior is its total weight;
• it gets one weight, and each record gets an equal share.

A group's records must share their area mask, which is checked exactly, a chunk at a time.

calibration/clone_and_assign.py: a split record keeps its original source_household_id, so survey support isn't overcounted.

Invariants (tested; Hypothesis where it fits)

1. Exact marginals. Each record's BRMA has exactly its household's single-draw distribution, whatever the walk order.
2. Support. A zero-probability or out-of-region BRMA is never chosen, including at boundary quantiles.
3. Spacing. A household's k quantiles fall one in each interval of width 1/k. For any outcome monotone in the walk order, the records' mean is within range/k of its expectation.
4. Conservation. A split household's records carry exactly its weight, people and benefit units. Ids stay unique, and other households are untouched.
5. Determinism. The same seed gives the same records.
6. Grouped calibration. Calibrating with groups equals calibrating the unsplit households on their mean rows, then splitting equally. This is checked against the saved float32 h5.
7. No-op at k = 1. With k = 1 the build is unchanged.

Mutation checks:

• These mutants are each killed by a named test: < for <=, a fixed phase, a missing /k, step offsets, expanding without dividing by the group size, prior means instead of totals, the first national row instead of the mean, and a stale rate cache.
• So are the old benunit-order check, string-only tenure, an off-by-one chunk, and a fingerprint compared with allclose.

Evidence so far

Every number comes from a real PolicyEngine UK run on seeded production-setting builds of #516's head (29e1bd7), 2026, PolicyEngine UK main 7b9fc3792 as of 10/2. Only aggregates are reported.

Weights held fixed: exact scoring of each scheme over 500 draws

One full simulation per BRMA slot gives every household's outcome under every BRMA of its region. That table reproduces the simulation of a dataset to the penny.

Scheme	UK UC SD	UK HB SD	AHC poverty SD
One draw (current)	£177m	£265m	53k
Stratified or systematic draws (any order, weighting or pooling)	£167m–187m	£221m–257m	52k–57k
k = 4 records at spaced, LHA-rate-ordered quantiles	£43m	£86m	13k
k = 8	£22m	£39m	7k
k = 16	£10m	£19m	4k
Expected LHA rate (no draw)	biased: UC +£0.52bn
Expected capped rent (no draw; needs a PE-UK change)	0	0	0, but +49k people in AHC poverty

• The noise is concentrated: 7 FRS households carry half the UC draw variance, and 2 carry half the poverty variance.
• Spreading draws across households can't cancel that. Giving those households several BRMAs can.
• Every sampled scheme's mean matches the exact expectation, within 2.5 SE.

Full rebuilds, recalibrated: SD across builds (mean minus the one-draw mean, ± SE)

Arm	Builds	UK UC	UK HB	AHC poverty	Scotland mean LHA
One draw, BRMA seed varied (current)	10	£214m	£59m	45k	£8.9
Split k = 4, grouped calibration	10	£111m (+£838m ± £76m)	£11m (+£3m ± £19m)	36k (+30k ± 18k)	£1.3 (+£4.8 ± £2.9)
Split k = 8, grouped calibration	5	£97m (+£806m ± £80m)	£5m (+£15m ± £19m)	21k (+62k ± 17k)	£0.4 (+£4.8 ± £2.8)
Split k = 4, records calibrated separately	5	£90m (−£936m ± £79m)	£13m (+£170m ± £20m)	17k (+177k ± 16k)	£3.4 (+£16.4 ± £3.2)
Stratified draw	5	£242m	£21m	84k	£9.3
Torch seed varied (calibration noise floor)	5	£39m	£50m	10k	£0.5

• Grouped calibration matters. Calibrating split records separately moved private renters' weight (dropout-to-mean at 1/k) and shifted UC by about −£0.9bn.
• The build at this PR's head reproduces the grouped k = 4 build to float32 rounding (largest gap: 0.12 benefit units of 7.4m).

Two open issues before this can land

1. The one-draw arm is not a clean comparator. Its BRMA seed is drawn before the imputations, and the seed also changes imputed values. Seed 1 against seed 0 differs in property wealth for 9,147 of 52,846 households, in savings for 538, in consumption for 300–900 per category, and in petrol spending for 23,470. The split arms hold all of that at seed 0. So the one-draw arm's SD and mean include an imputation channel that the split arms don't vary.

   The clean comparator is one fresh draw per private renter at the split's place in the pipeline (k = 1, imputations fixed). That arm is queued.

2. Level shift. The grouped split puts UK UC about £0.8bn above the one-draw mean, with HB unchanged. Calibration fit was about the same. In the 10/2 calibration both arms overshot the two OBR UC targets, and obr/universal_credit_outside_cap was off by 4.6×. Calibrate GB universal credit to one OBR total; reach the UC payment top band #530 has since replaced those two targets with one GB obr/universal_credit target, so the shift needs re-measuring on current main before anyone reads it.

Pending: re-measurement on current main

Builds go through the UK hub's rebuild queue:

• the k = 1 post-imputation redraw arm, with BRMA seed varied;
• grouped k = 4 and k = 8 arms.

All arms use the same imputations and are measured under the PE-UK version this branch locks. The tables above will be replaced with those results.

Cost (k = 4)

• Size. The enhanced FRS grows from 52,846 to 76,390 household records (+45%), and the h5 from 127 MB to 190 MB.
• Speed. One PE-UK simulation of the dataset took 9.3 s against 7.9 s (k = 8: 13.6 s).
• Calibration size. The local calibrations optimise one weight per group, so their size is unchanged. The saved area weights grow with the records.

Reviews

• GPT-6.1 Sol rounds 2 and 3 on earlier heads asked for changes, and all of them are fixed:
  • source identity through the clone step;
  • float32 expansion;
  • stronger invariants;
  • population-order and tenure checks;
  • exact mask comparison.
• An Opus round 4 delta review approved c5d3a02. Its nits are fixed in the last commit.
• A fresh review runs at the final head.

Not in this PR

• microcosm: per-clone spaced BRMAs across the K = 15 local clones. That PR is being ported to current main.
• Rent-conditioned BRMA probabilities (Draw private renters' BRMAs given their rent and bedrooms #521) compose with this.

🤖 Generated with Claude Code