Skip to content

UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) - #1426

Merged
juaristi22 merged 47 commits into
mainfrom
uk-1095-spi-age-band-hrad
Nov 4, 2026
Merged

juaristi22 merged 47 commits into
mainfrom
uk-1095-spi-age-band-hrad

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #1095. This PR collects the #1095 items that pass their measurement, and since 31 October the Phase B items too. Each item is measured in its own national build against the one before it. Every figure that rests on the SPI channel is given as a class only (SN 9422 and the disclosure rule).

A/B items (measured against the 28 October build, then against the 2 November control):

  1. The SPI stage-1 forests train and query on the SPI age band (uk-data#628).
  2. Households' additional-dwelling (HRAD) purchases at or below £250,000 are allocated from HMRC's Table 8a receipts.
  3. Drawn SPI employees' current pay is floored at what their twin's hours earn, and the tape's pay stays their year of pay (uk-data#629).

Phase B items (each measured against the build before it):

  • B5: Carer's Allowance claims are drawn after salary sacrifice, keyed on lineage.
  • B12: Scottish owners' council tax bands are drawn from their council area's band mix.
  • AC3: the two-child-limit rows date births by the engine's cutoff test.
  • B7: children in non-parental care are marked for the two-child limit's exception.
  • B10: SPI copies take their twin's contributory JSA and ESA within each benefit's own work limits.
  • B9: each household's student-loan balance moves to its borrowers.
  • B4: HMRC's HICBC liability count and revenue join as measured diagnostics.
  • B3, PR 2b (J2 and K): the NTS cars grade the zero-emission share by income, and each road fuel is levelled to its household-paid share.
  • B3, vehicle excise duty: each adult's car gets policyengine-uk's VED inputs from the NTS cars.

Phase B items that change nothing in the build:

Also since the first review:

How the items were measured

  • Control: María's national build of 2 November, cut f6e33b93 at SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428's 329a99e71 (policyengine-uk 2.165.1, feed 08962f1b), staged on populace-uk-private.

  • T1426: this branch with the A/B items and Vahid's fixes, built on this machine with the control's review date (2 November) and feed.

    • Its 17 stages before the SPI income stage give the same evidence and records as the control's. So the two machines' inputs agree.
  • Phase B: one build per group of items, in a chain: B-1 against T1426, B-2 against B-1, B-3 against B-2. All use the same review date, feed and lock, and all 1,339 rows are active in every build.

  • Main merged in: the branch now merges main at 312dc4d6b, which adds SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428's last commit (2dbf7cf, SPI banding on total income net of expenses). A control rebuilt on main and this branch's merged tree measure the whole PR on the current base (below).

  • Control (f6e33b93): 98.5% of rows within 10%, final loss 0.00529.

  • T1426 (A/B items and Vahid's fixes):

    • 98.8% within 10%, final loss 0.00527.
    • 21 rows move more than 5 points, all HMRC SPI regional and income-band rows. That is reseed-sized: a placebo that only reseeded the SPI stage moved 22.
  • B-1 (B5): 98.8% within 10%, final loss 0.00527. No row moves.

  • B-2 (AC3, B7): 98.7% within 10%, final loss 0.00532. No row moves more than 5 points.

  • B-3 (B10, B9): 98.7% within 10%, final loss 0.00522. No row moves more than 5 points.

  • Control on main (ctl-312d):

    • 98.7% within 10%, final loss 0.00539.
    • Its 17 stages before the SPI income stage give the same evidence and records as the control's; from that stage on it differs, as main's SPI banding commit should.
    • Against the control, 10 rows move more than 5 points, all HMRC income-band and regional rows.
  • This branch merged with main (t1426m):

    • 98.7% within 10%, final loss 0.00520.
    • Against the control on main, by stage:
      • Its stages before the SPI income stage give the same evidence and records, except frs_relationships (B7).
      • From the SPI income stage on, every stage's evidence differs, as this branch's SPI items should.
      • The stage order differs only where B5 moves the Carer's Allowance stage.
    • Against the control on main, by row:
      • 14 rows move more than 5 points, all HMRC income-band and regional rows. That is fewer than the 22 a reseed-only placebo moved.
      • Before calibration, obr.jobseekers_allowance's shortfall halves (B10).
      • After calibration, dwp.esa_claimants comes inside 10%. The UC households-by-children rows stay near the 10% line, as they do in both controls.
  • B-5a (B4, J2 and K) against t1426m:

    • 98.7% within 10%, final loss 0.00525. No row moves more than 5 points.
    • By stage, only lcfs_consumption differs.
    • dwp.uc.households_single_with_children slips outside 10%. It sits at the line in every build.
    • B-5a and the builds after it calibrate at review date 3 November, when B4's exclusions start. The 1,339 rows are the same.
  • b5sp (the SPI prior shape) against B-5a: 98.7% within 10%, final loss 0.00524. No row moves more than 5 points. The shape is off the branch (above).

  • B-5b (VED) against b5sp: only lcfs_consumption differs, and the calibration is identical: no target reads the new inputs.

  • b5j (JSA's own limits, with VED and without the shape) against B-5a:

    • It builds ccc18c08f: 0cddc8464 with the node's declared reads, on the 312dc4d6b base.
    • 98.9% within 10%, final loss 0.00525 (B-5a: 98.7% and 0.00525). No row moves more than 5 points.
    • By stage, only lcfs_consumption (VED), spi_benefit_coherence and uc_take_up differ.
    • Three rows come inside 10%: dwp.uc.households_children_3, dwp.uc.households_single_with_children and hmrc.spi_region.taxpayers_by_region_50000_70000@E12000008. None goes outside.

Every build above passes every spine gate and all seven calibration gates. They measure the items on main at 312dc4d6b. Main is now merged in at 79a533c72, so a control on that main and a build of the merged head are owed before a whole-PR comparison on the new base.

1. The SPI stage-1 forests on the SPI age band (uk-data#628)

Why

The 2022-23 SPI public-use tape gives age only as a band (AGERANGE). The SPI income stage drew an age for each donor inside its band and trained the stage-1 tape forest on it, so the forest learned no real income pattern within a band. Twins of one band were then drawn from conditionals that differed only by that noise. uk-data#628 reports the same defect in uk-data.

What changes

  • The stage-1 tape and reference forests read the SPI age band, one-hot, instead of age (Vahid's review). On the tape it is the AGERANGE code, with composite records (-1) as a band of their own. On a frame person it is the band of their whole-year age, through one helper shared by both frames.
  • at_or_over_spa still carries the State Pension split inside the 65-74 band.
  • The cross-fit fold cells key on the band.
  • The drawn donor ages stay on the donor for their other readers: the band pools' State Pension side and the property components' tape seed.

Measurement

  • Against the 28 October build, the item moved no more rows than a draw-seed placebo, and the UC rows ended closer to DWP's.
  • The copies' pay now rises with age inside the youngest bands nearly as FRS employees' does, and falls with age over 55 as theirs does. The placebo changed none of this.

2. Households' HRAD at or below £250,000 from Table 8a receipts

Why

The property purchases stage (#1308, porting uk-data#573) allocates households' HRAD purchases by individuals' share of all Table 8a residential transactions (0.881) and value (0.882). #1308's review found this probably overstates households' purchases.

The residential nil-rate band was £250,000 for all of 2024-25. A purchase at or below it owes no standard-rate SDLT, so other buyers' receipts in that band are surcharge.

What changes

  • The nil-rate band from receipts: households' HRAD at or below £250,000 is Table 6b's band less other buyers' HRAD implied by their Table 8a receipts. The stage refuses an implied figure outside Table 6b's band or above other buyers' own transactions there.
  • Above the band, uk-data#573's shares stay. Solving for other buyers' HRAD from receipts goes negative above £1m. The module docstring says the shares probably still overstate households' purchases. Table 8a doesn't split out the 2% non-resident surcharge, so the nil-band figure, if anything, understates them (Vahid's review).
  • Targets: households' 2024-25 HRAD target falls from 186,589 transactions and £68.81bn to 151,928 and £63.71bn.

All figures in this section are HMRC's published Annual Stamp Tax Statistics 2024 to 2025, Tables 6b and 8a, as the pinned feed carries them.

Measurement

Households' purchases fit their new target, and no row moved by more than 5 points against the 28 October build. obr.sdlt fits within 0.25% after calibration in every build.

3. Drawn SPI employees' current pay at their twin's hours (uk-data#629)

Why

The SPI income stage keeps each drawn copy's FRS twin's hours and job, while stage 1 redraws the copy's pay from the 2022-23 tape. The tape's pay covers the tax year and holds part-year workers, so part-year pay could land on a current full-time twin, at an hourly rate far below the minimum wage. Such a copy fails the minimum-income tests of the 30 hours' childcare offer, Tax-Free Childcare and the benefit cap more often than the FRS rows it copies.

What changes

  • The floor: floor_spi_employment_pay_at_minimum_wage lifts a drawn copy's pay to its twin's usual weekly paid hours (paid_hours, Vahid's review) × 52 × the lower of the minimum wage for the twin's age and the twin's own hourly pay.
  • Who it touches: only drawn recipients other than reserved band carriers, whose twin has an employee main job, no trade and usual hours, and whose drawn pay is positive. FRS rows are never modified.
  • The tape's year of pay: the floored pay is current pay only. employment_income_annual_share keeps the HMRC income rows on the tape's pay.
  • Receipt: an employment_pay_floor receipt and an hourly-pay check. A figure resting on one or two source households is withheld as "<3", on the shared floor of 3 (Vahid's review).
  • Gate: uk_stage_hmrc_spi_income_spine_hourly_pay refuses a spine where an eligible copy stays below its floor, or where drawn employees sit below the minimum wage more often than FRS employees by more than two points.
  • The UC take-up gate is at 0.055 again for each family type and for the reachable pools (María's ruling of 2 November). With SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428's pool joining, every family type ends closer to DWP's count after the redraw.

Measurement

After the floor, drawn employees sit below the minimum wage for their age less often than FRS employees, within the gate's margin, and no eligible row stays below its floor. The HMRC employment-income bands end as without the floor, since their annual measures read the tape's pay.

B5: Carer's Allowance after salary sacrifice

  • What changes: the Carer's Allowance take-up stage runs after salary_sacrifice, so its earnings test reads pay after sacrifice. That puts it after the CGT copies, so its draws are keyed on lineage: a copy draws as the member it copies, and a source and its copies in one stratum count as one record of their joint weight. Original rows keep their draws.
  • Measurement (B-1 against T1426):
    • Only the Carer's Allowance stage differs, and every calibration row is unchanged.
    • The change turns off the claim flag only on CGT copies that salary sacrifice puts over the earnings limit, and the engine already paid them nothing.
    • So it makes the flag consistent with what the engine pays.

B12: Scottish owners' council tax bands

  • What changes: Scottish owners' bands are drawn, at the same uniform, from their council area's chargeable-dwellings mix (Scottish Government, September 2025, A to H), vendored from the pinned feed as uk/scotgov_chargeable_dwellings_by_band.json. Renters and the per-copy bill stay with the bill-level issue.
  • Measurement: it runs only in dense builds, so the national builds here can't measure it. It is measured in the next dense build.

AC3: two-child-limit births by the engine's cutoff test

  • What changes: the 15 two-child-limit rows read the engine's uc_is_child_born_before_child_limit (a date of birth before 6 April 2017) instead of a birth year of 2017 or later. A child born from January to 5 April 2017 no longer counts as affected.
  • Measurement (B-2): the seven rows whose measure reads the cutoff start 1% to 4% lower, and all 15 still fit within 0.5%.

B7: non-parental care and the two-child limit

  • What changes: frs_relationships sets child_limit_non_parental_caring_exception for a child whom no adult of their own benefit unit claims as a natural, adopted or step child (household grid codes 3 and 4 from the child, 7 and 8 from the adult), and whom no local authority placed there. This covers kinship and friends care.
    • Multiple births stay at the engine's default: the FRS has no twin field, and its birth dates are blank (María's ruling of 2 November).
    • The flag leaves the signed engine-default register.
  • Measurement (B-2): inert in this build. The flag reaches the calibrated dataset, but no flagged child is a third or later child born after the cutoff in a paid UC claim, so no row moves.

B10: SPI copies' contributory JSA and ESA within each benefit's own work limits

  • Why: spi_benefit_coherence zeroed contributory JSA on every SPI copy and left contributory ESA with the stage-2 forest's draw, which knows nothing of the unit's claim history. A contributory award follows the person's National Insurance record and incapacity, which the copy keeps; what the copy changes is its work.
  • What changes:
    • An SPI copy takes its FRS twin's contributory ESA within permitted work (María's ruling of 29 October; ESA Regs 2008 reg 45(4)): under 16 hours a week, and earnings up to policyengine-uk's esa_exempt_work_higher_earnings_limit at the frame's year, £183.50 a week in 2024-25.
    • It takes the twin's contributory JSA within JSA's own limits (Vahid's second review): under 16 hours a week (JSA Regs 1996 reg 51(1)(a)), and earnings below the age-related amount plus the £5 disregard. Reg 56's prescribed amount is that sum less 1p: £95.49 a week from 25 in 2024-25. The award is the lower of the twin's and the age-related amount less the earnings above the disregard (Jobseekers Act 1995 s. 4(1)(b)).
    • The amounts are policyengine-uk's own parameters at the frame's year. Earnings are employment income plus any self-employment profit, over 52 weeks. Hours are the twin's usual hours, which the copy keeps and the pay floor reads.
    • The operation, with its hours and award, is in the source manifest and the sources schema, and the graph hands the node the hours and the source households.
    • The stage gate checks the rule on every copy. The receipt counts support by distinct source household and withholds a weighted figure that rests on one or two.
    • Contributory ESA leaves the QRF tail-concentration surface, since no forest draw survives in it.
  • Measurement (B-3 against B-2, with ESA's limit for both awards):
    • On the copies, contributory ESA falls (the forest had given them more than their twins report) and contributory JSA returns at a small level.
    • Before calibration, obr.jobseekers_allowance's shortfall halves. The ESA rows start further below DWP's and OBR's: without the forest's extra ESA, the FRS base's own shortfall shows.
    • All of them fit after calibration, and dwp.esa_claimants comes inside 10%.
  • Measurement (b5j against B-5a, with JSA's own limits):
    • JSA's own limits leave the copies' contributory JSA where B-5a had it, so obr.jobseekers_allowance starts where it did and B-3's halving stands.
    • ESA's hours test takes contributory ESA off some copies, so the ESA rows start slightly lower.
    • All of them fit after calibration.

B9: student-loan balances to borrowers

  • Why: was_wealth splits the household's WAS student-loan balance before any plan exists, so it could sit on a member with no loan while the household's borrower held none. policyengine-uk caps a borrower's repayment at a positive balance.
  • What changes: once student_loans sets the plans, each household's balance moves to its members who hold one: by repayments where any of them repays, equally otherwise. A household with no borrower keeps was_wealth's split, and no household's total moves. The stage opens a reader-isolation boundary in the UK graph, since it rewrites a cell an earlier open-surface stage reads.
  • Measurement (B-3): the SLC repayment rows' starting estimates move by under 1%, and they all fit after calibration. Their large starting overshoot is not the balance's: the WAS balance is a small part of SLC's loan book, so few repayments are capped.

B4: HICBC as measured diagnostics

  • What changes: two national rows from HMRC's Child Benefit statistics (August 2025, Table 16): the Self Assessment taxpayers liable to the charge in 2024-25 and its revenue, both provisional (310,000 and £360m). They are measured at the model's 2025 under a signed hold, since the thresholds are the same in 2024-25 and 2025-26, and held out of the solve by two signed exclusions (María's ruling of 3 November).
  • Measurement: the model's charge sits well above HMRC's, and more than half of it on SPI copies: too many high-income parents still receive Child Benefit. Families in payment and obr.child_benefit fit, so it is not the number of claims. The fix belongs in the SPI channel and is UK: HICBC runs well above HMRC's count, mostly on the SPI copies (too many high-income parents in payment) #1508. Neither the SPI prior shape nor PR 2b moved it.

B3, PR 2b (J2 and K): the NTS cars

  • What changes: lcfs_consumption reads the National Travel Survey's cars (SN 5340, survey years 2022 to 2024, at the interview weight W3).
    • J2: each car is zero-emission at DfT's licensed-car share (VEH1103) times its household income fifth's relative risk among the cars NTS households keep, normalised so the national share is unchanged.
    • K: household road fuel keeps ONS 07.2.2's level, and each fuel is levelled to its share of household-paid spend: HMRC's litres times DESNZ's cars share times the share of car miles a household pays for.
    • obr.fuel_duties_cars stays held (María's ruling of 29 October), its reason restated with the household-paid shares.
  • Measurement (B-5a against t1426m): only lcfs_consumption differs, and no row moves more than 5 points. The top income fifth now holds the most households whose cars are all zero-emission, and household road fuel moves from petrol to diesel at the same total.

B3: vehicle excise duty inputs

  • What changes: the same stage writes policyengine-uk's five car inputs for each adult who keeps a household car: fuel type, CO2 emissions, engine size, first registration date and the exemption flag (María's design of 3 November).
    • A household's own cars go to its adults, oldest first. Company cars are left out, and cars beyond the adults are dropped and counted in the receipt.
    • Each car's attributes come from an NTS car of the same income fifth and age band with linked CO2 bands, reweighted so the fleet keeps its age mix, since the newest and oldest cars carry the linked record less often. A zero-emission car has no engine size.
    • DLA higher-rate mobility and PIP enhanced mobility make the car exempt. The list price stays the engine's default. Registration dates are ISO strings.
    • The inputs join the export allow-list, so the battery's digests move.
  • Measurement (B-5b against b5sp): only lcfs_consumption differs, and the calibration is identical: no calibration target reads VED. policyengine-uk counts VED in household tax, so household net income falls by each household's duty; HBAI income does not change. An engine test checks each duty regime's rate on the step's own dates.
  • Limits:

Tests

CI at 1ac980d1d: all ten checks pass.

Local runs on the commits since 22703ee0d:

  • B10: test_uk_spi_benefit_coherence.py (34 tests: JSA's prescribed amount, award cut and hours, ESA's hours, the receipt's floor), the engine test of the limits at 2024 and 2025, and the node's declared reads in test_uk_graph.py.
  • On the merged tree: the gate battery pins, microcosm-data's contract, the transport bindings' identity, the country spec and the spec-engine inventory and field usage (629 tests), and B10's other readers (1,004 tests). None fail.
  • Generated surfaces:
    • The H2 parity fixture regenerates byte-identically.
    • The release input coverage manifest, the CI test plan and the spec-engine coverage are current.
    • The US seed protocol and seed map match main's pins.
  • Earlier: VED's tests (test_uk_nts_car_attributes.py, the LCFS and source-stage tests, and the engine test of the duty regimes), and the full engine-free suite at 2b49a00ac (351 files, 13,226 tests; two fail only on this machine: one expects the UK extra to be absent, and the Orrery reconstruction test meets a local Node error).

New and changed test files, against main:

  • New:
    • tests/engine/uk/test_uk_spi_permitted_work_limit.py reads ESA's and JSA's limits from policyengine-uk at 2024 and 2025;
    • tests/engine/uk/test_uk_vehicle_excise_duty_inputs.py charges each VED regime on the step's own dates;
    • test_uk_nts_car_attributes.py and test_uk_spi_prior_concentration.py.
  • Engine tier: test_uk_spi_income.py.
  • Engine-free, UK:
    • test_uk_carers_allowance_take_up.py, test_uk_frs_relationships.py, test_uk_frs_spine.py and test_uk_graph.py;
    • test_uk_lcfs_consumption.py, test_uk_measure_simulation.py, test_uk_measurement_period.py and test_uk_population_targets.py;
    • test_uk_property_localisation.py, test_uk_property_purchases.py, test_uk_receipt_counts.py and test_uk_source_stages.py;
    • test_uk_spi_benefit_coherence.py, test_uk_spi_income.py, test_uk_spi_pension_age.py, test_uk_spi_rank_transport.py and test_uk_spi_spine.py;
    • test_uk_spine_acceptance_receipt.py, test_uk_stage_health.py, test_uk_student_loans.py and test_uk_target_references.py;
    • test_uk_uc_capital_coherence.py, test_uk_uc_target_measurements.py and test_uk_weighted_integrity.py.
  • Engine-free, shared: test_country_spec.py, and microcosm-data's test_contract.py.
  • Test support: uk_atomic_support_fixtures.py and uk_student_loans.py.

Follow-up

Generated surfaces: source_stages.json (in step with sources.yaml), the sources schema, the release input coverage manifest, the H2 parity fixture, B4's target references and membership, and the gate digests mirrored in microcosm-data's contract.

🤖 Generated with Claude Code

juaristi22 and others added 6 commits October 29, 2026 19:35
…628)

The SPI tape gives age only as a band (AGERANGE), so the age drawn for each
donor inside its band carries no income pattern: the stage-1 forest learned
none, and twins in one band faced conditionals that differed only by that
noise. The tape and reference forests now read the band code instead: the
tape's own code, composite records (-1) as a band of their own, and the band
of a frame person's whole-year age. at_or_over_spa keeps the State Pension
split inside a band. The cross-fit fold cells key on the band, since they key
on the predictor values. The drawn ages stay on the donor for the band pools.

Regenerated: source_stages.json (in step with sources.yaml), the release
input coverage manifest and the H2 parity fixture. The SPI engine tests read
the band and count the reference forest once per cross-fit fold in use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e 8a receipts

The 2024-25 residential nil-rate band was GBP 250,000 all year, so a purchase
at or below it owes no standard-rate SDLT and other buyers' Table 8a receipts
there are surcharge. At Table 6b's gross HRAD receipts per transaction and per
pound they imply about 48,100 HRAD transactions and GBP 7.05bn for other
buyers, where individuals' share of all transactions, uk-data#573's rule, left
them about 13,400. Households now take Table 6b's band less that; above it the
shares stay (solving from receipts there goes negative above GBP 1m). The
household bands become the price mix.

Households' target: 151,928 transactions and GBP 63.71bn (was 186,589 and
GBP 68.81bn), above uk-data#573's lower bounds; the lowest band carries 43% of
households' purchases (was 54%). Published HMRC figures only.

Regenerated: source_stages.json (in step with sources.yaml) and the release
input coverage manifest. The H2 parity fixture is regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 1 of hmrc_spi_income_spine draws a recipient's annual tape pay on
age, sex, region and the bridge predictors, not on hours, and the tape
holds part-year workers. A copy whose FRS twin reports usual hours in an
employee job could therefore draw a year's pay below what those hours earn
at the minimum wage, and fail the 30 hours' and Tax-Free Childcare
minimum-income tests more often than FRS rows.

A new floor_spi_employment_pay_at_minimum_wage operation lifts the drawn
hmrc_spi_pay leaf to weekly usual hours (hours_worked read back at
365.25/7) times 52 weeks times policyengine-uk's
gov.hmrc.minimum_wage.non_apprentice rate for the twin's age, read at 30
April of the build year as the engine reads it (2024-25: 11.44 from 21,
8.60 from 18, 6.40 below). It runs after the stage-1 draw, the band-donor
resample and the State Pension leaf guard, and before employment_income
is derived from the leaves and stage 2 conditions on it; the transport
receipt keeps the unfloored draws' uniforms. Only drawn recipients other
than band carriers, whose twin has an employee main job, no trade and
usual hours, and whose drawn pay is positive, are floored. Tape PAY is
taxable pay; PAY alone is compared with the gross minimum because it is
the leaf of the engine's gross employment_income input.

The operation is declared in sources.yaml, its source_stages.json
projection, the sources schema, the operation vocabulary and the HMRC
income contract, and checked against a reviewed declaration. The stage
evidence gains an employment_pay_floor receipt: source households
rounded, weighted figures "<10" on one to nine source households, no
row's value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The employment pay floor receipt gave the floored weight, the share
floored and the amount added beside the eligible weight, each suppressed
only on the floored side's support. Any of them with the eligible weight
gives back the weight of the eligible rows the floor left alone, which
could rest on one to nine source households. They are now "<10" whenever
either side rests on one to nine, as the Carer's Allowance take-up
receipt gives its share; the mean uplift alone does not give a side back
and keeps the floored side's rule. The receipt declaration in
sources.yaml, source_stages.json and the reviewed spine mapping says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, gate hourly pay

uk-data#629: drawn SPI employees keep their FRS twin's hours while the tape
redraws their pay, and the tape's pay is the tax year's, so part-year tape
pay landed on current twins at very low hourly rates. On top of the floor
from uk-cc-spipay:

- The floor rate is the lower of the minimum wage for the twin's age and the
  twin's own hourly pay (its employment income before the draw over its
  annual hours), so no copy is held to a higher rate than the FRS person it
  copies; hours now read back at 52 a year (uk-data#531).
- The floored pay is the row's current pay only: its annual share becomes
  the unfloored over the floored employment income, so the annual measures
  (the HMRC income rows) still read the tape's pay.
- The receipt adds an hourly-pay check: the drawn employees' weighted share
  below the minimum wage before and after the floor, beside the FRS
  employees' share, and the eligible rows still below their floor; a share
  is withheld beside a side on one to nine source households.
- A release-blocking stage gate, uk_stage_hmrc_spi_income_spine_hourly_pay,
  holds the floor invariant and the drawn share to the FRS share plus two
  points; it joins the spine gate scope, and the battery and contract
  digests move with it.

Regenerated: source_stages.json (in step with sources.yaml), the sources
schema, the release input coverage manifest and the H2 parity fixture. The
floor's engine test gives each twin and its copy one age and pay, reads hours
at 52 weeks, and checks the twin cap, the kept year of pay and the employer
contribution that follows the floored pay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-29 (#1095). The SPI employment pay floor gives
fewer low-paid SPI copies Universal Credit, so single people's caseload
before the take-up redraw falls close to DWP's, and the redraw's usual
downward move reads as about twice the 0.055 allowance; after the redraw
single people end a little below the spine without the floor, all in the
middle award range. The shortfall of middle and higher awards that the
redraw cannot fill is the award fix's (#1095's UC awards below the top
band), and the tolerance returns to 0.055 when it lands. The one tolerance
also bounds the reachable pools, which realize DWP's count far more closely.

The contract re-pins the gate battery's policy, manifest and fingerprint
digests and the release certification's spine part; the stage-health test
pins 0.12 and adds the floor's pattern.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22 juaristi22 changed the title UK #1095 A/B fixes: the SPI age band (uk-data#628) and households' HRAD from Table 8a receipts UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) Oct 29, 2026
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 thanks, this is my review of 457af0274.

Verdict: the three items look sound, with four should items and a nit. The age band is the right fix for #628. The nil-band arithmetic reproduces the body's numbers: £265m over £625m/113,500 is about 48,100 transactions, and the target falls by about 34,700. The pay floor runs where it should, after stage 1 and the uprating and before employment_income, stage 2 and pension coherence, with the twin cap and the kept year of pay. Nothing here blocks the A/B arms.

Findings

  1. should: composites (-1) sit next to 16-24 on an ordinal predictor. spi_age_band is numeric, not one of SPI_STAGE1_CATEGORICAL_PREDICTORS (spi_income.py:2252, 2623). So any split above -1 that doesn't separate -1 from 1 puts composite records in the leaves 16-24 queries land in. Before, composites drew an age across 16-70, so they spread over every band. Now they can only mix with the youngest one, and composites are the tape's high-income amalgams. Please either keep composites out of the stage-1 training set, one-hot the band, or check on the arm that 16-24 draws' top tail is not higher than on main. A line in the predictor rule saying which would do.
  2. should: the new floor receipt hardcodes ten. spi_income.py:936, 940, 951-952 and 961 use < 10 and "<10", and the gate docstring says "fewer than ten" (stage_health.py:1425). UK: take up #1095's actionable items on policyengine-uk 2.165.1 #1422 moves every UK floor onto receipt_counts.UK_DISCLOSURE_FLOOR. Whichever lands second should move these onto the constant and the helpers (small_cell_share, rounded_count), or this receipt will be the only one still on ten. The gate's "a withheld share is not judged" branch then follows too.
  3. should: one tolerance loosens two checks. uk_stage_uc_take_up goes from 0.055 to 0.12 (gates.json:633). That widens the "no further from DWP than before the redraw" leg, which the floor needs. But the same number bounds the reachable pools, which the notes say realize DWP's count within 0.0015. A second parameter for the pools, left at 0.055 or tighter, would keep that check meaningful while the other is wide. Please also name the issue that tracks the return to 0.055, so the revert isn't left to memory.
  4. should (a question): does Table 8a's nil-band receipts figure for other buyers include the 2% non-resident surcharge? If it does, part of those receipts is not HRAD surcharge. Then other buyers' implied HRAD is too high and households' too low (property_purchases.py:473). The direction is the safe one, and it may be small. One sentence in the module docstring saying which way it goes would do.
  5. nit: paid_weeks (spi_income.py:893) is annual paid hours (hours / 52 * 52), not weeks. paid_hours would read better.

What I ran. I read the full diff at 457af0274 and checked the HRAD figures by hand. I also checked that the SPI leaf reconciliation runs on the donor tape, before the floor, so the floor can't break it. CI is green at this head. I didn't run tests locally or build anything, and I didn't check the private A/B figures.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Oct 31, 2026
Vahid's review of #1422 (its second and third points).

- Since the move to a floor of three, rounded_count printed three to nine
  exactly, while extreme_count and support_band print "3-9" and the
  module's own rule is that no exact count narrows a small group by
  difference. It now reads "3-9" there too. Two tests that read an
  incidental exact count follow.
- The module docstring and the changelog say that the SPI Public Use
  Tape's two conditions (SN 9422) are unchanged by the move to three.
- A guard test fails when a UK runtime module, the UK spec, the gate notes
  or a tool still writes the old "<10" label, so that a receipt left on
  the floor of ten cannot land unnoticed. #1426's SPI pay-floor receipt
  writes it today; whichever of the two lands second moves it onto
  UK_DISCLOSURE_FLOOR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 15 commits October 31, 2026 11:22
#1426 now builds on #1422, which moves every UK disclosure floor to three
(receipt_counts.UK_DISCLOSURE_FLOOR) and regenerates the national surfaces
on Chronicle 53815b2 with policyengine-uk 2.165.1.

- The SPI pay-floor receipt withholds through UK_DISCLOSURE_FLOOR and
  UK_WITHHELD: a figure on one or two source households reads "<3" where it
  read "<10" (Vahid's review of #1426). Its description in spi_spine.py,
  sources.yaml and source_stages.json, the hourly-pay gate's note and
  docstring, and its tests follow. Two test cases shrink their small side to
  two households, so they still test the withholding.
- Conflicts: the SPI income imports (both sides kept), the stage-1 fold key
  (#1426's age band) beside #1422's savings-interest method text, and the UC
  take-up gate test.
- Regenerated: the release input coverage manifest. The gate battery and
  spine-part digests are re-pinned.
- The H2 parity fixture takes #1422's side here; a later commit regenerates
  it on the merged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hours / 52 * weeks_in_year is a year's paid hours, not weeks (Vahid's
review of #1426).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uk_stage_uc_take_up's one tolerance bounded two checks: how far a family
type's caseload may move from DWP's across the redraw, which the SPI pay
floor needs at 0.12, and how closely the reachable pools realize DWP's
count, which the draw meets within 0.0015. The pools now keep 0.055 as
maximum_reachable_pool_deviation while maximum_caseload_deviation stays at
0.12 until the UC award fix of microcosm#1095 (Vahid's review of #1426).
The gate battery and spine-part digests are re-pinned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spi_age_band was an ordinal predictor, so any split that did not separate
the composite records' code (-1) from 16-24 put the tape's high-income
amalgams in the leaves 16-24 queries land in. The band is now one of the
stage-1 categorical predictors, one-hot encoded with gender, region and main
source, so composites are a category of their own (Vahid's review of #1426).
The predictor rule, the spec's categorical list, the changelog and the
coverage manifest follow; the cross-fit fold cells still key on the band's
value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Table 8a's receipts do not split out the 2% non-resident surcharge, which a
non-resident buyer pays on top at or below GBP 250,000. Read as HRAD
surcharge, it raises other buyers' implied HRAD and lowers households', so
households' nil-band purchases are, if anything, understated (Vahid's review
of #1426).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the gate digests conflict; they are re-pinned on the merged gates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tools/graph_uk_spine_fixture.py on #1426's items over #1422's branch, after
the review fixes; --check regenerates it byte-identically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stage-1 queries carry the age band's one-hot columns, not the band
itself, and the pay floor's four floored households read "3-9".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the gate digests conflict; they are re-pinned on the merged gates.
#1422's new floor-of-ten guard passes on #1426's SPI pay floor and age
band code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gate digests and the coverage manifest's source digest conflict; they
are re-pinned and regenerated on the merged gates and spec. #1422's
broader floor-of-ten guard passes on #1426's SPI code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixture conflicts, as #1426's SPI stages differ from #1422's; it is
regenerated on the merged tree, --check reproduces it byte-identically, and
the H2 parity test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. The exact-count floor guard passes on #1426's SPI and HRAD
code, as do the stage, ledger and reference tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 2 commits November 1, 2026 15:48
No conflicts. The stricter floor guard passes on #1426's SPI and HRAD
code, as do the ledger, reference, stage and graph tests. The gate
digests and the coverage manifest are unchanged, and the H2 fixture
regenerates byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. The broader floor guard passes on #1426's SPI and HRAD
code, as do the ledger, gate, reference, stage and graph tests. The
gate digests and the coverage manifest are unchanged, and the H2
fixture regenerates byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1422 landed (11c1f4e), followed by #1430's company cars and #1432's
default load. Only pinned and generated files conflicted. The gate
battery and spine-part digests are re-pinned on the merged gates spec
(five move), and the release input coverage manifest and the charter-H2
fixture are regenerated; the fixture checks byte-identical.
policyengine-uk stays at 2.165.1, the lock main and #1428 share, and the
feed pin moves to main's 08962f1.

The H2 parity test and 86 related engine-free files pass (5,682 tests,
53 skipped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 Round 2 at 2b49a00ac. I re-checked my five points from 457af0274, read your two replies, and read the Phase B diff against main (312dc4d6b): B5, B7, AC3, B9, B10 and B12. I checked B10 and B7 against the legislation. I ran the PR's engine-free tests for the stages it touches (spi_benefit_coherence, student_loans, frs_relationships, carers_allowance_take_up, stage_health, property_localisation, uc_target_measurements): 281 passed. I didn't build anything or look at the private figures.

Earlier points

  1. Composites next to 16-24: fixed. spi_age_band is the first of SPI_STAGE1_CATEGORICAL_PREDICTORS (spi_income.py:185-190), so _encode_predictor_frames one-hots it. Tape and query codes are both float, so their dummy columns match (spi_income.py:2663, 3028-3030). The query side never produces -1, so the composite column is always 0 for frame people, as it should be.
  2. The floor receipt's ten: fixed. It now uses UK_DISCLOSURE_FLOOR/UK_WITHHELD throughout (spi_income.py:984-1042).
  3. One tolerance, two checks: fixed. maximum_reachable_pool_deviation is separate (stage_health.py:1542-1546, 1586) and both values are 0.055 (gates.json:655-656). See below on your follow-up.
  4. Non-resident surcharge: fixed. The module docstring says which way it goes (property_purchases.py:39-44).
  5. paid_weeks: fixed, now paid_hours (spi_income.py:939).

On your point-3 follow-up. I agree the caseload check can go back to 0.055 and nothing is left to track. One caveat on where the evidence comes from. The gate note cites the T1426 spine at 97fd4d0e4 (gates.json:658). That merges #1428 at 329a99e71, before #1428's SPI banding commit 2dbf7cf56, so the "every family type ends closer after the redraw" reading was taken on the old SPI stage. The body says B-1 to B-3 and t1426m all pass every spine gate, so the 0.055 gate has also passed on the current base. Please name t1426m's commit in the gate note next to 97fd4d0e4, since that is the spine the gate now runs on. If the receipt shows how far inside 0.055 the closest family type sits on t1426m, one figure there would also help. The body already says the T1426 and B-chain A/B measurements were taken against the pre-2dbf7cf56 control, and that ctl-312d/t1426m measure the PR on the current base, so I have nothing to add there.

New findings

  1. Should: B10 gives contributory JSA ESA's permitted-work limit, which is about twice JSA's own earnings limit (spi_benefit_coherence.py:517-530). Contribution-based JSA needs the claimant's earnings not to exceed the prescribed amount (Jobseekers Act 1995 s.2(1)(c); JSA Regs 1996 reg 56: the age-related amount plus the earnings disregard, less 1p). For a claimant aged 25 or over in 2024-25 that is £90.50 + £5 − £0.01 = £95.49 a week. Below that, s.4(1)(b) reduces the award by earnings above the disregard. Failure case: an SPI copy aged 30 earning £7,800 a year (£150 a week), whose twin reports £4,706 of contributory JSA (£90.50 × 52), keeps all £4,706 at this head. Under the law it would get nothing, because £150 is above £95.49. At £50 a week it keeps £4,706 where the award would be about £2,366 a year (£45.50 a week). The docstring says the ESA limit is only a ceiling for JSA (spi_benefit_coherence.py:66-68). But the body puts part of the halving of obr.jobseekers_allowance's pre-calibration shortfall down to B10, so the gap matters here. Fix: test jsa_contrib_reported against reg 56's amount at the copy's age (the engine's JSA age-related amount + £5), and either reduce the award by earnings above £5 or zero it above the prescribed amount. Or, if María keeps the ESA limit for both, have the receipt report the copies' weighted JSA between £95.49 and £183.50 a week, so the JSA row's movement can be read with that in mind.
  2. Nit: B10's "the test reads no hours" doesn't quite hold. spi_benefit_coherence.py:66 says the SPI tape has no hours. But the copy carries its twin's usual hours, which the pay floor reads (SPI_EMPLOYMENT_PAY_FLOOR_HOURS_COLUMN). ESA reg 45(4) also needs fewer than 16 hours a week. Either test the hours too, or reword the docstring to say why the twin's hours aren't used.
  3. Nit: B7's flag is an upper bound on the legal exception (frs_relationships.py:204-218). UC Regs 2013 Sch 12 para 4(2) (before its revocation from 6 April 2026) needs a friend or family carer to meet one of conditions (a) to (h): a child arrangements order, special guardianship, guardian's allowance, a kinship care order, guardianship, a permanence order, or taking the child on where they would otherwise likely be looked after. The FRS grid shows none of these. So every child with no parent or step-parent in their benefit unit is flagged, including informal arrangements that wouldn't qualify. It is inert in this build, so one line in the rule's comment calling it an upper bound would do.

Checked and fine

  • B5: lineage_roots refuses an unmapped or duplicated lineage. Originals keep their person_id draws. lineage_balanced_positions treats a source and its copies in one stratum as one record, and the within-one-record assert is widened to match.
  • B9: allocate_balance_to_borrowers keeps each household's total (with a hard check), leaves households with no borrower or no balance alone, and splits by repayments, or equally where nobody repays.
  • AC3: the measure now reads uc_is_child_born_before_child_limit, so a January to 5 April 2017 birth no longer counts as affected.
  • B10 limit: 16 × £11.44 = £183.04, rounded up to £183.50 (ESA Regs 2008 reg 45(4) and (9A)), which matches the test constant.
  • B12: the loader refuses a council area missing a band or repeating one, Northern Ireland and renters keep their bands, and the Scottish receipt is separate.

CI at 2b49a00ac: all ten checks pass (engine-free and engine-uk on 3.13 and 3.14, engine-us, integration-uk, lint, wheels).

Verdict: my earlier points are all fixed and I agree with the 0.055 return. B10's JSA limit is the one item to settle before the JSA row's movement is quoted.

🤖 Generated with Claude Code

juaristi22 and others added 6 commits November 3, 2026 14:10
…ts household share (#1095 B3: PR 2b J2 + K)

The LCFS consumption stage now reads the National Travel Survey's cars (UKDS SN 5340, End User Licence; the vehicle and household tabs, survey years 2022-2024, at the interview weight W3), as the 16 October plan designed J2 and K.

- J2: each car is zero-emission at VEH1103's licensed-car share times its household income fifth's relative risk among the cars NTS households keep, normalised over the frame's cars so the national share is unchanged (rule per_car_zero_emission_share_by_income_quintile). The frame ranks household_gross_income over the engine's before-housing-costs equivalence scale, as the NTS ranks gross income McClements-equivalised. A car is zero-emission in the DVLA-linked 0 g/km band; a car without a linked band is left out. The uniforms are unchanged, so only each household's threshold moves.
- K: level_road_fuel keeps ONS 07.2.2's household level but levels each fuel to its share of household-paid spend (mix rule household_paid_litres): HMRC's litres times DESNZ's cars share times the share of car miles a household pays for (a kept car's private and commuting miles), at the 2024 pump prices. A car whose miles the release does not split is left out; a blank is missing, never a zero. The road_fuel_level gate rechecks each fuel against the mix it recomputes from the vendored rows.
- Spec: both operations declare the same nts_cars block, which the code refuses to see differ; the NTS household tab joins lcfs_consumption under the bus stage's pin, and the vehicle tab is a new pinned private input (--nts-vehicle-tab).
- obr.fuel_duties_cars stays held, its reason restated with the household-paid shares (María's ruling of 29 October 2026).
- Surfaces: the schema, the source-stage projection, the graph's split sources, the spine tool, the fixture kernels, the H2 fixture (a synthetic vehicle table, W3 on its households), the coverage manifest, and the kept-tens register (the pooling floor of 10 is statistical).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spi_support_channel stage draws its 10,000 FRS households uniformly
and gave every copy in a region x pension-age stratum the same prior
weight, which reproduced the FRS respondent mix rather than the design's
population (young families, single adults and one-earner couples over-
represented, households of three or more adults under-represented).

Copy j of stratum s now takes M_s * share_s * f_j / sum_k f_k, with f the
copy's source-household design weight (the incoming raw FRS GROSS4) over
the stratum copies' mean, damped to the square root. Each stratum's
channel mass stays exactly its declared share and the pension-age share
stays 0.2; the hosts and their incomes do not move. The shape is declared
on allocate_zero_weight_prior_mass (prior_shape: design_damped,
prior_shape_exponent: 0.5; design_bounded with prior_shape_bounds is also
understood), validated by the schema and refused by the stage parser if it
drifts from the reviewed shape. A design shape requires DESIGN input
weights, so shaping by fitted weights (microcosm#630) is refused, and the
#420 guard that every copy's source household has a positive design weight
is restored.

The square root was chosen by re-solving the release calibration on the
same hosts: the [0.5, 2] clip put more rows at the weight-ratio cap than
equal weights and widened the GBP 125,000-150,000 band's standard error
past the damped shape's; the damped shape kept fewer cap rows, the highest
Kish ESS and loss within 2.2 percent of equal weights. The stage receipt
records the shape and its adjustment's 10th, 50th and 90th percentiles
with banded counts (no minimum or maximum), and the mass-change record
names the shape, so the release input-coverage manifest needs its pin-pass
regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit d6c0f25)
…ed reason

The prior-shape receipt gave its percentiles and floored clipped shares
whatever the channel's size, so a small scale-ladder rung of one to nine
copies would report statistics resting on fewer than ten source
households. It now writes "<10" in their place, as the take-up receipts do.

The coverage-manifest test pinned the old allocation sentence, so it
would have failed even after the pin pass regenerates the manifest; it
now expects the sentence naming the design-damped shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 1d22dd5)
…#1095 residual 12)

d6c0f25 and 1d22dd5 left #1338 before it merged (they broke that build's property_purchases guard on its older base) and stayed on its pre-rebase branch; María's ruling of 3 November 2026 brings them back here, to be measured against B-5a.

- The prior-shape receipt withholds its spread and shares for a channel of one or two copies ("<3"), on the shared floor of 3 (María's ruling of 29 October 2026), where it had used ten; its banded counts follow receipt_counts.
- The pin pass: the release input coverage manifest (the support stage's mass-change reason now names the design-damped shape) and the H2 fixture regenerate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1338 review, item 4)

tools/diagnose_uk_spi_prior_concentration.py reports, on any UK spine or calibrated dataset, the cell the SPI prior shape is meant to help: households with a benefit unit that has a child aged 0 to 4 and an adult whose adjusted net income is above £100,000. It gives the cell's weighted households, its Kish effective size over households and over FRS source households (a source and its copies as one unit), the share of its weight on its ten largest source households, its share on SPI-channel copies and its support in source households, rounded; a cell on one or two source households gives none of them. Development diagnostic, aggregate JSON only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s overflow the engine's

On a spine, household and benefit-unit ids run past the engine's 32-bit
entity ids, so the engine's household_id wrapped and the cell's households
were not found in the household table. The receipt now takes memberships,
source households and the SPI flag from the dataset's own tables and only
values from the engine (age, adjusted net income, household weight), in the
tables' row order, which it checks on the stored ages. On a calibrated
dataset the release ids are 1 to N and in table order, so there the engine's
ids and the tables' agree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 2 commits November 3, 2026 18:15
…1095 B3: B-5b, VED)

No stage wrote policyengine-uk's car inputs, so every household's vehicle
excise duty was zero. The lcfs_consumption stage gains a declared
assign_car_attributes_from_nts step (nts_car_attributes.py) on the same NTS
kept cars as J2 and K:

- Which cars: a household's vehicles less its members' company cars (an
  employer pays their duty) go one to each other adult, oldest first. A car
  beyond the adults is left out and counted (policyengine-uk holds one car a
  person).
- Which fuel: a household the fuel flag leaves without road fuel keeps
  zero-emission cars only; one that buys fuel holds a fuel-burning car first,
  and each further car is zero-emission at its own per-car share.
- Which car: a kept NTS car of its class in the household's income fifth,
  drawn by keyed uniforms at its interview weight scaled to its fifth and age
  band's kept cars, since fewer of the newest and oldest cars carry the
  registration record the CO2 bands come from. A fifth and age band with kept
  cars but no linked one refuses.
- The inputs: car_fuel_type (a diesel registered from 2021 meets RDE2),
  car_co2_emissions (the band's midpoint on the scale its duty regime reads),
  car_engine_size (none for a zero-emission car), car_first_registration_date
  (1 October 2025 less the car's age, drawn within its NTS band, as an ISO
  date) and car_is_ved_exempt (DLA's higher-rate or PIP's enhanced mobility
  component exempts the member's own car, or the oldest car-holder's in their
  benefit unit). car_list_price keeps the engine's default.
- Surfaces: the spec operation and notes (sources.yaml and
  source_stages.json), the schema branch and allowed kinds, the graph's person
  cells, the stage's person outputs (UK_LCFS_CONSUMPTION_PERSON_OUTPUT_COLUMNS,
  as frs_take_up's), the release export's allow-list in terminal_gates and
  gates.json (so the battery's policy, manifest and fingerprint and the
  release-cut part digests move in the microcosm-data contract), the coverage
  manifest, and the H2 fixture (synthetic vehicle columns; the audited string
  columns gain the fuel type and the date, and the doc's count is computed).
- An engine test holds the step to policyengine-uk: the inputs and their
  defaults, the regimes' start dates, each band midpoint inside its engine
  band, and each regime's duty charged on the step's ISO dates.

VED is measured, never bound: policyengine-uk 2.165.1 does not add it into
household tax, and no target carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… of 3 November)

Reverts 0702f7a, 0f5be84 and 674b8a1. The square-root shape left
the fit and the over-£100,000 young-child cell's concentration where
they were (b5sp against b5a), and moved HICBC liability by about 3%.
The SPI stages, their tests, the stage's schema branch and the spec
operation return to 6fca272; the vehicle excise duty step stays. The
release input coverage manifest and the H2 spine fixture are
regenerated (the fixture regenerates byte-identically on --check).
The concentration tool stays: it sizes the cell on any spine or
calibrated dataset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 4 commits November 4, 2026 00:04
…' hours (Vahid's review of #1426, round 2)

B10 gave contributory JSA ESA's permitted-work limit, about twice JSA's
own. A copy now keeps its twin's contributory JSA only under 16 hours a
week (JSA Regs 1996 reg 51(1)(a)) and below the age-related amount plus
the earnings disregard (reg 56's prescribed amount is that sum less 1p),
at the lower of the twin's award and the age-related amount less the
earnings above the disregard (Jobseekers Act 1995 s. 4(1)(b), reg 80).
ESA keeps permitted work and gains its 16-hour test (ESA Regs 2008 reg
45(4)). The amounts are policyengine-uk's gov.dwp.JSA.contrib and
remunerative-work parameters and ESA's hours limit at the frame's year,
beside esa_exempt_work_higher_earnings_limit; the hours are the twin's
usual hours, which the copy keeps and the SPI pay floor reads.

The within-limit receipt now counts support in distinct source
households, banded, and withholds a weighted figure that rests on one or
two of them, or whose before and after would give back a change on one
or two (receipt_counts). It gave row counts.

Also from the review: B7's flag is called an upper bound on the
exception, since para 4(2)'s conditions are not in the FRS grid, and
the UC take-up gate's note names t1426m (2b49a00), where single
people, the family type that moves least, end 0.016 of DWP's count
closer after the redraw.

The stage spec and its schema branch (hours, weekly_hours_limit,
award), the gate notes, the battery's manifest and fingerprint and the
spine part digest (mirrored in microcosm-data), the release input
coverage manifest and the H2 fixture follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main was red on the transport bindings' module count (#1342, counted by
#1495) and the US spec-engine seed pins (re-pinned by #1496), which is
what failed #1426's engine-free and engine-us jobs. The merge is clean.
On the merged tree the US seed protocol and seed map match main's pins,
the gate battery and certification pins match their mirrors, and the UK
release input coverage manifest, the spec-engine coverage and the CI
test plan are current.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ests read (#1095 B10)

The node gets only its declared cells, and 0cddc84's work tests read
the twin's hours_worked and its receipt counts by household_source_id,
so the b5j spine stopped at the node. The stage now declares both, and a
test holds its declaration to the work tests' columns, as the
property_components test does since the 13 October twin build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…reads

8607310 gave the node hours_worked and household_source_id, so the
fixture's spi_benefit_coherence inputs gain those two columns; no other
node moves, and main's merge moves none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi thanks for round 2. Your finding, both nits and the gate-note caveat are addressed, and pushed at 1ac980d1d. Main is merged in at 79a533c72 for its own CI fixes.

Your point-3 follow-up (the gate note). The UC take-up gate's note now names t1426m (#1426 at 2b49a00ac) next to the T1426 spine, since that is the spine on the current base. On t1426m, single people, the family type that moves least, end 0.016 of DWP's count closer than before the redraw. The gate allows 0.055 further.

1. Contributory JSA now has JSA's own limits (spi_benefit_coherence.py, _contributory_work_limits).

  • A copy keeps its twin's contributory JSA only if it works under 16 hours a week (JSA Regs 1996 reg 51(1)(a)) and earns below the age-related amount plus the £5 disregard. Reg 56's prescribed amount is that sum less 1p.
  • Within those limits, the award is the lower of the twin's and the age-related amount less the earnings above £5 (s. 4(1)(b)). Above them it is nil.
  • The amounts are policyengine-uk's own parameters at the frame's year (gov.dwp.JSA.contrib, gov.dwp.JSA.remunerative_work.claimant_hours): £71.70 under 25 and £90.50 from 25 in 2024-25, so the limit from 25 is £95.49 a week. An engine test reads them at 2024 and 2025.
  • Your two cases are tests. At £150 a week the copy gets nothing. At £50 a week it gets £45.50 a week, £2,366 a year.
  • ESA keeps its permitted-work limit, now with its hours test (below).
  • Measured in b5j against B-5a, the build before it on the same base: JSA's own limits leave the copies' contributory JSA where B-5a had it, so obr.jobseekers_allowance starts where it did and the halving the body puts down to B10 stands. ESA's hours test takes contributory ESA off some copies, so the ESA rows start slightly lower. All of them fit after calibration. The build passes all seven gates with 98.9% of the 1,339 rows within 10% (98.7% in B-5a), and no row moves more than 5 points.

2. Hours. Both awards now read hours. The copy keeps its twin's usual hours (hours_worked over 52 weeks), which the pay floor already reads as the copy's. JSA needs under 16 hours a week (reg 51(1)(a)) and ESA under 16 (ESA Regs 2008 reg 45(4)). The docstring says so, and adds that neither award is reduced for the copy's pension payments.

3. B7's flag is now called an upper bound on the exception in the rule's comment (frs_relationships.py). It names para 4(2)'s conditions, which the FRS grid doesn't show.

Also in B10: the within-limit receipt now counts support by distinct source household and withholds a weighted figure that rests on one or two of them, as the other receipts do (receipt_counts). It gave row counts before. The graph now hands the node the hours and the source households: the first b5j spine stopped at the node without them, and a test now holds the node's declared reads to the work tests' columns.

Since your review (details in the body): the SPI prior shape came onto the branch and came off again (María's ruling of 3 November), and B4's HICBC diagnostics, PR 2b's J2 and K, and the VED inputs are on the branch. The HICBC gap is #1508. The builds measure each item on 312dc4d6b, so a control on 79a533c72 and a build of the merged head are still owed before a whole-PR comparison on the new base.

CI at 1ac980d1d: all ten checks pass.

@juaristi22
juaristi22 marked this pull request as ready for review November 4, 2026 09:51
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 Round 3 at 1ac980d1d. I read 0cddc8464, 860731004 and 1ac980d1d and your reply, and checked the JSA limits against the Jobseekers Act 1995, the JSA Regs 1996 and policyengine-uk's parameters. I ran the engine-free tests for spi_benefit_coherence, the graph and frs_relationships (94 passed). I also re-ran my two cases through _contributory_work_limits on a synthetic frame. I didn't build anything or look at the private figures.

Earlier points

  1. B10's JSA limit: fixed. The new limits match the law. Reg 56's prescribed amount is A + D − 1p, where A is the s.4(2) age-related amount and D is the reg 99(2) disregard. Reg 99(3) switches off Sch 6 paras 5 to 8 and 11 for contribution-based JSA, so the general £5 of para 12 applies. The age-related amounts are £71.70 under 25 and £90.50 from 25 in 2024-25, and £72.90 and £92.05 in 2025-26, matching gov.dwp.JSA.contrib. The 16-hour test follows reg 51(1)(a). The award is cut pound for pound above £5 (s.4(1)(b), reg 80) and converted to weekly and annual amounts at 52. My cases at this head: £150 a week gives 0, £50 a week gives £2,366 a year, £95.49 a week gives £0.52, and £50 a week under 25 gives £1,388.40.
  2. Hours in the docstring: fixed. Both awards now test the twin's hours_worked / 52 against 16, and the graph declares hours_worked and household_source_id, with a test that holds the node to them.
  3. B7 upper bound: fixed (frs_relationships.py:204-220).
  4. Gate note: fixed. It names t1426m at 2b49a00ac and gives the 0.016 margin for single people (gates.json:658).

New findings

  1. Nit: neither award is cut for the copy's pension payments, and the copies carry SPI pension income (spi_benefit_coherence.py:77; private_pension_income comes from the tape's PENSION, spi_income.py:416). The docstring now says so, but not why. JSA Regs 1996 reg 81 takes off every pound of pension payments above £50 a week. ESA Regs 2008 reg 74 takes off half of every pound above £85 (subject to reg 75, which leaves out survivors' pensions, war pensions and some health insurance). Failure case: a copy aged 58 with no earnings and a £15,600 a year private pension (£300 a week), whose twin reports £4,706 of contributory ESA (£90.50 × 52), keeps all £4,706. Under reg 74 the cut is half of (£300 − £85) = £107.50 a week, which is more than the award, so it gets nothing. On JSA the cut is £250 a week, so again nothing. Fix: cap each award at 52 × the weekly rate less the reg 81 or reg 74 deduction on private_pension_income / 52, and add that column to the node's declared reads in graph.py. Or, if you leave it out, have the receipt report the copies' weighted ESA and JSA where pension income is above £85 or £50 a week, so the reader can see how much rests on it.
  2. Nit: the weekly tests read annual SPI pay over 52 weeks, which is strict for anyone who worked part of the year (_weekly_earnings, spi_benefit_coherence.py:863-869). The body already says the tape's pay covers the tax year and holds part-year workers, for the pay floor. The same averaging bears on the work limits. Contribution-based JSA lasts at most 182 days (Jobseekers Act 1995 s.5(1)), and the usual claimant has just left a job. A copy whose tape pay is £10,000 earned in 26 weeks, followed by 26 weeks on JSA, averages £192 a week and gets nothing, though its earnings in the weeks it claimed were nil. You measured the JSA row unchanged from B-5a, so a docstring line saying the averaged test understates part-year claimants is enough.

Checked and fine

  • The parameters are read at the frame's year with pe-uk's fiscal-year values: the engine test asserts £71.70 and £90.50 at 2024 and £72.90 and £92.05 at 2025, and it passes in CI's engine-uk jobs (I couldn't install the engine here).
  • Strict < against £95.50 vs reg 56's "not more than £95.49": the two differ only between £95.49 and £95.50, where the cap already gives under 1p a week.
  • min(twin, cap) never lifts a copy above its twin's award (tested).
  • The higher £20 disregard of Sch 6 para 9 (part-time firefighters, lifeboat crew, auxiliary coastguards, reserves) can't be seen on the tape. Leaving it out is right.
  • The receipt now counts distinct source households, and it withholds a weighted figure, and the before/after change, when one or two households carry it.

CI at 1ac980d1d: all ten checks pass.

Verdict: all four round-2 items are fixed. The two new points are Nits: a docstring line or a receipt figure would cover each, and neither blocks.

🤖 Generated with Claude Code

@juaristi22
juaristi22 merged commit 2ed66b2 into main Nov 4, 2026
10 checks passed
anth-volk added a commit that referenced this pull request Nov 4, 2026
Brings #1465 (telemetry required mode) and #1426.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 4, 2026
… payments (microcosm#1426 review)

Vahid's third round on #1426 left two nits on spi_benefit_coherence, and
#1426 merged without them.

Neither contributory award is reduced for the copy's pension payments,
though the copies carry the SPI tape's (private_pension_income, from
PENSION). JSA Regs 1996 reg 81 takes every pound above GBP 50 a week off
contribution-based JSA, and ESA Regs 2008 reg 74 half of every pound above
GBP 85 a week off contributory ESA. B10 tests the copy's work alone, so the
awards stand. The receipt now gives, for each award, the support and
weighted award of the copies whose pension payments are above its
threshold, the most a reduction could take off. A figure is withheld where
one or two source households carry it or the rest of the award. JSA's
threshold is read from policyengine-uk's gov.dwp.JSA.contrib.pension_disregard
with the other JSA limits. ESA's is a constant, because policyengine-uk has
no parameter for it. The node declares private_pension_income, so the H2
fixture moves.

The docstring also says that the weekly work tests average the tape's
annual pay over 52 weeks, which understates part-year claimants, above all
of contribution-based JSA, which lasts at most 182 days (Jobseekers Act 1995
s. 5(1)).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi thanks for round 3. #1426 merged before the two nits went in, so they are in #1391, in its last commit (70bd560):

  1. Pension payments. The awards still aren't reduced, since B10 tests work alone. The receipt now shows how much rests on that. For each award, it gives the support and weighted award of the SPI copies whose private_pension_income over 52 weeks is above the award's threshold. For JSA that is reg 81's £50, read from policyengine-uk's gov.dwp.JSA.contrib.pension_disregard with the other JSA limits. For ESA it is reg 74's £85, a constant because policyengine-uk has no parameter for it. That figure is the most the reductions could take off. A figure is withheld where one or two source households carry it, or carry the rest of the award, which it would give back beside the whole. The node now declares private_pension_income, the graph test holds it to that, and the H2 fixture follows. Applying the reductions would change the awards beyond B10, so the receipt sizes that move first.
  2. Part-year workers. The module docstring now says that the 52-week average understates part-year claimants, above all of contribution-based JSA, which lasts at most 182 days (Jobseekers Act 1995 s. 5(1)). _weekly_earnings points to it.

🤖 Generated with Claude Code

MaxGhenis added a commit that referenced this pull request Nov 4, 2026
This PR's CI passed at 01:5xZ against main f8f543a. Main has since
taken six PRs (#1345, #1493, #1467, #1426, #1465, #1391), 178 files,
none of them this PR's. #1493 changes how the target-compilation
kernels hash their code, so CI reruns here on today's main before the
PR lands. The merge is clean over the single base f8f543a, and
against main it changes only this PR's own lines, file for file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Nov 4, 2026
This PR's CI passed at 01:5xZ against main f8f543a. Main has since
taken six PRs (#1345, #1493, #1467, #1426, #1465, #1391), 178 files,
none of them this PR's. #1493 changes how the target-compilation
kernels hash their code, so CI reruns here on today's main before the
PR lands. The merge is clean over the single base f8f543a, and
against main it changes only this PR's own lines, file for file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 4, 2026
Main brings #1426 (SPI age bands and #1095's B3 to B10 items, with HMRC's two
High Income Child Benefit Charge rows), #1391 (QRF gate calibration) and the
rest merged since 3 November.

Resolved by hand:
- uk_population_targets.json keeps both sides' rows: the five company car rows,
  then the two HICBC rows, in the targets and the registry scope, and both
  sides' unmapped declarations;
- uprating_holds.json keeps both sides' holds;
- the counts in test_uk_population_targets.py (467 targets, 405 in the registry
  scope, 220 unmapped declarations) and test_uk_target_references.py (1,425
  active references) add both sides' rows.

Regenerated on the merged tree, after syncing the venv to the merged lock:
- the target references and their membership, on Chronicle 08962f1. Against
  main only the five car rows are new; no existing row changes;
- the national compile-parity receipt;
- the release input coverage manifest: the source-manifest hash, and
  company_cars' two pay columns;
- the H2 UK spine fixture: company_cars' declaration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Nov 4, 2026
Seven PRs reached main after this PR's last CI run, which tested 52fc34c on main f8f543a: #1345, #1493, #1467, #1426, #1465, #1391 and #1473. None of them changes a file this PR changes, and git merge-tree writes tree ed52268 with no conflict. This merge starts CI on main as it now stands, including the new installed-emitter job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants