Repository navigation
Conversation
On the 1.58.0 enhanced FRS, 76 households have negative VAT (uk-data#565): 44 because an imputed housing, water and electricity amount is negative and outweighs their other consumption, and 32 because their imputed full-rate VAT expenditure rate is negative. Two more households have a rate above 1, one of 55. Both imputations are QRFs, which predict values they were trained on. Nine LCFS households record a negative P604. Four ETB households report expenditure at or below their VAT, and thirteen a ratio above 1. - Clamp the consumption training table's amounts at zero. - Keep the VAT training rate a share of expenditure: drop rows with no expenditure left after VAT, and cap the rest at 1. - Fail the build if an imputed amount is negative or a rate leaves [0, 1]. - Mark both models' metadata so cached models from the old tables are retrained. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 1 review nits (Sol, APPROVE WITH NITS at d14e0c7): impute_consumption now has a test that fails if its non-negativity guard is disconnected, and the comments say the QRF samples conditional quantiles within the training range rather than reproducing training values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration fit against the base build: pass
A drop of more than 2.0% fails. Notes
National targets in a worse error band on the head (10)
Flags are for review: builds of the same code move 2 to 12 national targets to a worse band (#598). Bands are of |estimate / target - 1|; no target's estimate or error is published. Income distribution and single ages against the base build: passHBAI FYE 2025 (DWP), period 2024: equivalised income in £ a week, rates and Gini in per cent. A value outside HBAI's 95% interval is marked (outside).
FlagsNone. These measures report and flag but do not fail yet: their same-code spread is unmeasured (#598). Aggregate impact of the data change: passBase dataset Checks: 0 broken by this change, 3 failing on both sides, 1 fixed. Moves flagged for review: 188 (thresholds in Notes
Fixed by this change
Moves flagged for review
148 more flags are in the report artifact. Receipts by tax head
Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Components the model counts twice
Household income totals
Population
Poverty rates
People in poverty
Inequality
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileThe datasets hold different households, so households are not matched; compare the mean income in each side's own deciles. Checks failing on both base and head (3)
Counted-once checkCounted-once check not run: not requested. Provenance
Base dataset: Head dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Components the model counts twice: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
|
Hand-off to the UK hub (session
|
Fixes #565.
Status: ready for review. Approved by Sol at this head (below), and CI passes. uk-data merges only in a batched release (d833), so this PR is not to be merged on its own.
What is wrong on 1.58.0
Measured on the released file (
enhanced_frs_2024_25.h5, SHA-2565ba399ea5ab9a6c7d4e9aaf6c623080beb34d9ad5f330f6244dbad72b8d960b6) with policyengine-uk 2.125.1 and policyengine-core 3.33.0:vat = (consumption × full_rate_vat_expenditure_rate × standard rate + consumption × reduced-rate share × reduced rate) / microdata_vat_coverage(vat.py), whereconsumptionis the sum of twelve COICOP inputs. The formula needs no floor if its inputs are non-negative.housing_water_and_electricity_consumptionthat outweighs its other eleven categories. In all, 94 households store a negative value there; the lowest is −£5,498.full_rate_vat_expenditure_rate(the lowest is −1.02).Why
Both imputations are quantile regression forests. They sample conditional quantiles of their training values, so their predictions stay within the range of those values. A negative training value can therefore reach the dataset.
consumption.py,generate_lcfs_table):P604) in 9 of 4,202 households. That is 0.2% by weight, and the lowest is −£265 a week.vat.py,generate_etb_table):(totvat × (1 − reduced-rate share) / standard rate) / (expdis − totvat).expdis − totvat ≤ 0, which gives a negative rate (down to −3.4).What this PR changes
impute_consumptionandimpute_vatfail the build if an imputed amount is negative or a rate falls outside [0, 1].Impact
Source: this PR's CI (run 40190348100, head
9fd9164e). It builds main and this branch the same way: TESTING settings (32 epochs), one OA clone, and the head's lockfile. It then measures both builds with policyengine-uk 2.122.2, with a full microsimulation for each year on each side. All figures are £bn.test_built_vat_is_non_negative_for_every_householdandtest_built_vat_inputs_are_non_negativepass, out of 2,674 tests passed.sign:vatcheck fails on the base build and passes on the head. That is the comparison's one "fixed" check; none is broken.dropout_weightsdrawstorch.rand_like), so the two builds also differ by calibration noise.household_taxaddsvat_change, notvat, so baseline VAT does not enter household net income.obr/vatis in the same error band on both builds.Invariants
impute_consumptionraises if its output has a negative amount. A test runs the real function with a mocked model, and it fails when the guard call is removed.policyengine_uk.Microsimulation. The test has no reduced-build slack, because the property belongs to the inputs, not to the calibration.Reviews
d14e0c7b: APPROVE WITH NITS. It found two P3 nits, both fixed in6be52fe8:9fd9164e: APPROVE, with no findings.Related
num_vehicleson 1.58.0 (the lowest is −8). That comes from a different imputation, the WAS wealth imputation, and does not reach VAT: 16 households have a negative vehicle count on 1.58.0 (WAS missing-value code in the wealth imputation) #574.microdata_vat_coveragethat this branch's locked policyengine-uk 2.122.2 uses. policyengine-uk#2187 (0.469, first released in 2.135.7) addresses that, not this PR.axiom: n/a: data
🤖 Generated with Claude Code