Repository navigation
Conversation
The cash-receipts parser targeted Table 3.8 "National insurance contributions" (total NICs, £200.08bn in 2025-26) on the ni_employee variable, which _parse_nics also targets at the Class 1 employee figure (£49.54bn). The two targets on one matrix column contradicted each other; the seeded constituency calibration on main b45c373 ended with obr/ni at 0.242 of target. Drop the row rather than retarget it: the Table 3.4 class targets (employee, employer, self-employed) already cover every NIC class the data populates, and a total target on total_national_insurance would conflict with them through the statutory payment recoveries and "Other NIC" lines PE-UK does not model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- test_obr_target_mapping: pin the variable of every target parsed from the committed receipts workbook, the Table 3.4 rows behind the NIC class targets, and the source identity that rules out a total-NICs target. - test_obr_target_conflicts: no two calibration targets whose matrix column is the same plain variable sum or count may disagree on a year's value, checked over the OBR targets offline and over every target the national matrix uses; it flags the removed obr/ni mapping, and a seeded random check holds the grouped check to its pairwise definition. - build_loss_matrix.calibration_targets: the target list create_target_matrix already built inline, extracted so the test checks the same set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PolicyEngine UK's year N is tax year N to N+1: convert_to_fiscal_year_parameters gives every parameter, the uprating indices included, its value at 30 April N. hmrc_spi mapped the 2023-24 SPI to 2024 and hmrc_cgt followed, so both treated 2023-24 outturns as 2024-25 values and every projected year carried one year too little uprating. The FRS release (2024-25 -> 2024), OBR targets, the SPI microdata (2022-23 -> 2022) and #533's salary sacrifice tables already use the start year; the end-year mapping arrived without stated reasoning in the February 2026 registry refactor (de061fc). - _SPI_YEAR and _CGT_BASE_YEAR move to 2023; incomes_projection.csv is regenerated from the same ODS (its 2023 rows equal main's 2024 rows). - The CGT base year is now gated at the £6,000 AEA the outturn was realised under. - Tests tie _SPI_YEAR to the tax year in the configured ODS name, check the CGT base-year AEA against policyengine-uk, and (as in #541) check that the committed projection is the SPI table uprated with the committed factors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_loss_matrix fetches targets without a year filter, and the registry keeps only the first target of each name. hmrc_spi emitted each band twice: the ODS outturn (base year only), then the projection (later years). The projected target was dropped, so _resolve_value carried the base-year outturn forward unuprated: the 2025 calibration has targeted raw 2023-24 SPI values since the February 2026 registry refactor. get_targets now merges the two into one target per band, and a test checks every projected band, variable and year through the calibration's own resolution path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derive_is_in_approved_training_from_frs_person read a `train` column. No FRS release has one (2020-21 to 2024-25 carry TRAIN2 and TRAINEE), and the function returned False when the column was missing, so every record was coded as not in approved training. DWP's FRS counts a 16- to 19-year-old as a dependent child only when in full-time non-advanced education or unwaged government training. In FRS 2024-25 all 20 child-table records aged 16 to 19 outside full-time education have TRAINEE = 1 (33.0k grossed), and uk-data coded every one as failing the qualifying-young-person education-or-training condition. Read TRAINEE = 1 (yes; the same yes/no format as EDUCFT), and fail the build when the column is missing instead of defaulting to False. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #561 (Subfleet 20261010-193520-ud561-review-r1) found that an adult-table trainee aged 16 to 19 would become a qualifying young person of their own benefit unit. FRS 2024-25 has none, but 2020-21 has one and 2023-24 three. The FRS has already decided those records are not dependants, and the adult TRAINEE question does not say the training is unwaged, so it may be training under a contract of employment (UC reg. 5(3)). - Flag TRAINEE = 1 on child-table records only. On FRS 2020-21, 2022-23, 2023-24 and 2024-25 the result equals the child-table records aged 16 to 19 outside full-time education exactly (8, 26, 16 and 20 records). - Say why TRAINEE and not TRAIN2: `train` was TRAIN2's question name, only the adult table has TRAIN2, and its value labels are not in the repo. - Property test over numeric, float and string codes and adult/child status; a create_frs test with an adult-table trainee in the same household. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delta review r2 of #561 (Subfleet 20261010-201405-ud561-review-r2, APPROVE WITH NITS at c952694): - The docstring said a dependant outside full-time education "carries TRAINEE = 1" as if it were guaranteed. It is what the four FRS releases checked show, so say that, and call the adult exclusion a proxy. - The adult-trainee fixture set age80 = 18 but kept age = 30, and create_frs adds the two, so it built a 48-year-old. Set age = 0 and assert age 18. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r2 (APPROVE_WITH_NITS): with the live ODS download failing, the merge test passed even without the merge, because only the projection targets were emitted. The test now serves get_income_band_table from incomes.csv, checks the outturn year as well as the projected years through the calibration resolver, and requires every SPI target to carry all of them. The comment now names 30 April N, the date convert_to_fiscal_year_parameters samples. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 1.58.0 enhanced FRS carries £53bn of gains above the annual exempt amount, and -£15bn of losses on 1.4m people, against the £65.9bn the build targeted and the £119.3bn HMRC now reports for individuals in 2024-25 (uk-data#570). The calibration started 17 times over on CGT taxpayers and 5 times over on gains: the has-gains copy of each zero-weight SPI household kept a zero weight, so it received a full zero-weight prior, the same as its no-gains original. After 512 epochs the taxpayer count was still 56% over while the £12,300 to £250,000 bands had overshot to 23-64% under. - Split every household's mass between itself and a has-gains twin by the Advani-Summers share for its income band, in closed form, and split a zero-weight household the same way. - Give no twin to draws at or under the lowest HMRC band limit, losses included: no target describes them. - Stack donors only for the bands from £500,000 up. - Scale the calibration's starting weights so each HMRC size band's taxpayer count begins on target (rake_priors_to_targets). - Target Table 2.1a for 2024-25 (2026 release), individuals only, from the £3,000 exempt amount up, with the aggregates as the band sums. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The has-gains copies of zero-weight SPI households now carry a weight, so that the calibration starts from the right share of each. Salary sacrifice, student loan plans and Pension Credit take-up count people on survey weights and draw per record. Run before capital gains, they see neither those weights nor the copies, and a copy matches its original in everything but the gain. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- test_obr_efo_fallback: the receipts and NICs set from #536 listed obr/ni, which this branch no longer emits. - test_obr_target_conflicts: #490/#530 added Target.countries, and create_target_matrix zeroes a column outside them, so the column key now includes countries. The random differential check uses Hypothesis, now a dev dependency, and draws country restrictions too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- obr.py: name what PE-UK lacks (statutory payment recoveries, Class 1A, 1B, settlements, unallocated) and say ni_class_3 exists but is unfilled; the _parse_nics docstring no longer claims it covers Class 3. - test_obr_target_conflicts: compare every pair in a shared column (isclose is not transitive), key countries as a set (restrict_to_countries uses np.isin), guard the registry-wide test against a silently empty registry, draw carry_forward and both orders of a two-country restriction, and pin the tolerance edge with an example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sol review r2: the pairwise reference checked a pair only in its own listed years, so two carried-forward targets that disagree in a year only a third target lists were reported by the grouped check alone. The reference now uses the column's full year set, and a regression pins the reviewer's three-target case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OBR targets came from whatever obr.uk served, falling back to committed workbooks matched by URL substring, with the vintage label taken from sources.yaml rather than the bytes, year columns fixed by letter, and missing rows logged and dropped. A manifest per EFO vintage (targets/source_manifests/) now names the two workbooks with their SHA-256, size and own title, the unit and financial years every parsed table must head, and the targets they must yield. Builds read only the verified committed bytes; each table's years come from its header row; a missing target or year, or an undeclared one, fails the build. `python -m policyengine_uk_data.targets.obr_sources ingest` adopts a new vintage from browser-downloaded workbooks and prints each target's change. On the March 2026 bytes all 29 OBR targets are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
calibrate_local_areas saved and returned the weights only when epoch % 10 == 0, so a 512-epoch build shipped the state after step 511 and a 32-epoch build the state after step 31, dropping the last update (#456). Save at the last epoch too, in both the verbose and the non-verbose loop, so the saved file, the returned dataset and the logged fit are the final optimiser state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
create_target_matrix skipped a target whose year did not resolve with no log line (#456), skipped one whose column raised with a warning, and accepted columns no household contributes to (#593). It now records each offered target's status and fails unless every target is in the matrix with a finite value and signal, or is listed in targets/known_gaps.yaml under that status with a reason and an issue. Errors and non-finite values can never be listed; a listed gap that no longer applies is logged as stale. known_gaps.yaml lists the 34 gaps measured on uk-data 1.58.1 at the calibration year 2025: 15 two-child-limit targets dated 2026 only, and 19 targets with no signal. The matrix built is unchanged. The diagnostic get_loss_results and the CPS transfer reweighting opt out. `python -m policyengine_uk_data.targets.coverage` reports every target's status on a dataset (names, statuses and counts only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pull request CI now also builds the PR's base, in the same way as the head (TESTING, one clone) and with the head's lockfile, so both run one model and environment. A new job compares the two final-epoch fits from their calibration logs. It fails when the share of national or local targets within 10% falls by more than 2pp, flags national targets whose error rises by more than 0.2, and posts the comparison on the PR. The thresholds (targets/fit_policy.yaml, adopted under d1344) come from five builds of the same data code: their shares moved by at most 0.93pp (national) and 0.69pp (local), and they flagged 0 to 6 targets per pair. Per-target moves only flag while #598 (two weight sets from one code) is open. Only calibration logs are uploaded, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n unit Sol review of #592: - The manifest's vintage, its label and both workbook titles must agree (march_2026, "March 2026", "March 2026 Economic and fiscal outlook ... receipts"), at load and at ingest, so verified March bytes cannot be labelled October. - A row whose label states its own unit ("Standard colour licence fee (£)") is refused under a "£ billion unless otherwise stated" table, and the TV licence target reads the "BBC licence fee receipts" row only. - Ingest checks both workbooks and parses them in a staging folder before copying anything, and never replaces an ingested vintage. - Coverage checks every workbook target, including one without the vintage. - The NICs docstring no longer claims Class 3 or that partial results pass. All 29 OBR targets are still identical to main's on the March bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration fits grouped targets; these checks measure what the groups hide, on the built dataset. Each PR build and its base build report person-weighted equivalised income medians, means and Gini, and relative low income for everyone, children, working-age adults and pensioners, against DWP's HBAI FYE 2025 estimates and 95% intervals (tables 8a and 8b, read from the published ODS files), at period 2024. They also report people by sex and single year of age against the ONS 2024-based projection at the calibration year. The comparison job adds them to the PR comment and flags a measure that leaves its interval; it fails only on a broken or mismatched report until the same-code spread is measured (#598). On uk-data 1.58.1, 5 of 14 HBAI measures are outside their intervals (median AHC, both Gini coefficients, child low income BHC and AHC), and single ages 0 to 79 are off by 19.5% on average. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first CI build of this branch failed the gate on three single-claimant UC payment bands (24,000 to 28,800 a year). Pull request (TESTING) builds hold fewer records than the release: across the final calibration logs of 12 recent ones, these three have no signal in 11 or all 12, while the released 1.58.1 build has signal in all three. A listed gap that applies to one kind of build only warns as stale in the other. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…book The Scottish band targets came from a live gov.scot download. When gov.scot's CloudFront challenge blocked it, as in two of five recent pull request builds of the same code, the loader substituted a fixed 2025 total spread by fixed shares, so identical code calibrated to different targets (#596). Commit the September 2025 chargeable-dwellings workbook, verify its SHA-256, and fail if it is missing or changed. Its Scotland row equals the targets of the builds whose download succeeded. Fixes #596. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both builds are measured with policyengine-uk's aggregate-impact tool at a pinned commit (policyengine-uk#2425, stacked on #2335), under the head's locked model: receipts by tax head, spending and caseloads, poverty, inequality, deciles and the ten standard reform costings, 2025-26 to 2031-32. The comparison job runs `compare --axis data`, which fails unless both sides ran one model, runtime and set of definitions, and adds the result to the PR comment. It fails on a check the head breaks that the base passes. Only each side's result.json (aggregates) is uploaded; the tool's private household detail stays on the runner. Under the locked 2.122.2 the tool falls back to policyengine-uk main's partition, whose lists differ from that model's, so closure and two identities fail on both sides (pre-existing) until uk-data locks a model that ships its own partition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
obr.uk refuses scripted downloads, so on Budget day a person downloads the two EFO workbooks in a browser. The manifest now records each workbook's download time with its SHA-256 and size: the file's modification time, which a browser sets on saving, unless --downloaded-at is given. `ingest --dry-run` checks everything and prints titles, hashes, sizes, download times and every target's change, writing nothing; on the active vintage it says whether the bytes match the committed workbooks, so a March 2026 download rehearses the Budget-day path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sol review of #600: - Negative incomes before housing costs are reset to zero, and income after housing costs is derived from the adjusted figure (DWP HBAI methodology FYE 2025, "Negative incomes"). Weekly amounts use 365.25 / 7, as the FRS loader annualises them. - A report missing any reference measure, or with a missing or non-finite single-age summary, fails the comparison. - A missing distribution report fails the job. On uk-data 1.58.1, four of 14 measures are now outside their intervals (both Gini coefficients, child low income BHC and AHC); median AHC moves inside. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rint commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Raw calibration logs stay on the runner. Each build uploads a summary: whether every national and local target is within 10%, and the error of national targets supported by at least 10 households only (the constituency log gains a records column). An estimate from a few records, divided by a published count, could give one household's value back. The test job's raw-log uploads are replaced by the summary too. - The gate compares shares over the targets both builds have; added and removed targets are notes. - An empty national or local group, or a last epoch with fewer rows than the one before, fails. - The comment step writes a failure comment when the comparison never ran. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With #557 each SPI income band is emitted once, so its 156 duplicate entries go; with #604 the ages are 80 and over, so the two ons/*_85_90 zero-signal entries go. The shipped-file counts move from 19 to 17 without signal and from 159 to 3 duplicates (the ONS three, #605). The ten SPI bands of £1m and over keep their zero-signal entries: #557 gives them no signal. On uk-data 1.58.1 at 2025 the coverage CLI reports 626 in the matrix, 3 duplicates, 15 with no 2025 value and 17 without signal, with no problems and no stale gaps, as a release build and as a testing build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(review nit) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… map the 2023-24 outturn to 2023) at 473adaf
# Conflicts: # policyengine_uk_data/targets/build_loss_matrix.py
133632a # Conflicts: # policyengine_uk_data/targets/sources/obr.py # policyengine_uk_data/tests/test_obr_efo_fallback.py # policyengine_uk_data/tests/test_obr_receipts_sheet.py
…ainst its base) at 45d4742
…mmitted workbook) at 0cf62aa
… model fixed) at 0822699
…ifacts) at 5881ce3 # Conflicts: # .github/workflows/pull_request.yaml
…GT prior raking (full PR workflow, unseeded, as #608's gate runs)
Calibration fit against the base build: fail
A drop of more than 2.0% fails. Failures
Notes
National targets in a worse error band on the head (33)
3 more in the job log. Flags are for review: builds of the same code move 2 to 12 national targets to a worse band (#598). Bands are of |estimate / target - 1|; no target's estimate or error is published. Income distribution and single ages against the base build: passHBAI FYE 2025 (DWP), period 2024: equivalised income in £ a week, rates and Gini in per cent. A value outside HBAI's 95% interval is marked (outside).
FlagsNone. These measures report and flag but do not fail yet: their same-code spread is unmeasured (#598). Aggregate impact of the data change: passBase dataset Checks: 0 broken by this change, 4 failing on both sides, 0 fixed. Moves flagged for review: 422 (thresholds in Notes
Moves flagged for review
382 more flags are in the report artifact. Receipts by tax head
Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Components the model counts twice
Household income totals
Population
Poverty rates
People in poverty
Inequality
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileThe datasets hold different households, so households are not matched; compare the mean income in each side's own deciles. Checks failing on both base and head (4)
Counted-once checkCounted-once check not run: not requested. Provenance
Base dataset: Head dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Components the model counts twice: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
Contributor
Author
|
Closing: this was a diagnostic build for the batch-3 fit attribution, never meant to merge. Max ruled d1678 = A, and the release, #608, merged at 🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DO NOT MERGE. This is a diagnostic for the batch-3 release #608 (option C). The UK hub closes it, and deletes its branch, once read.
The question: with #571's CGT prior raking removed from both local calibrations, but its targets and twinning kept, does #608's real CI fit gate pass?
cb92e401.Context, from production-setting (512-epoch) diagnostics:
C keeps #571's CGT level: £25.9bn in 2025-26, against £25.7bn with the raking.
🤖 Generated with Claude Code