Skip to content

Pair Autumn Budget 2026 claims with the dashboard's Budget-day runs (tranche 3 of #230) - #249

Merged
MaxGhenis merged 21 commits into
mainfrom
uk/ab2026-tranche3
Nov 5, 2026
Merged

MaxGhenis merged 21 commits into
mainfrom
uk/ab2026-tranche3

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

In tranche 3 of #230, we pair the Autumn Budget 2026 claims on the registry's eight pe_modifier measures with PolicyEngine's Budget-day estimates. We read those estimates from the UK Autumn Budget dashboard's published artifacts at 1eca3182, the commit we pin in the registry, and run no microsimulation. We branched from #247's branch, which merged as 1615e267, and retargeted this PR to main.

Changes

The 6.11.1 pair (8d2d9f5)

We add pair-pe6.11.1_uk2.151.0_microcosm-uk-2024-25-national-20261025T091414Z-a6299504 to data/uk/certified_pairs.json, the pair we name in the registry's engine_pin. The dashboard's generator writes the wrapper, model, core and data versions, the dataset's digest and URI, the certificate and the data years into every receipt. It writes no package RECORD or wheel digest, bundle manifest digest, dataset size or requirements freeze. We take those from the replay variant the receiver qualified for the same pair on main (#242) and copy its freeze byte for byte, so the pair's identity digest equals that variant's. The dashboard ran these jobs on Modal under Python 3.13.3 and its own lock, which we don't bind in the record, so we list the local executor only.

Vendored artifacts (b11011b)

pipeline/stage_uk_ab2026.py vendor copies the eight receipts and the six CSVs we pair from into results/uk/ab2026/. It checks each receipt against the registry's receipt_file_sha256 and each CSV against its receipt's files digest, and records each file's path, sha256, source path and source commit in MANIFEST.json.

Pairing (b11011b, eb7c13b, and the commits after review)

pipeline/stage_uk_ab2026.py stage reads the 368 claims tranche 2's ingest writes on these measures and pairs each with one cell, or tallies it under one of 16 written reasons. We read src/uk_budget_data/calculators.py at the pin for what each column measures.

  • We pair a claim on a whole measure's UK effect in one fiscal year with the change in gov_balance from budgetary_impact.csv, and orient that change to the sign convention signs.py reads from the claim. We pair 34 claims this way: 17 HM Treasury Table 6.1 lines, and 17 OBR measure totals from the "Scorecard measures" section of its breakdown, which we key to Table 6.1 lines.
  • We pair HM Treasury's 500,000 Pension Tax Allowance gainers in 2027-28 with people_directly_affected from age_impact.csv. We count people whose own income tax, National Insurance, student loan repayment or employment income changes by at least £1, excluding households without an income decile. Of these four, the government changes only income tax through this allowance.

We name each row's file, column and row, and we annotate it to say that we estimate with no behavioural response, measure from current law in policyengine-uk 2.151.0, and count liabilities where producers count receipts. On the OBR's rows, we add to the baseline annotation that the OBR measures from its forecast before the Budget's measures. HM Treasury names no baseline in its table, and we say so on its rows.

outcome claims
counterpart 35
a part of the exchequer effect (head, line or block grant adjustment) 141
equivalised vigintiles after housing costs 95
UK cost with Barnett shares the run doesn't model 24
zero on both sides 22
share of households 14
typical energy bill 8
fuel duty on every litre of taxable fuel 6
worked example 6
cumulative real-terms sum 4
Pension Tax Allowance after it ends 4
household count 3
motorists 2
carers gaining a benefit 1
pensioners taken out of tax 1
no fiscal year 1
an average across households in England 1
total 368

We pin these counts per measure and each paired claim's cell in pipeline/stage_uk_ab2026_expected.json. The stager raises on drift, on a claim shape it has no rule for, and on a nonzero claim against a zero from our run outside the Pension Tax Allowance's ended years.

Results and lanes (b11011b)

The stager writes results/uk/staged_ab2026/ab2026_counterparts.jsonl in AB2025's campaign form, which ingest_campaign attaches by claim id. We add STAGED_COUNTERPARTS and advance_lanes to ingest_uk_ab2026, and advance_lanes moves a lane to computed once its claims have counterparts.

Invariants (tested)

  1. Every vendored file hashes to the manifest, each CSV to its receipt, and each receipt to the registry at the pinned commit, and the pair record agrees with every receipt. vendor writes the committed manifest and files from those bytes, and refuses a receipt or CSV whose bytes changed.
  2. We rederive every counterpart from its CSV cell and the producer's sign convention, byte for byte. We use every vendored CSV in a counterpart, and the year we derive from the claim matches the year we derive from the cell.
  3. The accounting closes per measure and in total, each claim lands in one bucket, and Hypothesis samples the claims and requires the identity for every sample.
  4. Hypothesis flips a claim's sign convention and value, and the sign relation flips while the counterpart changes sign and keeps its magnitude.
  5. Two runs give the same output, and Hypothesis shuffles the claims without changing a byte.
  6. No claim gets two computed answers. When we build a temporary database with the AB2026 ingest and these results, it shows each comparison once, and advance_lanes moves only the official lane.
  7. We give each row the annotations its producer, measure and year call for and no others, and the tests compare the whole list.
  8. We give each exchequer row one sentence naming every nonzero line of the OBR's breakdown for its measure and year, as a cost or a gain with the amount the OBR prints. We leave out only the receipts heads, the welfare line and the behavioural line, and we state the behavioural line on the property rows from 2028-29. Any other line we can't name, two claims on one named line, or a named line under another sign convention or none stops the stager. We tally or annotate each measure for which we name a nation, Great Britain or a Scottish or Welsh group in the dashboard's receipt.
  9. Tranche 2's differential test compares stage_uk_ab2025's output, the attached counterparts and the advanced lanes with origin/main. CI fetches origin/main for it since Ingest the Autumn Budget 2026 harvest (tranche 2 of #230) #247, and the test fails rather than skips in CI if the ref is missing.
  10. We reject claims with a different geography or unit, exchequer claims without a readable sign, and claims on the OBR costings lane.
  11. We name in each reason the producers whose claims it covers, or write "the producer" when it covers several, and we state the dashboard's £1 minimum and income decile filter in the two count reasons.

What wiring it in would add

We built a temporary database at a58ddc3 with the AB2026 ingest, its verdicts, this staging and advance_lanes, on a copied lanes feed, then ran export_populations into a scratch path. We find 4,687 AB2026 claims in the database, 3,207 of them with a result: 35 constructed and 3,172 pe_gap. The export writes 4,661 rows, 3,172 pe_gap, 1,454 not computed and 35 constructed, and we count 35 answered of 3,828 on the official lane, 0 of 586 on the microsimulation lane and 0 of 273 on the admin-other lane.

The public site

scorecard_db/build_db.py calls neither ingest_uk_ab2026 nor this stager, so when we merge, we change nothing on the site and leave data/lanes.json, data/populations.json and data/comparison.json as they are. Max decides when we publish these comparisons on the site.

Review fixes

After the review of 2cfdf22, we merged tranche 2 at 589f822 without moving any claim, value or cell. We also named HM Treasury as the actor in the baseline annotation on its 18 rows ("HM Treasury names no baseline in its table"), and named the actors in docstrings and comments (a21cabd).

The review of 75c4a1f

The same cloud session approved 75c4a1f with one low finding. No test pins claim f59d84db9343c682f25b, so in the L4 bullet below we now plan to regenerate STAGING_UNANSWERED.jsonl, where the stager lists the claim by id, in place of updating pins.

The review of a58ddc3

The cloud session that reviewed a21cabd requested changes at a58ddc3, with one medium finding and four low ones, and we answered them in 75c4a1f. We changed only the counterparts test, so no claim, value, cell or staged row moved.

  • The annotations bullet (cloud M1). When we took out the average in a58ddc3, we also rewrote the record of 53cdc41 in "The reviews of a21cabd". We restored that record and added the later change.
  • The sign guard (cloud L1). The test fed the stager only conventions it reads as unknown, so a stager that rejected only unknown signs passed it (mutant Y1). We added the convention that leaves the direction to a label, on which the stager raises "sign magnitude".
  • The reasons (cloud L2). We now write the 16 reasons out in the test and require the stager's to match, so a test fails if we change a word in any reason (mutants Y2 and Y4).
  • Actors (cloud L3). We gave "we" or tranche 2's ingest the verbs in the six sentences the reviewer named, where a merge or commit had them. In a full pass we also gave "we" the verbs where the dashboard, a CSV, the site or the staged rows had them.
  • The behavioural claim (cloud L4). We still give claim f59d84db9343c682f25b the aggregation "annual average from 2028-29" on main. We'll set it to "annual" in the follow-up PR where we stage the OBR's paragraph 4.30 and Table 6.3 rows, and regenerate STAGING_UNANSWERED.jsonl, where the stager lists the claim by id.

At 75c4a1f, the same nine test files gave 2,470 passed and 1 skipped with four workers and the same two deselections. A test killed each of the 36 earlier mutants and the reviewer's three survivors.

The review of 44d226d

GPT-6.1 Sol approved 44d226d with three low findings. We merged tranche 2 at a28223f in 570ed6c without moving any claim, and answered the findings in a58ddc3. No paired claim, value or cell moved, and we changed the staged rows only in the behavioural sentence on eight property rows.

  • The behavioural sentence (Sol N1). In paragraph 3.17 the OBR estimates a yield of £0.4 billion a year on average from 2028-29 from the further rise in property income tax, and says behavioural responses cut it by £0.1 billion a year. We had put the average on the £0.1 billion in 53cdc41, so we dropped it from the sentence and the stager's comment.
  • Tests (Sol N2). Ten of the mutants Sol ran survived the tests, and the committed stager handles each of their inputs correctly. We now test the gainer and exchequer guards on each nation, Great Britain and no geography, and on more units. We test the OBR costings lane on the keys in its registry and on any other text, and we test exchequer claims without a readable sign. We also test named and behavioural OBR lines under a reversed convention or none, and the opposite-sign annotation on a flipped claim. A new test compares each reason with the producers whose claims it covers, and another requires the £1 minimum and the income decile filter in the two count reasons.
  • Text (Sol N3). We named the reviewer as the actor in the three sentences below that opened "The review of".

At a58ddc3, the same nine test files gave 2,468 passed and 1 skipped with four workers and the same two deselections, and a test killed each of the 36 mutants we ran, including Sol's ten.

The merge of 8a3e498

We merged tranche 2 at 8a3e498 in 44d226d. #247's ingest now keeps OBR row 2740 on decisions_effect_on_borrowing under the macro verdict and gives it one pe_gap verdict, so we store one more result and export one fewer row as not computed. No counterpart, tally or annotation moved, and stage reproduces the committed files.

We ran the same nine test files with four workers at 44d226d. They gave 2,421 passed and 1 skipped, with the same two deselections, and a test killed each of the 22 mutants again.

The reviews of a21cabd

GPT-6.1 Sol and a cloud session each approved a21cabd with low findings only. We merged tranche 2 at 0193b91 in b3d917b and answered both reviews in 53cdc41. No paired claim, value or cell moved, and we changed the staged rows only in their annotations.

  • IFS row 46. Since 0193b91, tranche 2's ingest scores this claim on average_household_income_change, so the stager reads it. IFS averages January 2027's Household Energy Payment at £80 across households in England, and we publish £67.44 across all UK households for 2026-27, with no figure for England alone. We tally the claim under a 16th reason and name the measure and England in the rule, so the stager still raises on a spending-decile claim on another measure or area.
  • Carer's Allowance (Sol F1). We now also say on the four rows that we count the extra allowance in full against Universal Credit but omit the PAYE deduction from earnings that DWP would apply, so we can show some carers losing overall. We state this limit in the dashboard's receipt and put no amount on it.
  • Reasons (Sol F2, F3, F6; cloud L8). We named the producer as the actor in each reason. We now cover properties and households missing the payment in the share reason, and state the £1 minimum and the income decile filter in the two count reasons.
  • Annotations (cloud L3, L4). In 53cdc41 we wrote that the OBR expects more landlords to incorporate to save tax and more to underpay, cutting the yield by £0.1 billion a year on average from 2028-29. We took the average out in a58ddc3 (Sol N1). We say in the trading allowance sentence that we may overstate the Exchequer's cost for some traders and understate it for others.
  • Guards (cloud L2). The stager now raises when it reads another sign convention on a named line of the OBR's breakdown or on its behavioural line, because we read cost or gain from the OBR's sign.
  • Tests (Sol F4, F5; cloud L1). The tests now compare each row's whole annotation list, reject claims of another geography or unit or on the OBR costings lane, and execute vendor against the committed bytes. We now rederive each counterpart without the stager's sign helper. A test killed each of the 22 mutants we ran on these guards and sentences, including the trading sentence on every row, which passed before.
  • Text (Sol F6, F7; cloud L5, L7). We rewrote the module docstring's scope bullets and fiscal-year passage, and the sentences below that the reviewers named.

We ran nine test files with four workers at 53cdc41: test_ab2026_harvest_contract, test_ab2026_key_decisions, test_ab2026_registry, test_uk_ab2026_harvest, test_uk_ab2026_ingest, test_uk_ab2026_counterparts, test_signs, test_harvest_ingest and test_uk_certified_pairs. They gave 2,397 passed and 1 skipped, the registry's regeneration, which needs the 2.151.0 environment. We deselected test_every_exported_row_carries_a_declared_sign and test_app_data_copies_match_committed_data, because our sparse tree lacks their inputs, and CI runs both.

The review of 2cfdf22

The reviewer of 2cfdf22 found that the stager skipped the OBR's behavioural line on property income without saying so, and that the tests didn't tie each amount to its line.

  • The stager now skips only the receipts heads ("Receipts:" and "Tax head (receipts)"), the welfare line and the behavioural line, and raises on any other nonzero line it can't name. It also raises when two OBR claims share a named line for one measure and year. Claims may share a receipts head, since we read no receipts head into an annotation and electricity's claims already share "Receipts: VAT".
  • The OBR prints one behavioural line on property income, about £0.1 billion a year, which it puts down to more tax-motivated incorporation and higher non-compliance (paragraph 3.17). We add one sentence to the property rows for 2028-29 to 2031-32 from both producers, reading the amount from that claim, and none to 2027-28. We end the sentence with "and we hold incomes fixed", because we say in the dashboard's receipt that we hold incomes fixed in the run, with no behavioural response.
  • We now write every line of a breakdown sentence as a cost or a gain, which we read from the OBR's sign, with the OBR and "we" as actors. For example, on the OBR's 2027-28 trading allowance row we write "The OBR also counts an £8m cost from the Scottish block grant adjustment and a £7m gain from Scottish self-financed expenditure in this year, and we model neither."
  • We now say on the trading allowance rows that we have no gross trading receipts in our data, so we relieve profits up to £3,000 in full and none above it, as we state in the dashboard's receipt.
  • We took the review's wording for the property, electricity and Pension Tax Allowance sentences and the first gainer sentence. We name the government as the actor in the second gainer sentence and the property sentence, and end the property sentence "as the government does from 6 April 2027", the date in the registry's title.
  • We now say in the reason for the Pension Tax Allowance's ended years that HM Treasury and the OBR "count a change in receipts" in them. HM Treasury prints a £50m gain in 2030-31 and a £15m cost in 2031-32.
  • The annotation test builds each breakdown sentence from the OBR's claims with its own names, articles and signs, and asserts every scope sentence in full.
  • We merged tranche 2 at 5ff3c4b, and its ingest drops three IFS claims that split households by spending (1b2e6f7f, 35db1466, 61178ad2). The stager then reads 367 claims, no longer needs the spending-decile reason, and raises on a claim on that axis. We also merged b227956 and 002d90c, in which we added only tests to tranche 2, and no paired claim, value or cell moved in any of the three merges.

The review of 229bbc1

The reviewer of 229bbc1 found that we left Scottish lines the OBR prints off the trading allowance's and the Pension Tax Allowance's rows, and that the scope test copied the stager's choices.

  • We annotate every kept exchequer row with each nonzero line of the OBR's breakdown for that measure and year that our run doesn't model, on both HM Treasury's and the OBR's rows. We name the Scottish block grant adjustment, Scottish self-financed expenditure, departmental resource spending and other annually managed expenditure, and the stager reads each amount from the OBR's claim. It leaves out the receipts heads and the welfare line, which gov_balance itself counts, and raises on any other line it can't name.
  • We keep the four electricity VAT rows with annotations naming household, charity, residential care and business consumption in HM Treasury's base and household electricity in ours. HM Treasury estimates that base from ONS consumer expenditure data (costing note, page 12), calls the business share small, and gives no separate amount for the charity or care shares.
  • We now say in the Carer's Allowance annotation that we leave Northern Ireland's Carer's Allowance at current law, and in the property income annotation that we raise finance cost relief for landlords across the UK.
  • We now say in the netting annotation that we net off any change in means-tested benefits, and in the gainer annotation that we exclude households without an income decile and that HM Treasury states no minimum.
  • The scope test builds its expected lines from the OBR's claims, and reads Great Britain, GB, Scottish and Welsh in each receipt as well as the four nations.
  • We now also pin each paired claim's value in the expected file, so a value we change in Ingest the Autumn Budget 2026 harvest (tranche 2 of #230) #247 shows in the diff.
  • We kept "PE's own orientation" in the sign annotation, because signs.py reads that phrase to orient an exported row, and we use the same phrases on AB2025's rows.
  • We merged tranche 2 at c58128e, and its ingest drops HM Treasury's two red book repetitions of the Household Energy Payment's 2026-27 cost (73043569, 73a625cf), which we tallied as devolved shares, so the stager reads 370 claims. No paired claim, value or cell moved.

The review of eb7c13b

The reviewer of eb7c13b found that three measures' totals cover more than our run does, and that we compare a producer's figure on four rows with a year our run leaves at zero. We now tally 32 more claims and keep 35 comparisons, down from 67.

  • We tally HM Treasury's six fuel duty rows, since HM Treasury scores the duty on every litre of taxable fuel in the UK and we estimate the duty and VAT on the fuel households buy. The OBR's fuel duty rows never reach the comparison, because we mark them as restated in the harvest and the ingest drops restated rows.
  • We keep the Carer's Allowance and property income comparisons and annotate each with the Scottish lines the OBR prints for that year, which the stager reads from the OBR's component claims. We name the nations each run covers in both of the dashboard's receipts, so we misstated these two measures in the last hand-off ("Those receipts state no excluded share").
  • We tally the Pension Tax Allowance's four rows for 2030-31 and 2031-32 because the government ends the allowance on 5 April 2030; we estimate no later change while both producers count receipts.
  • We tally the 22 claims where we and the producer both estimate no change. We attach zeroes before measures start in AB2025, but here we exclude them, because both sides agree only on the dates.
  • We annotate the Pension Tax Allowance's six remaining exchequer rows because we estimate the change in government revenue less spending, including any change in means-tested benefits, while the OBR counts no welfare change.
  • We corrected this description's account of the gainer column, which counts changes in income tax, National Insurance, student loan repayment and employment income, not income tax alone.
  • We list the local executor only on the 6.11.1 pair, and in its note we name the image the dashboard's Modal runs used.
  • We rewrote the reasons and annotations without internal keys, and we now pin each paired claim's file, column and row in stage_uk_ab2026_expected.json.

🤖 Generated with Claude Code

Every receipt of the UK Autumn Budget dashboard's Budget-day runs records
policyengine 6.11.1, policyengine-uk 2.151.0, policyengine-core 3.33.3 and
the Microcosm UK cut of 25 October 2026. The registry's engine_pin asks for a
pair record before tranche 3 ingests any counterpart.

No receipt states the package RECORD or wheel digests, the packaged bundle
manifest digest, the dataset's size or a requirements freeze. We take those
from the replay variant the receiver qualified for the same pair (#242), and
copy its requirements freeze byte for byte, so the record's identity digest
equals that variant's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6GLASJqd8EV4budnw3
@vercel

vercel Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-scorecard Ready Ready Preview Nov 5, 2026 9:20pm UTC

Request Review

pipeline/stage_uk_ab2026.py vendors the eight pe_modifier receipts and the
seven CSVs a pairing reads from PolicyEngine/uk-autumn-budget-dashboard-2026
at 1eca3182, and writes MANIFEST.json with each file's sha256 and source. It
checks each receipt against the registry and each CSV against its receipt.

It then pairs each of the 372 claims on those measures with one cell, or
tallies one of 13 written reasons. 67 claims take a counterpart: the whole
exchequer effect from HM Treasury's Table 6.1 and the OBR's Table 1.1 totals
against budgetary_impact.csv, and HM Treasury's count of Pension Tax
Allowance gainers against age_impact.csv. stage_uk_ab2026_expected.json pins
the counts per measure.

ingest_uk_ab2026 gains STAGED_COUNTERPARTS and advance_lanes. build_db.py
calls neither, so the site doesn't change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6GLASJqd8EV4budnw3
tests/test_uk_ab2026_counterparts.py checks that every vendored file hashes
to the manifest, its receipt and the registry; that the 6.11.1 pair record
states what each receipt states; that each counterpart re-derives from its
cell byte for byte; and that the accounting closes. Hypothesis flips claims'
signs, shuffles and samples the claims. A temporary database built with the
AB2026 ingest and these results shows each comparison once, and
advance_lanes moves only the official lane.

The AB2025 differential test now also compares stage_uk_ab2025's output, the
attached counterparts and the advanced lanes with origin/main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6GLASJqd8EV4budnw3
The dashboard counts each year's simulated liabilities and payments, while
HM Treasury and the OBR score receipts when they reach the Exchequer. For
example, HM Treasury puts £35m of property income yield in 2027-28 against
our £399m. Each exchequer counterpart now names that axis. A README in
results/uk/ab2026/ says what we vendor and how the stager uses it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6GLASJqd8EV4budnw3
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review

Head: eb7c13b2d685b037c674a5d66b17eb71e98cc347, on uk/ab2026-tranche2 at 3812ce4, which hasn't moved since we branched.

Steps

  1. Pair record (8d2d9f5). We added pair-pe6.11.1_uk2.151.0_microcosm-uk-2024-25-national-20261025T091414Z-a6299504. The receipts state the versions, the dataset's revision, digest and URI, the certificate and the data years. No receipt states these fields the record requires: the package RECORD and wheel digests, packaged_bundle_manifest_sha256, dataset.size_bytes and the requirements freeze. We took them from main's qualified 6.11.1 replay variant (Certify the policyengine.py 6.11.1 replay results #242) and copied its freeze byte for byte. The identity digest equals that variant's, 20f9f81d….
  2. Vendoring (b11011b). We vendored 15 files at 1eca3182 into results/uk/ab2026/: eight receipts and seven CSVs (six budgetary_impact.csv and the Pension Tax Allowance's age_impact.csv). The vendor step checks each receipt against receipt_file_sha256 and each CSV against its receipt's files map. We copied no other CSV, because no counterpart reads one.
  3. Pairing (b11011b, eb7c13b). pipeline/stage_uk_ab2026.py pairs or tallies all 372 claims on the eight measures. We read calculators.py at the pin for each column we use: budgetary_impact.value gives Δgov_balance in £bn, and people_directly_affected counts people whose own income tax, NI, student loan repayment or earnings move by £1 or more. Signs go through signs.classify, and we orient our value to the claim, as AB2025 does.
  4. Accounting. pipeline/stage_uk_ab2026_expected.json pins the counts per measure and in total. check raises on drift, and classify raises on any claim shape no rule names.
  5. Results and lanes. We stage the rows in AB2025's campaign form under results/uk/staged_ab2026/. ingest_uk_ab2026 gains STAGED_COUNTERPARTS and advance_lanes.
  6. Later runs. See the table below.

Tests

  • Focused, -n 4: the new test_uk_ab2026_counterparts.py (20 tests) with test_uk_ab2026_ingest.py, test_ab2026_registry.py, test_uk_certified_pairs.py, test_uk_ab2025_counterparts.py and test_signs.py gave 141 passed and 9 skipped.
  • The extended AB2025 differential test passes against origin/main. It failed, as it should, when we changed one word in stage_uk_ab2025.py's annotations.
  • Locally, build_db followed by git diff --exit-code shows no drift.
  • The full suite locally, -n 4, gave 3,516 passed and 2 failed, and neither failure involves this branch. test_app_data_copies_match_committed_data passes alone, so its failure came from parallel tests rewriting lanes.json. The Modal factory test needs a Python 3.12 interpreter, and this session runs 3.13.
  • CI on eb7c13b passes every job, pytest · ruff format included.

Headline counts

Of the 372 claims, 67 take a counterpart: 36 HM Treasury Table 6.1 lines, 30 OBR Table 1.1 totals and one HM Treasury gainer count. We tally 305 under 13 reasons, listed in the PR body. Among the 67, 36 have the same sign on both sides, 26 read zero on both sides, and none has opposite signs. In four, the producer prints a figure where our run gives zero: the Pension Tax Allowance in 2030-31 and 2031-32. In one, HM Treasury prints zero and we estimate −£110m: the trading allowance in 2027-28.

Judgment calls

  • We tally the Household Energy Payment's 26 exchequer claims. Its receipt says the run doesn't model the Barnett shares, and the claims count the UK cost (£2.6bn against our £1.97bn). We rejected comparing them as constructed with an annotation.
  • We compare the other measures' UK totals, even where the OBR's breakdown names a Scottish block grant adjustment. Those receipts state no excluded share. We rejected tallying them on the OBR's component lines alone.
  • We compare across timing bases and annotate it. The dashboard counts each year's liabilities, and producers score when receipts arrive (property income in 2027-28: £35m against our £399m). We rejected a tally, since the measure, year and quantity match.
  • We pair HM Treasury's 500,000 Pension Tax Allowance gainers with people_directly_affected (665,907). The allowance moves only income tax, and only down. We rejected pairing the 290,000 taken out of tax, since no column counts them.
  • The pair record's identity follows Certify the policyengine.py 6.11.1 replay results #242's, including Python 3.12. We rejected binding the receipts' 3.13.3 runtime, since no freeze or digests exist for the dashboard's environment.
  • All 67 rows take constructed, against current law at 2.151.0. We rejected comparable, since every row differs on static scoring, baseline world or timing.
  • We vendor only the files a counterpart reads, plus every receipt. We rejected vendoring the full artifact sets.

Later runs

The dashboard's main (9ad634cb, 268 commits on) regenerated every artifact on 29 October with the same policyengine 6.11.1, policyengine-uk 2.151.0 and dataset. For each of the seven measures, budgetary_impact.csv, age_impact.csv, winners_losers.csv and metrics.csv match the pinned files byte for byte. Only the receipts differ.

measure (£bn) 2026 2027 2028 2029 2030 2031
household energy payment, pin = main −1.969 0 0 0 0 0
electricity VAT, pin = main −0.840 −0.353 0 0 0 0
fuel duty, pin = main −0.301 −0.521 0 0 0 0
carer's allowance taper, pin = main 0 0 0 0 −0.024 −0.048
trading allowance, pin = main 0 −0.110 −0.118 −0.119 −0.120 −0.130
pension tax allowance, pin = main 0 −0.030 −0.083 −0.144 0 0
property income, pin = main 0 0.399 0.412 0.429 0.446 0.459
combined seven, pin −3.110 −0.615 0.210 0.166 0.301 0.281
combined eight, main (adds the bus fare cap) −3.174 −0.805 0.210 0.166 0.301 0.281
bus fare cap, main only −0.064 −0.190 0 0 0 0

The dashboard labels each year by its fiscal year's start. Only the combined run and the new bus fare cap differ, and no claim pairs with either, so the choice of vintage changes no counterpart here.

What wiring it in would add

We built a database in a scratch worktree with build_db.py calling the AB2026 ingest, verdicts, this staging and advance_lanes, then ran export_populations.

  • Lanes: three new lanes in data/lanes.json. uk-ab2026-official reads computed with "67 claims with a PolicyEngine counterpart from the dashboard's Budget-day runs"; the microsimulation and admin-other lanes read ingested.
  • Claims: 4,691 new rows in data/populations.json (the total grows from 5,896 to 10,587): 3,130 pe_gap, 1,494 not computed and 67 constructed.
  • On the page: each constructed row shows our value beside the producer's, with sign_relative_to_pe same. That reads result_oriented_to_claim for 66 rows and level for the gainer count. Each row also shows a ratio and the annotations naming the axes.

We didn't check data/comparison.json, which build_comparison.py writes separately.

Not done

  • We ran no microsimulation. No claim needed a new run to answer as it stands, but a household-level breakdown on the dashboard's deciles would. The microsimulation lane's 95 vigintile claims and 3 spending-decile claims would each need a run on the producer's breakdown.
  • The pair record takes its digests from Certify the policyengine.py 6.11.1 replay results #242, not from the receipts, since the receipts state none.

Generated by Claude Code

MaxGhenis pushed a commit that referenced this pull request Nov 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis pushed a commit that referenced this pull request Nov 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis pushed a commit that referenced this pull request Nov 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis pushed a commit that referenced this pull request Nov 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stager now tallies HM Treasury's six fuel duty rows (it scores every
litre of taxable fuel; we estimate household fuel), the Pension Tax
Allowance's four rows after it ends on 5 April 2030, and the 22 years
where the claim and our run both read zero. Carer's Allowance and
property income rows keep their comparison and name the OBR's Scottish
lines for the year, read from the OBR's component claims at stage time;
the Pension Tax Allowance rows say our figure nets off means-tested
benefits. Reasons and annotations take plain sentences, the unreachable
sign_not_this_quantity reason goes, the expected file pins each paired
claim's file, column and row, and the 6.11.1 pair lists the local
executor only. Comparisons fall from 67 to 35; tallied claims rise from
305 to 337.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JktXnPP5pkAJncdhu
…erparts tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JktXnPP5pkAJncdhu
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review

Head: 229bbc1 on uk/ab2026-tranche3 (commits 9770c6a, 229bbc1 on top of eb7c13b). origin/uk/ab2026-tranche2 still sits at 3812ce4, so nothing needed merging or re-pinning.

Findings and what we did

finding what we did
F2 fuel duty We reproduced it: fuel_duty in policyengine-uk 2.151.0 carries the label "Fuel duty (cars only)" and multiplies household petrol_litres and diesel_litres, and the receipt says the run includes "the VAT on the lower pump price". classify now tallies all six HM Treasury rows under fuel_duty_base before the zero test. We removed the fuel duty budgetary_impact.csv from the vendored set, since no counterpart reads it, and re-vendored the manifest from the pinned commit.
F1 Carer's Allowance We reproduced it from the receipt ("In England and Wales") and the OBR's Current AME: Scottish BGA (current) claims (−£5m in 2030-31, −£10m in 2031-32). The four nonzero rows keep their comparison and carry the brief's annotation, with the amount the stager reads from the OBR's claim for that year on both HM Treasury's and the OBR's rows; a missing claim raises Unmatched.
F3 property income We reproduced it from the receipt ("in England, Wales and Northern Ireland") and the OBR's Scottish BGA (£27m to £29m) and self-financed expenditure (£5m to £6m) claims. The ten nonzero rows carry the brief's annotation with both amounts read at stage time.
F5 Pension Tax Allowance after 5 April 2030 We reproduced it: the cell reads 0.0 in 2030 and 2031 against +£50m and −£15m (HM Treasury) and +£50m and −£16m (OBR). MEASURE_ENDED limits the reason to this measure and these two years, requires a zero cell, and any other nonzero claim against a zero cell raises.
F9 zero rows We tally the rows where the claim and the cell both read zero; the test reads each claim's value and each cell. 22 rows take the reason, since the six fuel duty rows (four of them zero) now go to fuel_duty_base first. The description says why we depart from AB2025.
Item 6, the Pension Tax Allowance's cost See below. We annotate the six remaining rows.
F7 executor executor reads ["local"], and the note names the image src/uk_budget_data/modal_generate.py builds at 1eca3182: Python from the dashboard's .python-version (3.13; the receipts record 3.13.3) and its uv.lock (sha256 92a05b09…).
F8 We deleted sign_not_this_quantity and its branch.
F10, F12 Every reason and annotation now reads as section F gives it, with the three corrections: "often in a later year"; HM Treasury's half of the baseline annotation says "HM Treasury's table names no baseline", because none of HM Treasury's 44 revenue_change claims carries a baseline_policy, while all 189 of the OBR's carry obr_pre_measures_autumn_budget_2026; the sign annotation keeps "PE's own orientation", and signs.py stays untouched.
F11 test_every_exchequer_row_carries_timing_baseline_sign_and_scope asserts the timing, baseline, sign and fiscal-year annotations on every exchequer row, and builds each scope annotation's expected text from the OBR's claims independently of the stager.
Section E scope test test_every_sub_uk_measure_takes_a_tally_or_a_scope_annotation reads each receipt's dashboard.description. Three name a nation: Carer's Allowance, the Household Energy Payment and property income. The property income receipt's "Scottish landlords' included" refers to finance cost relief, and the measure takes its scope annotation either way. The test requires the Household Energy Payment's tally or, for a measure with exchequer rows, a scope annotation the test builds itself on every row.
Rebasing stage_uk_ab2026_expected.json now pins each computed row's claim id, file, column and row, and check raises "counterpart pairing drifted" on any change. test_the_cell_year_and_the_claim_year_agree derives the year from the claim's fy and from the cell's row and compares them, after checking the receipt's year_label reads start_year.
F4, F6 and the description We corrected the gainer sentence and the "Those receipts state no excluded share" claim, applied section F's sentence table, said that CI doesn't run the AB2025 differential until #247's fix lands here, dropped "the measure, year and quantity match", and replaced the wiring counts with the run below. The description's prose measures 47 sentences, mean 23.1 words, median 23, longest 36.
Docs The module docstring states the scope rule and no longer says the claim and cell "measure the same thing"; the README bullet drops its label colon and counts five CSVs.

Not applied

We applied every item. The scope test, as the brief specifies it, leaves one gap. The electricity VAT receipt says the zero rate "applies in Great Britain from October 2026 and across the UK from April 2027", and since "Great Britain" isn't among the names, the test requires no tally or annotation there. We didn't decide that measure's scope, since HM Treasury's and the OBR's rows on it would need reading against the costing first.

Counts by reason

outcome at eb7c13b now
counterpart 67 35 (17 HM Treasury, 17 OBR, one gainer count)
exchequer component 141 141
vigintiles after housing costs 95 95
Barnett shares not modelled 26 26
zero on both sides – 22
share of households 14 14
typical bill 8 8
fuel duty base – 6
worked example 6 6
cumulative real terms 4 4
allowance ended – 4
household count 3 3
spending deciles 3 3
motorists 2 2
benefit recipients 1 1
taken out of tax 1 1
no fiscal year 1 1
total tallied 305 337

The brief expected 35 and 337, and the run gives both. The diff of ab2026_counterparts.jsonl removes 32 rows (22 zero, 6 fuel duty, 4 allowance ended) and changes only the annotations field of the 35 that remain; no pe_value, trace or pairing moved.

Item 6: the Pension Tax Allowance's cost

  • calculators.py:86-93 at the pin sums the change in gov_balance. In policyengine-uk 2.151.0, gov_balance adds gov_tax and subtracts gov_spending, whose list includes housing_benefit, pension_credit and universal_credit, and pension_credit_earnings_income_tax.py reads income_tax. So our figure nets off the change in Pension Credit, and other means-tested benefits, that follows from lower income tax.
  • The OBR's Current AME: Welfare inside cap claim reads 0 for this measure in every year, and the stager raises if it ever reads otherwise.
  • No artifact at the pin states the measure's income tax alone: budgetary_impact.csv gives the net total, and metrics.csv gives people affected, Gini and poverty. So the annotation states no income tax amount, and the post's −£42m stays unverified.
  • The same netting runs through every measure's gov_balance, and property income and the trading allowance also move income tax. Neither the OBR's breakdown nor any artifact at the pin lets us size it on those measures, so we annotate none of them. Carer's Allowance carries an OBR welfare line (−£33m and −£69m), and its receipt says the run counts the allowance as unearned income for Universal Credit; we left that measure's annotation to the brief's scope text.

Mutations

Each mutation ran in its own worktree twice: A against the committed staging, B after stage regenerated it. We ran the counterparts and ingest test files with -x.

mutation A B (regenerated)
drop the timing annotation killed: committed-staging test killed: annotation test
drop the baseline annotation killed: committed-staging test killed: annotation test
drop the Carer's Allowance scope annotation killed: committed-staging test killed: annotation test
drop the Pension Tax Allowance netting annotation killed: committed-staging test killed: annotation test
type £10m in place of the amount the claim states killed: committed-staging test killed: annotation test (2030-31 reads £5m)
pair the fuel duty rows again killed at collection: no vendored CSV stage refuses: no vendored CSV
pair the Pension Tax Allowance's 2030-31 and 2031-32 rows again killed: committed-staging test stage refuses: accounting drifted
pair the zero rows again killed: committed-staging test stage refuses: accounting drifted
flip the sign killed: committed-staging test killed: rederive test
shift one row's year in the staged file killed: committed-staging test –
shift one row's year in the expected file's pairing killed: pinned-pairing test –

Nothing survived. If someone copied the drifted tally and pairing into the expected file, a named test would still fail. test_zero_rows_read_zero_in_the_claim_and_the_cell requires every paired exchequer row to read nonzero, and the fuel duty and ended-allowance tests assert each row's reason. We read those tests for this; we didn't run that double mutation.

Tests

The focused command (-n 4, five files) gives 140 passed and 9 skipped; ruff format --check passes on CI's paths. The skips need a policyengine-uk 2.151.0 venv (one) and an in-tree dashboard checkout (eight).

Wiring counts

We built a temporary database (ScorecardDB, ingest_uk_ab2026.ingest with a copied lanes feed, ingest_verdicts, ingest_campaign on the staged counterparts, advance_lanes), then ran export_populations.export into a scratch path. The database holds 4,717 AB2026 claims, 1,552 of them without a result. The export carries 4,691 rows, 3,130 pe_gap, 1,526 not_computed and 35 constructed, since it drops the 26 Tax Policy Associates claims that have neither a result nor publish_without_result. At eb7c13b the same 26 explain the gap between the review's 1,520 and the hand-off's 1,494, so both held; the tally adds 32 to each. advance_lanes reads 35 answered of 3,839 on the official lane, and 0 on the microsimulation and admin-other lanes.

What we couldn't verify

  • The post's −£42m of income tax, which no artifact at the pin states.
  • HM Treasury's costing PDF (page 13, "every litre of taxable fuel") and the Budget document's sentence on property income rates. We took both from the brief and checked the receipts' wording, not the PDFs.
  • build_comparison.py's output and the app's rendering of the new annotations.
  • CI on 229bbc1, which hadn't reported when we posted.

🤖 Generated with Claude Code


Generated by Claude Code

MaxGhenis pushed a commit that referenced this pull request Nov 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…drop

#247 drops HM Treasury's two red book repetitions of the payment's
2026-27 cost (73043569, 73a625cf), which tranche 3 tallied as devolved
shares, so the tally falls from 372 to 370. No paired claim, value or
cell moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JktXnPP5pkAJncdhu
Each kept exchequer row now names every nonzero line of the OBR's
breakdown for its measure and year outside the receipts heads and the
welfare line (Scottish block grant adjustment, Scottish self-financed
expenditure, departmental resource spending, other AME), with the amount
read from the OBR's claim; an unnamed line raises. Electricity VAT rows
say HM Treasury's costing counts charity, residential care and a little
business use; Carer's Allowance rows say we leave Northern Ireland at
current law; property income names UK-wide finance cost relief. The
netting, gainer and two reasons' text follow the 229bbc1 review. The
tests build the expected lines from the OBR's claims, read Great Britain,
GB, Scottish and Welsh in receipts, feed a nonzero welfare line, and the
expected file pins each paired claim's value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JktXnPP5pkAJncdhu
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review

Head: 2cfdf22 on uk/ab2026-tranche3. It merges origin/uk/ab2026-tranche2 at c58128e (0136b01), and c58128e still heads that branch as I post. The commit after the merge, 2cfdf22, applies the 229bbc1 review. I didn't mark the PR ready or merge it.

What changed, by ruling

ruling what we did
R1, R3 scope_annotations now adds one sentence to every kept exchequer row, on both producers' rows, naming each nonzero Current AME:/Current DEL: line of the OBR's breakdown for that measure and year, apart from Welfare inside cap, with the amount read from the OBR's claim when the stager runs. Example: "The OBR's breakdown of this measure also shows £8m of Scottish block grant adjustment and £7m of Scottish self-financed expenditure in this year, which our run doesn't model." A line with no reader name raises Unmatched. The trading allowance (10 rows), the Pension Tax Allowance (6, including £1m of departmental resource spending in 2027-28 to 2029-30) and electricity VAT (4, other AME £21m and £8m) now carry it, as do Carer's Allowance and property income.
Electricity VAT costing The four rows stay and carry "HM Treasury's costing counts household, charity, residential care and a small amount of business consumption, and we estimate household electricity only." The description records the call and the contrast with fuel duty.
R2 The annotation test builds each row's expected amounts from the OBR's measure_component claims, outside the receipts heads and welfare, and requires exactly that many amounts in the one breakdown sentence (or none). The scope test reads England, Wales, Scotland, Northern Ireland, Great Britain, GB, Scottish and Welsh as whole words, finds four measures, and requires a tally or the stager's scope annotations on every row. Invariant 8 now states what the code enforces.
R4 "We apply the taper in England and Wales and leave Northern Ireland's Carer's Allowance at current law." The four rows stay.
R5 The property income sentence adds finance cost relief for landlords across the UK.
R6 "We net off any change in means-tested benefits that follows from lower income tax, and the OBR's breakdown shows no change in welfare spending."
R7 Your text, as two sentences.
R8 Skipped as ruled. The description says signs.py reads the phrase and AB2025's rows carry it.
R9 measure_ended and devolved_share_not_modelled take the review's text, and the description says "the allowance lowers only income tax".
R10 The description ends with the required 🤖 line.
R11 pairing pins each paired claim's claim_value, and check raises "pairing drifted" when a value moves; a test moves one value by £1m.
R12 A test sets the Pension Tax Allowance's welfare line to −£5m and expects Unmatched, and another feeds an unnamed AME line.
Section E The description now says "17 OBR measure totals from the 'Scorecard measures' section of its breakdown, each keyed to its Table 6.1 line", which is what the claims' sheet_section, component "Total" and table_61_line show. It also takes the review's replacement for "Reasons and annotations read as plain sentences", and invariant 9 says CI now fetches origin/main for the AB2025 differential. Its prose runs 62 sentences, mean 23.7 words, longest 36.

The merge

c58128e drops HM Treasury's two red book repetitions of the Household Energy Payment's 2026-27 cost of £2.6bn (73043569, 73a625cf), which tranche 3 had tallied under devolved_share_not_modelled. The stager reads 370 claims, not 372. No paired claim id, claim value or cell changed, and ab2026_counterparts.jsonl came out byte-identical after the merge, so the expected file changed only in those two tally counts. The one conflict, in test_ab2025_ingest_matches_main's docstring, kept both sides: tranche 3's docstring and #247's fail-in-CI guard.

Counts by reason

outcome at 229bbc1 now
counterpart (17 HM Treasury, 17 OBR, one gainer count) 35 35
exchequer component 141 141
vigintiles after housing costs 95 95
Barnett shares not modelled 26 24
zero on both sides 22 22
share of households 14 14
typical bill 8 8
fuel duty base 6 6
worked example 6 6
cumulative real terms 4 4
allowance ended 4 4
household count 3 3
spending deciles 3 3
motorists 2 2
benefit recipients, taken out of tax, no fiscal year 1 each 1 each
total 372 370

Tests and mutations

The focused command (-n 4, the five files the brief names) gives 146 passed and 9 skipped. The skips need a policyengine-uk 2.151.0 environment (one) or an in-tree dashboard checkout (eight). ruff format passes on the files I touched.

I ran 13 mutations at 2cfdf22, each in its own worktree: B after stage regenerated the staging, A against the committed pin. All 13 were killed.

mutation outcome
drop the breakdown sentence annotation test
type £10m for each line's amount annotation test
skip the self-financed expenditure line annotation test
drop "benefit payments" from the timing sentence annotation test
drop the electricity costing sentence electricity test
name Scotland in place of Northern Ireland on Carer's Allowance Carer's Allowance test
disable the welfare guard (if False:) welfare test (R12's survivor)
flip the sign rederive test
pair fuel duty again stage refuses: no vendored CSV
pair the ended allowance years again stage refuses: accounting drifted
pair the zero rows again stage refuses: accounting drifted
move one paired claim's value in the pin pinned-pairing test
shift one paired row's year in the pin pinned-pairing test

Left undone

  • I didn't rerun export_populations after the merge. The two dropped Household Energy Payment claims carried no result, so the earlier 1,526 not-computed rows should fall by two at most. I haven't checked that.
  • What the OBR's other AME line on electricity VAT holds, and whether the OBR's VAT line counts Northern Ireland from April 2027, still rest on the costing note, which I didn't read. The annotation states the amount only.
  • CI on 2cfdf22 hadn't reported when I posted.

🤖 Generated with Claude Code


Generated by Claude Code

MaxGhenis pushed a commit that referenced this pull request Nov 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis pushed a commit that referenced this pull request Nov 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis pushed a commit that referenced this pull request Nov 5, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-pins the tally after merging tranche 2 at 5ff3c4b, which drops three
IFS spending-decile claims (1b2e6f7f, 35db1466, 61178ad2), so 367 claims
and no spending_decile_breakdown reason; no paired claim, value or cell
moves.

The stager now skips only the receipts heads, the welfare line and the
property behavioural line, raises on any other unnamed line and on two
claims sharing a named line, and writes each line as a cost or a gain
with "we" as the actor. Property rows from 2028-29 carry the OBR's £0.1
billion behavioural erosion, read from its claim; trading allowance rows
carry the receipt's caveat on gross receipts. The gainer, property,
electricity, pension and measure_ended text follow the 2cfdf22 review.
The tests build each breakdown sentence from the claims with their own
names, articles and signs, and assert every scope sentence in full.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6JktXnPP5pkAJncdhu
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review

Head: b8a8a11 on uk/ab2026-tranche3, which merges origin/uk/ab2026-tranche2 at 002d90c, its head when I posted. Commits since 2cfdf22: the merge of 5ff3c4b (f28367f), the review fixes (69275e3), the merge of b227956 (04224e6), one test (3e10674) and the merge of 002d90c (b8a8a11). I didn't mark the PR ready or merge it.

The merges

  • 5ff3c4b drops three IFS claims that split households by spending (1b2e6f7f, 35db1466, 61178ad2, on the payment, fuel duty and electricity VAT). The stager reads 367 claims, not 370, and spending_decile_breakdown has no claims left. I deleted that reason and its branch, so a claim on a spending axis now raises Unmatched rather than taking an unused reason.
  • b227956 and 002d90c add tests to test_uk_ab2026_ingest.py only, and the claims came out identical.
  • None of the three merges moved a paired claim id, claim value, file, column or row. stage never raised "pairing drifted", and the pairing block of stage_uk_ab2026_expected.json is byte-identical to 2cfdf22's. I re-pinned only the tally: 367 claims, Household Energy Payment 59, fuel duty 28 and electricity VAT 74, each losing its one spending-decile claim.

Findings

finding what we did
F1 unmodelled_lines skips only names starting "Receipts:" or "Tax head (receipts)", the welfare line, and the behavioural line on a measure in BEHAVIOURAL_YEARS. It raises on any other nonzero measure_component without a reader name. components() now reads only row_type == "measure_component". A new behavioural_annotation adds "The OBR expects more tax-motivated incorporation and higher non-compliance to cut the yield by £0.1 billion a year, and we hold incomes fixed." to the property rows for 2028-29 to 2031-32 from both producers (eight rows) and to none in 2027-28. It reads the amount from claim f59d84db, and raises unless exactly one such claim exists. The test's _breakdown writes out the same exclusions itself. The docstring, the description bullet and invariant 8 say what the code does.
F2 The test maps each OBR line to its reader name in its own READER_NAME, builds the whole expected sentence and compares it for equality, so each amount binds to its line. M1 (swap the names) now dies.
F3 Both gainer strings carry your text, and the test asserts each one in full.
F4 cost_or_gain reads the OBR's sign (negative is a cost). lines_annotation writes "which we don't model" for one line, "and we model neither" for two, and "and we model none of them" for three. article gives "an" where the amount starts with 8, or is 11 or 18 (and 11,000 to 11,999 or 18,000 to 18,999). test_the_article_follows_the_spoken_amount checks 1, 5, 8, 10, 11, 18, 21, 80, 89, 90, 100, 110, 180, 800 and 11,000. test_a_lines_sign_chooses_cost_or_gain checks both directions and asserts row 27's full sentence: "an £8m cost from the Scottish block grant adjustment and a £7m gain from Scottish self-financed expenditure". The property rows now read "a £27m gain" for the Scottish block grant adjustment, since the OBR prints +£27m there.
F5 SCOPE writes out each measure's sentence in full (Carer's Allowance, property, electricity, Pension Tax Allowance, trading allowance), and test_each_measure_carries_its_scope_sentence_in_full asserts it on every row. M3 and M4 now die.
F6 We took the review's three replacements, and the property sentence ends "as the government does from 6 April 2027."
F7 gh api -X PATCH swaps the description's last line for _Generated by [Claude Code](https://claude.ai/code)_, which is why R10 never landed. I set the body through the GitHub connector, and pulls/249 .body now ends with "🤖 Generated with Claude Code".
F8 Rerun at b8a8a11, as in section C (copied lanes feed, ingest, ingest_verdicts, ingest_campaign, advance_lanes, then export_populations.export into a scratch path, with the lanes argument left out). The database holds 4,686 AB2026 claims, 3,206 with a result: 35 constructed and 3,171 pe_gap. The export carries 4,660 rows: 3,171 pe_gap, 1,454 not_computed and 35 constructed. The lanes read official 35 of 3,828, microsimulation 0 of 585 and admin-other 0 of 273. 04224e6 gave the same counts. The description states them.
F9 components() raises "two OBR claims share this line" on a second claim for one (measure, year, line), except on the receipts heads. Tests add a twin of a Scottish block grant adjustment claim, which raises, and a twin of a receipts head with the opposite sign, which passes. Today's harvest raises nowhere.
F10 Each trading allowance row carries your two sentences, in full in SCOPE.
Section E The description takes the review's replacements for the electricity sentence, "The annotations name…" and "raises on any other line it can't name". The module docstring takes its electricity sentence and states the F1 skip list. On REASONS["measure_ended"] I made a call, below.

Calls I made

  • I didn't take "count lower receipts" for measure_ended. HM Treasury prints +£50m for 2030-31 and the OBR +£50m, so in that year both count higher receipts. The reason now reads "…while HM Treasury and the OBR count a change in receipts in it."
  • article covers the amounts the ruling names and the larger cases with the same leading sound. The test's own rule lists 8, 11, 18, 80 to 89 and 800 to 899, so it can't follow the stager by construction.
  • N1 (skip the behavioural line on every measure) survived the first pass, because only property income carries one today. I added test_a_behavioural_line_on_another_measure_raises, which feeds one to the trading allowance and expects Unmatched, and N1 then died.
  • The brief named five test files, and the earlier review's "six" included one more. I ran the five plus tests/test_ab2026_key_decisions.py, since the merges touched it.

Counts by reason

outcome at 2cfdf22 now
counterpart (17 HM Treasury, 17 OBR, one gainer count) 35 35
exchequer component 141 141
vigintiles after housing costs 95 95
Barnett shares not modelled 24 24
zero on both sides 22 22
share of households 14 14
typical bill 8 8
fuel duty base 6 6
worked example 6 6
cumulative real terms 4 4
allowance ended 4 4
household count 3 3
spending deciles 3 –
motorists 2 2
benefit recipients, taken out of tax, no fiscal year 1 each 1 each
total 370 367

The diff of ab2026_counterparts.jsonl against 2cfdf22 changes only the annotations field of the 35 rows.

Tests

At b8a8a11, the five files plus test_ab2026_key_decisions.py (-n 4 -rs) give 181 passed and 9 skipped. The skips need a policyengine-uk 2.151.0 environment (one) or an in-tree dashboard checkout (eight). ruff format --check passes on CI's paths.

Mutations

Each mutation ran at 04224e6 in its own worktree. I mutated the stager, ran stage (which regenerates the staging) and then pytest tests/test_uk_ab2026_counterparts.py -x. All 22 die; N1 needed the added test.

mutation outcome
H1 drop the breakdown sentence killed: annotation and cost-or-gain tests
H2 type £10m for each amount killed: annotation and cost-or-gain tests
H3 drop the self-financed expenditure name stage raises "no reader name"
H5 drop the electricity sentence killed: scope-sentence test
H6 Scotland in place of Northern Ireland killed: scope-sentence test
H7 disable the welfare guard killed: welfare test
H8 flip the sign killed: rederive test
M1 swap the block grant and self-financed names killed: annotation test
M3 drop "and finance cost relief for landlords across the UK" killed: scope-sentence test
M4 "we estimate it on all of it, not electricity only" killed: scope-sentence test
M5 drop the departmental spending name stage raises "no reader name"
M6 "households with any income" killed: gainer test
N1 skip the behavioural line on every measure killed: behavioural-line test (added in 3e10674)
N2 drop the behavioural sentence killed: behavioural test
N3 add it to 2027-28 killed: behavioural test
N4 type £0.2 billion killed: behavioural test
N5 every line a cost killed: cost-or-gain test
N6 always "a" killed: article and cost-or-gain tests
N7 drop the collision check killed: collision test
N8 restore the old prefix filter killed: unnamed-line test ("a new line")
N9 drop the trading allowance sentence killed: scope-sentence test
N10 "neither" for three lines killed: annotation test

Sentence lengths

  • The description has 82 sentences outside tables and headings, mean 24.8 words, median 26, longest 39, with none over 40.
  • The annotations hold 33 distinct sentences, longest 37: the Pension Tax Allowance's three-line breakdown sentence. The shortest is the trading allowance's "As a result, we may overstate the cost for some people and understate it for others." (17 words), which follows a 25-word sentence.
  • No paragraph or annotation runs three short sentences in a row.

Left undone

  • I didn't open HM Treasury's or the OBR's PDFs. The paragraph 3.17 reading and the page 12 citation rest on your check and the harvested text.
  • CI on b8a8a11 hadn't reported when I posted.

Generated by Claude Code

MaxGhenis and others added 2 commits November 5, 2026 07:20
…26-tranche3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The baseline annotation on HM Treasury's rows now reads "HM Treasury names no
baseline in its table", so `results/uk/staged_ab2026/ab2026_counterparts.jsonl`
changes on those 18 rows and no paired claim, value or cell moves. The rest
changes only docstrings and comments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits November 5, 2026 08:59
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merging tranche 2 at 0193b91 brought IFS row 46 to the stager, because #247
scores it on average_household_income_change. IFS averages the Household Energy
Payment across households in England, and we publish an average across all UK
households, so we tally the claim under a 16th written reason.

We also add the Universal Credit limit to the Carer's Allowance rows, name the
actors in the reasons and docstrings, and raise on a named OBR line under another
sign convention. The tests now compare each row's whole annotation list, reject
claims of another geography or unit or on the OBR costings lane, execute vendor,
and rederive each counterpart from the producer's sign convention.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#247 keeps OBR row 2740 on the borrowing metric, adding one pe_gap verdict.
No counterpart, claim or tally in this tranche moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits November 5, 2026 11:25
#247's last review changed its preservation test helper, the ingest's
printed-amount regex and its branch for registered programs. No claim moves, so
no counterpart, tally or unanswered row in this tranche moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In paragraph 3.17 the OBR says the measure raises £0.4 billion a year on
average from 2028-29, and that behavioural responses cut the yield by £0.1
billion a year. We had put the average on the £0.1 billion, so we drop it from
the behavioural sentence and the stager's comment. Eight staged annotations
change, and no value does.

Ten mutants that Sol ran at 44d226d survived the tests, though the committed
guards reject each input. We now also test:

- the gainer and exchequer guards on each nation, Great Britain, no geography
  and more units;
- the OBR costings lane on any key, and on the keys in its registry;
- exchequer claims without a readable sign;
- named and behavioural OBR lines under a reversed convention or none;
- the opposite-sign annotation on a flipped claim;
- each reason against the producers whose claims it covers, and the £1 minimum
  in the two count reasons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis changed the base branch from uk/ab2026-tranche2 to main November 5, 2026 16:54
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

We merged #247 and retargeted this PR to main, so we close and reopen it to run CI against the merge with main. The head stays at a58ddc3e.

@MaxGhenis MaxGhenis closed this Nov 5, 2026
@MaxGhenis MaxGhenis reopened this Nov 5, 2026
We answer L1 and L2 of the cloud review of a58ddc3.

- We now feed the unreadable-sign test the magnitude convention as well,
  so a stager that rejects only unknown signs fails it (mutant Y1).
- We write the 16 reasons out in the test and require the stager's to
  match, so a test fails if we change a word in any reason (mutants Y2
  and Y4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review November 5, 2026 21:41
@MaxGhenis
MaxGhenis merged commit e3240b2 into main Nov 5, 2026
15 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

We merged 75c4a1ff with a merge commit, under the merge-your-own-PR gates.

  • Independent review. An Opus 5.5 cloud session approved 75c4a1ff with one low finding, and we answered it in the description before merging (hub-notes/pr249-review-75c4a1ff). The same session requested changes at a58ddc3e over one false sentence in the description, and we answered that review in 75c4a1ff and the description (hub-notes/pr249-review-a58ddc3e). We'll fix its L4, a claim's aggregation on main, in the follow-up PR where we stage the OBR's paragraph 4.30 and Table 6.3 rows. GPT-6.1 Sol approved 44d226dc with three low findings, and we answered them in a58ddc3e. Subfleet held the Codex lanes this afternoon, so no Sol review ran at this head.
  • CI. gh pr checks returned 0 at this head, with all 15 checks listed, and GitHub reported the merge state as clean.
  • Scope. In this PR we paired 35 of the 368 claims on the registry's eight pe_modifier measures with the dashboard's Budget-day estimates, and we tallied the other 333 under 16 reasons. We changed the certified 6.11.1 pair, the stager and its pin, the vendored artifacts, the staged counterparts, the ingest's STAGED_COUNTERPARTS and advance_lanes, and their tests. We changed no harvested claim, no file under app/, and no lane data.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

This branch was successfully deployed

1 active deployment
Preview — 75c4a1ff Deployed Nov 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants