Skip to content

UK: take up #1095's actionable items on policyengine-uk 2.165.1 - #1422

Merged
juaristi22 merged 46 commits into
mainfrom
uk-1095-next
Nov 1, 2026
Merged

juaristi22 merged 46 commits into
mainfrom
uk-1095-next

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR takes the engine-free items of #1095 that María ruled on 29 October (Phase A), and the first re-bump past #1421 (Phase C): policyengine-uk 2.165.1. Main is merged in after #1420 and #1421, and the national and local target surfaces are regenerated on the pinned feed (Chronicle 53815b2). The spine and calibration items (Phase B) move to #1426, each measured in its own arm.

policyengine-uk 2.165.1

The lock moves from 2.164.2, which #1421 brought, to the newest release.

  • Other investment income in market, HBAI, net and capital income (2.164.3, Count other investment income in market and HBAI income policyengine-uk#2736; 2.164.4, Count other investment income in capital income, and test it in every income measure policyengine-uk#2746). The build writes it from National Savings and gilt interest on FRS rows and from the SPI's other investment income on SPI rows. The engine already taxed it, and now also counts it in market income, HBAI income, household net income and capital_income. The stages that read those incomes:
    • The LCFS, NEED and NTS stages band recipients on the engine's HBAI income with its deductions added back, so their gross income now counts it, as the surveys' gross incomes do.
    • was_wealth, and was_lisa through its predictors, read household net income against WAS's DVTotinc_bhcR8, whose investment income covers every account and investment WAS asks about, National Savings bonds and gilts included. was_wealth also reads capital_income as a predictor, which now matches the donor's gross income from all financial investments (DVGIINVR8).
    • etb_services reads the engine's HBAI net income, and etb_vat its HBAI gross income less the ETB's direct taxes, against the ETB's disinc. The ETB's original income includes investment income, and its records come from the Household Finances Survey from 2018. Whether that survey's investment income counts National Savings and gilt interest is still to check against its documentation.
  • Company car and car fuel benefits (2.165.0, Model the company car and car fuel benefits, with Class 1A NICs policyengine-uk#2750). The build writes none of the nine person inputs, so no one has a company car and both benefits and their Class 1A NICs are zero, as before. The inputs are signed at their default in UK_ENGINE_DEFAULT_INPUTS, for María's signature. The company car PR for the Autumn Budget dashboard supplies six of them.
  • Tax-Free Childcare over a smaller UC award (2.165.1, Keep Tax-Free Childcare over a smaller UC award, and take no deductions from awards only a reform creates policyengine-uk#2754). A family that would claim UC keeps Tax-Free Childcare instead when the top-up is worth more than the award. No debt deductions come out of an award only a reform creates. Every new variable is a formula.
  • No economic assumption moves. The vendored uprating pins and the bonus-sacrifice anchor change only their version stamps, and no gate digest moves.

Regeneration on feed 53815b2

The national and local references, membership and compile-parity receipts are regenerated on the merged tree, on main's feed pin (Chronicle 53815b2, #1420's re-pin). The vendored resources check against the full artifact unchanged. The local census, the local references, the local parity receipt and the coverage manifest regenerate unchanged.

  • National surface: 1,409 active references, up from 1,403, and no row left without a fact (four before). The new rows are the three OBR council tax nation rows (D14e) and SLC's three nation rows (R4). The DfE rows, the R7 windows and obr.council_tax move as described below. SLC's three nation rows differ from the incumbent's devolved metrics only by their uprating to 2025.
  • SPI uprating holds catch up with Budget 2026 ISA data, PR B of #1418: ISA subscriptions per adult, the Lifetime ISA anchor, additive ISA income, policyengine-uk 2.164.2 #1421: main's surfaces were generated before Budget 2026 ISA data, PR B of #1418: ISA subscriptions per adult, the Lifetime ISA anchor, additive ISA income, policyengine-uk 2.164.2 #1421 refreshed the HMRC uprating pins with the October EFO, so the regeneration moves 21 SPI holds. The eight self-employment amount band holds go from 1.1176 to 1.1115, as per-capita mixed income from 2023 to 2025 falls from 1.0384 to 1.0327. The 13 dividend holds by income band go from 1.088147 to 1.088151 (OBR per-capita GDP).
  • Local surface: 27,168 active references, every status count unchanged. The signed deferrals fall from 94 to 77, as the support-floor deferrals now follow each target's nations.
  • A window fix the regeneration found: R7's calendar-2025 window on dfi_ni.bus.passenger_receipts failed to compile, because its selector pins two services and the window counted series only from list-valued source keys. A dimension pinned to several values now names one series per value. The window of the sum is the sum of the windowed series, and a service short of a year, or with no facts at all, refuses the window. No other UK, US or NZ window reference pins several dimension values.
  • After the independent reviews: 43c3da74e regenerates the surfaces once more. The national references, membership and parity receipt move only for the three OBR council tax nation rows, now scaled by obr.council_tax's main-residence share, and for the restated DfE and council tax rationales. The local census gives D18a's circularity reason for ONS's small-area income rows, and ce9344c68 gives the local parity receipt's 903 rows of them the same reason.

Targets

  • OBR council tax by nation (D14e): the three nation rows are pinned to the UK and held as measured diagnostics. Each takes obr.council_tax's main-residence share (D14f), so the three sum to the bound row and read each nation on its basis.
  • Declared geography wins (R8, R9): a selector's declared geography_id is returned before the inferred pin, and the authoring refuses a disagreeing pin.
  • SLC devolved repayments (R4): slc.repayments.devolved_total becomes one row per nation. Wales binds. Scotland and Northern Ireland are held out by María's ruling, as their plan thresholds are not modelled. The three rows carry SLC's 2024-25 outturn to 2025 by policyengine-uk's OBR average earnings index, since repayments are a share of earnings above the plan thresholds (María's ruling of 30 October). This replaces the matched-period hold of 29 October, which declared the frame's own year: the solve honours a declared period only at the calibration year, or at the frame's year for a stored column, so it would have refused the Welsh row.
  • Calendar-2025 window (R7): Scottish Child Payment, the TFC top-up, Northern Ireland bus receipts and the four SLC England repayment rows.
  • Ledger observation_period: a reference can select its facts at a declared period while the target keeps its own. A restamp to another target period moves the observation period by the same number of years (restamp_reference_period), reading it as its match policy does: under the selector's period type for an exact match, and untyped for the default cutoff. A spelling read two ways refuses rather than guess. A row whose observation period cannot move stops the compile and is named. The generators move the contract's observation periods from its base (UK_OBSERVATION_PERIOD_BASE, 2025), which is kept separate from their default period. So the DfE rows read the January census after their model year at any period, including in the childcare fit tool and the UC matrix diagnostic.
  • DfE (D5): the eight rows bind the January 2026 census at model year 2025. The three in_year_snapshot holds are retired.
  • Window-ratio complement: a numerator can be its denominator less the listed cells; its metadata records that numerator and the formula numerator / denominator.
  • OBR council tax (D14f): binds the main-residence share: England's chargeable dwellings less second homes and empty homes, over chargeable dwellings (MHCLG, October 2025, about 0.968). Scotland and Wales take England's share, as a stated approximation.
  • Signed exclusions: D3 keeps Scotland's three-plus-children rows out with a corrected reason. D18a signs the MSOA income deferral. D18c keeps Northern Ireland's UC area cells out.

Spine and stage records

Gates

  • Zero-weight strata (Adjudicate the UK spine grain at the ingest/calibration seam #688): both SPI-synthetic strata are capped at 0. The June caps of 100,000 counted the retired ten-clone grain, and the national seam ships no zero-weight row.
  • Export surface (WS-E spine deferrals: known bugs and signed inconsistencies awaiting their increment #796): the all-zero incapacity_benefit_reported exclusion never matched the pinned reference, so it is dropped. The release boundary still drops the column.
  • macro_realism (Swap the microcosm-built UK spine into the national build and retire the incumbent feed #757): not declared, and the battery's policy text records why. The solve binds the income components directly, and the national accounts' household disposable income differs in scope. The release contract refuses a not_applicable entry in this spec vintage.
  • Take-up flags at their stages (María's ruling, "do both"): frs_take_up, frs_person_draws and frs_household_draws record their flags' shares at the stage's weights, and childcare_copy_take_up the five childcare flags it re-draws for the copies. Four stage gates hold them to their contract targets, at the terminal gate's bounds until a spine receipt measures them. Three run at the assembled phase. The copies' gate runs at the transferred phase, since its stage runs after the assembled boundary (frs_brma), and a test fails on any assembled-phase stage gate that reads a later stage. A flag whose population rests on one or two source households is flagged unchecked at its stage, never failed (María's ruling of 26 October); the terminal uk_take_up_signal gate, which stays release-blocking, still measures it.
  • The battery, spine-part, release-cut and dense digests in microcosm-data's contract move with the manifest.

Disclosure floor of 3 (A1)

María ruled on 29 October to move every UK disclosure floor from ten source households to three, CD171's absolute floor (recorded on #1095). One constant, receipt_counts.UK_DISCLOSURE_FLOOR, now sets it for the stage receipts, the stage-health gates and the report tools. A withheld figure reads "<3", and counts from three to nine read "3-9", rounded record counts included (Vahid's review). The SPI tape's two SN 9422 conditions are unchanged. The high-value-homes support gate and the three tools' SDC defaults move to three as well. Two guard tests fail on a floor of ten left in the UK runtime, the UK spec, the hand-written UK JSON, the UK tools or microcosm-data. One checks the "<10" label. The other checks comparisons, floor-named constants, defaults and keys in any case, and prose, including prose split across lines. Each ten it keeps is named by its phrase. The statistical floors stay, each saying so:

  • the zero-support floor;
  • the pay concept ratio's ten households: a ratio on fewer is held to its bounds as noise, its receipt says so, and its gate reads it as unmeasured;
  • the SPI transport's ten rows for a uniformity distance, whose positive-row count is now rounded like every receipt count;
  • the regional renter minimum;
  • the private-rent tool's ten-row group rule;
  • the SPI coherence tool's floor for medians and quantiles.

Band and rounding edges, the SPI tape's licence terms and history stay too. So do facts that still hold at three: the "fewer than 10" notes on the council tax and SPI rows, the two signed £2m exclusions, and the rules in local_small_cell_rule.json.

Text

Stale tracker text in notes, docstrings and the vendored bus resources' purposes. The Incapacity Benefit leaf note now names the FRS 2024-25. The charter-H2 fixture and the release input coverage manifest follow the stage contract digests.

Not in this PR

Tests

  • Engine tier on policyengine-uk 2.165.1: every UK engine test passes, run in full locally because CI stops at the first failure. Three needed updates: Tax-Free Childcare's top-up now lives in a shared helper, which still reads the spend the build routes; the company car's list price and capital contributions are person-level stocks that are no one's capital; and the graph build's stage records list each stage's rewrites beside their outputs (Carry manifest rewrites into the shared Stage/StageRecord producer graph #721). CI passed in full at 821e342d1.

  • Full engine-free suite at b46be0044 (673 files, in 30-file chunks): 22,969 tests, 233 skipped, counted by --junitxml. Sixteen fail:

    • ten fixed since: the SLC nation rows' declared period (four test_uk_measurement_period tests) and the pins and fixture rows that follow the regenerated surface (six). At 652835c1f, the 60 files that read the regenerated surfaces pass: 3,920 tests, 62 skipped;
    • six only on this machine: four Orrery tests (a missing Node package), test_engine_is_reported_unavailable_without_the_uk_extra (the uk extra is installed here) and test_spawned_workers_score_identically_to_sequential_cli (needs Python 3.14.7; this machine has 3.14.6).
  • The "3-9" band (254be6d41): the tests of every rounded_count caller pass: 39 engine-free files (1,384 tests) and 8 engine-tier files (72 tests).

  • After the Promotion for the constant-id national line: pointer and certified-loader extension #823 revert (1f46653b6): the 18 files that resolve or load a UK dataset pass, bar the live Hub pointer test, which needs private credentials (CI skips it). Main's version of that test expects the old release naming; UK: make the promoted national line the default load (#823) #1432 fixes it.

  • Feed-reading tests on 53815b2 (the 15 feed-gated files, one process each, which CI skips without the feed): 235 tests, two skipped because the Scotland support and the authenticated national artifact are not on this machine. Two tests in test_uk_target_references.py still expected values from before D5 and R7; 02df9df2b moves them to DfE's January 2026 census and the calendar-2025 windows, each value checked against the feed's facts and the committed membership. The two drift checks that regenerate the vendored resources and the target surfaces from the feed were left out: the regeneration wrote those files, and the vendoring check passed in it.

  • After Vahid's second review: f83d7a3c2 links the ruling's record from receipt_counts and the changelog, and its tests pass. ddd82aa2f adds a test that a pinned service with no facts refuses the window; the 460 tests of test_ledger_targets.py pass.

  • After each of the five independent reviews (c0dd6eae2, 5b693b736, 821e342d1, 0da33d0cd and 4af602d81), the following pass locally:

    • the touched test files;
    • the H2 fixture check and the H2 parity test;
    • the coverage manifest check and the gate digests.

    The fourth and fifth rounds' tests also replay the tool's reasons over both committed parity receipts, starting each row from the classifier's reason, so CI now catches a stale, changed or removed reason. CI passed every job at 0da33d0cd.

  • Checks: tools/ci_test_plan.py verify, tools/spec_engine_coverage.py --check, the vendoring check against the full artifact, and ruff on the edited files.

🤖 Generated with Claude Code

juaristi22 and others added 21 commits October 29, 2026 12:59
OBR's table 4.1 is UK-wide, so Chronicle stamps its England, Scotland and
Wales council tax rows K02000001, and the substring rule's nation pins
compiled no fact. The rows now pin to the UK, as obr.council_tax and
obr.domestic_rates do; each binding still scopes the model column to its
nation. obr.council_tax binds their sum, so measure exclusions hold the three
as diagnostics, measured on every evaluation as evidence for CT-C (María's
ruling of 2026-10-29, microcosm#1095 D14e). The references and membership
regenerate with the PR's shared regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 R8 and R9: a wrong geography pin reads as "no fact", and the
generator's inference (one-by-one pins, prefix pins, then the substring rule)
silently overrode any geography a target's selector declared, because the
authoring merged the pin over the selector. María ruled on 2026-10-29 that a
declared geography wins. The UK generator now returns a selector's declared
geography_id first and refuses a one-by-one pin that disagrees with it, and
the shared authoring refuses a pin that differs from a declared selector key
instead of overwriting it. No committed selector declares a geography yet, so
no reference changes; the devolved SLC rows are the first to use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 R4. slc.repayments.devolved_total took one fact per devolved
nation, but the "slc." rule pinned it to Great Britain, where no fact sits, so
it never compiled. It becomes slc.repayments.wales_total, .scotland_total and
.northern_ireland_total, each declaring its nation's geography in its selector
(the R8 rule), measuring the frame's own year (measurement_period 2024)
against SLC's 2024-25 outturn, and filtering the model to its nation. The
incumbent registry's three per-nation metrics now map one to one.

María ruled on 2026-10-29: bind Wales, and hold Scotland and Northern Ireland
until their borrowers' plan thresholds are modelled (plan_4_imputation is
False; policyengine-uk#2636 closed as not planned). Measure exclusions hold
the two. The matched-period hold for the three rows waits for her signature,
so the membership assertion in test_uk_target_references fails until the
shared regeneration, which needs that hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follows the split of slc.repayments.devolved_total (microcosm#1095 R4): the
contract has 453 targets, the registry scope 393, and the incumbent's three
devolved metrics map one to one, so 185 mapped targets. geography_id joins
the national selector vocabulary, as the R8 rule lets a selector declare its
fact's geography.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 R7: Scottish Child Payment spending, HMRC's Tax-Free Childcare
top-up, Northern Ireland's Translink bus passenger receipts and SLC's four
England repayment rows took their 2025-26 fact alone, outside María's rule of
2026-09-22 that fiscal-year facts take only the months to the end of the
calibration calendar year. She extended the rule to them on 2026-10-29, so
each declares calendar_year_window: three twelfths of 2024-25 and nine of
2025-26. The pinned feed has both years for all seven (Scottish Child Payment
has 2026-27 too). No hold or measurement period names them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 D5: a target is measured at the model's period, but some
publications observe a period with another label. DfE's January census for
model year Y is January Y + 1, the spring term of the year's entitlements, so
the rows could only bind the census a year out: every selection path refused
a fact after the target period.

LedgerTargetReference gains observation_period. Fact eligibility, the
latest-not-after choice, the exact-period match and the resolved-fact guard
read it in place of the target period; the TargetSpec keeps the target period,
and its metadata records ledger_observation_period. Operations that compose or
pin periods themselves (the calendar-year windows and average, latest_plateau,
the two ratio translations) and the monthly windows refuse it, as does an
unparseable period. The authoring passes a target's observation_period into
its reference rows and checks eligibility at it. Nothing declares one yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 D5. The eight dfe.funded_childcare rows bound the January 2025
census, but model year 2025's entitlement year runs from April 2025 to March
2026: its spring term starts on 1 January 2026, and the spring-term measures
count children by their age on 31 December 2025, the basis DfE's January 2026
census uses. The January 2025 census was a year out, held as an in-year
snapshot. María ruled on 2026-10-29 to bind January 2026.

Each row declares observation_period 2026-01, so its fact is selected there
(the universal-only difference's two operands too) while the target stays at
2025. A fact after the target period carries no uprating hold, so the three
in_year_snapshot holds retire. The reference and membership assertions wait
for the shared regeneration; the childcare take-up refit follows on the arm's
spine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 D14f needs OBR's council tax receipts restated for the main
residences households hold: England's chargeable dwellings less its second
homes and empty dwellings, over the chargeable dwellings. No published cell
is that remainder, and a window-ratio operand can only sum cells.

A calendar_year_window_scaled_by_ratio numerator may now declare complement:
true. Its cells are taken off the denominator before the quotient, which
keeps the (0, 1] subset-share check. complement is operand metadata, never a
selector field; only a numerator may declare it, and only as true. The spec
records ledger_value_numerator_rule, and the formula reads
"(denominator - numerator) / denominator".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 D14f. obr.council_tax bound OBR's Great Britain receipts,
which include council tax on second homes and empty dwellings. Households do
pay it on property they own, but the frame's council_tax is each household's
main-residence bill only and the engine has no input for council tax on
other property, so the solve met the whole line by pushing main-residence
bills up. María ruled on 2026-10-29 to bind the main-residence share.

The row now scales its calendar-2025 window by England's chargeable dwellings
less its second homes and empty dwellings, over its chargeable dwellings
(MHCLG Council Taxbase, October 2025), through the window ratio's new
complement numerator. A private compile of the row against the pinned feed
resolves the window and the three taxbase cells once each. The count-based
share and England standing for Scotland and Wales are stated in the notes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095, María's rulings of 2026-10-29:
- D3: ons.population.scotland_households_3plus_children stays signed out,
  but its reason was wrong: a fact exists, Chronicle's Scotland cut, and it
  counts couple households only, so binding it would fix part of the group
  and leave lone parents free. The register entry takes the corrected reason
  (couples_only_composition_fact), her approval of 2026-10-29 and an
  adjudication line; it tracks #1095, as #736 closed.
- D18a: ONS's small-area income rows (BHC and AHC means, housing costs) were
  deferred "pending the signed mean-aggregation design". ONS models them
  from the FRS, so binding them would partly fit the FRS to itself: they are
  kept out as circular (frs_model_based_circularity).
- D18c: Northern Ireland's UC area cells stay out; the reason now records
  that DfC publishes claimants, counting a joint claim twice (uk-data#590).

The membership and local surfaces carry the new texts at the shared
regeneration; test_uk_target_references pins the D3 text to the new reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095's "stale register and gate text" items that need no signature:
- weight_year's docstring quoted 0.72% growth from 2024 to 2025, the figure
  before policyengine-uk 2.155.0's population; it now points at the pinned
  dataset_projection block, which every lock re-pins.
- The certifier and the calibration run said local-candidate gates stay out
  of national certification "until microcosm#146"; #146 closed when the local
  variant shipped, so they now say the local line certifies them.
- was_wealth's bedrooms note pointed at the closed #145; the question is
  #1095's "num_bedrooms as a WAS predictor" (uk-data#463).
- dwp.uc.households_lcwra_element's note quoted a September baseline and an
  open #882 question; it now records pe-uk#1989's proxy and María's NS6
  ruling of 2026-10-29 to keep it.
- obr.universal_credit's note said both "UK incl. Northern Ireland" and
  "Scoped to Great Britain" (CF1); it now says Great Britain.
- obr.sdlt's note called pe-uk#2322 unreleased; it has been in the lock since
  #1400, and the note now states how its projection compares with the row's.

Signed register text (the JSA and ORR reasons, tracking fields of closed
issues) waits for the renewals PR, so each entry is re-signed once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#721 (tracked in #1095): a stage can declare rewrites, columns an
earlier stage produced that it replaces, but StageRecord kept only the
declared outputs, so provenance credited a rewritten column to its first
producer, and the UK spine's final nonzero-share list kept a hard-coded
FRS_EDUCATION_GRANT_REWRITES carve-out (redundant, as frs_spine already
produces education_grants).

StageRecord gains rewrote, the stage's declared rewrites, and its
nonzero_share covers them as they stand after the stage. A rewrite absent
after the stage is refused, like a missing output. The spine manifest's
stage_records carry rewrote, and the final list reads every record's
outputs and rewrites, so the carve-out and its import go. #721 can close
once this lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 (from #796): the support-floor loop deferred E06000053 and
E09000001 for every local-authority target outside the bound families,
including the nine Welsh and eight Scottish families whose area_scope holds
no English authority, so their membership listed deferrals for cells they
never have. The loop now reads each target's local_authority gss_prefixes,
keeps only the areas in scope, and skips a target with none. Thirty-two
targets take the deferral (59 area entries, E06000053 in 27 and E09000001 in
32), down from 49 (93; 44 and 49). The local references and membership
regenerate with the shared regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 items from #688, #796 and #757.

- uk_zero_weight_strata: the two strata kept June's caps of 100,000 each,
  10,000 SPI-synthetic households times ten output-area clones, a grain the
  national seam no longer builds. Its calibrated control of 2026-10-28
  (4374e0bf) holds 78,093 households and no zero-weight row in any of the
  four SPI-synthetic and capital-gains-clone strata, because the SPI stage
  gives the stacked donors their prior mass before calibration. Both strata
  are re-declared at a cap of 0 under current names; they keep their
  selectors so that a zero-weight row there is named by stratum.
- uk_export_surface: the reviewed exclusion for
  person.incapacity_benefit_reported never matched. The pinned reference
  (efrs_parity_reference.json) lists the all-zero column among its zero-share
  inputs, not on its export surface, so the gate reported the exclusion as
  unused. The release boundary still drops the column, and the shared check
  refuses it as an unlisted extra, so the guard holds without it.
- macro_realism: María ruled on 2026-10-29 to drop it and record why. The
  plan declared it not_applicable, but the release contract's verifier
  refuses an excused entry in this spec vintage (test_gate_battery_contract_
  pins holds every UK entry to not_applicable None). So the battery's policy
  text records the reason: the solve binds the household income components
  directly, and the national accounts' household disposable income differs
  in scope (imputed rent and, in the headline series, non-profit
  institutions serving households).

The battery, part and dense digests in microcosm-data's contract move with
the manifest, as in a094f5d. The take-up stage-time checks (her ruling of
2026-10-29) follow in their own commit. The Carer's Allowance stage-health
gate waits on her: the take-up contract records her 2026-10-20 ruling of no
such gate until a caseload target binds the rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d2a1f9c gave StageRecord its rewrote field and read the spine's final
nonzero-share list from every record's outputs and rewrites, but only the
legacy plan path filled it. The graph build, the production path, projects
its records from node artifacts and left rewrote empty, so its records
dropped the rewrite provenance microcosm#721 asks for, and
test_driver_writes_spine_h5_sidecars_and_logbook lost education_grants from
the final list (the real roster still lists it through frs_spine's outputs).

A declared rewrite is an owned cell of its node, as an output is, so the
graph projection now reads its share from the node's artifact and records
the stage's declared rewrites.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-29 on microcosm#1095 ("do both"): keep the
terminal uk_take_up_signal gate release-blocking on the calibrated shares,
and check the same flags at stage time.

- The twelve flags the terminal gate measures are drawn by three stages
  before the assembled boundary: frs_take_up (the four childcare flags, the
  UC childcare element and the under-2 usage), frs_person_draws (Marriage
  Allowance, Scottish Child Payment) and frs_household_draws (TV ownership,
  licence evasion, the first-home and purchase flags).
  take_up_signal_outputs selects each stage's flags, and a test holds the
  three sets to a partition of the terminal gate's.
- The terminal gate's per-flag measurement moves into
  _measure_take_up_signal, unchanged, so that uk_take_up_signal_receipt
  measures the same thing at the stage's weights. The receipt is
  aggregate-only: support is a band of distinct source households, and a
  share or target resting on fewer than ten is withheld.
- Each stage records the receipt in its checkpoint evidence, and the graph
  hashes frs_take_up and receipt_counts with the two draw stages.
- stage_health gains a take_up_signal check: the receipt must carry
  exactly the declared outputs; each must vary, sit within the declared
  bound of its target, and keep the universal hours nested; a withheld
  share fails, as it cannot be measured.
- gates.json declares uk_stage_frs_take_up_signal,
  uk_stage_frs_person_draws_take_up_signal and
  uk_stage_frs_household_draws_take_up_signal (assembled, release-blocking,
  evidence_absent_blocks, population_fact_check) at the terminal gate's
  bounds, 0.05 and 0.10 for the under-2 usage, until a spine build's
  receipt measures the stage-time deviations. The spine scope, the data
  contract's entry, evidence and spine-part mirrors and the battery, spine
  and release-cut digests follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#823 (tracked in #1095). The registry kept the UK 2024-25 national
line off the default variant until its first certified cut existed on the
Hub, so that resolve("uk") never chased an absent pointer; its comment named
the flip a one-line follow-up after promotion (review of #966, Max's #823
ruling). Checked on 2026-10-29: latest-national.json on
policyengine/populace-uk-private names
microcosm-uk-2024-25-national-20261025T091414Z-a6299504 (updated
2026-10-26), with its release certification, built on policyengine-uk
2.151.0, the version #1095 names for the flip.

The entry's variant becomes DEFAULT_VARIANT, so resolve("uk") and
load("uk", 2025) follow latest-national.json, and load("uk", 2023) stays on
latest.json. This changes the default UK load for every consumer of
microcosm.data; no other local repository names the retired national
variant. The registry, loader and assembler tests and the release runbook
follow; the live pointer test now branches on the line pointer, not the
variant, and resolves the promoted cut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two items from microcosm#1095's stale-text list, both without a signature:

- The three vendored bus resources said that the LCFS and ETB bus-fare and
  bus-support rakes consumed them. microcosm#930 and María's ruling of
  2026-09-23 replaced those rakes with the bus fare and bus support pricing.
  The purposes in the selection register and in the vendored files now name
  the pricing, and dft_bus_value_anchors.json lists bus_support_pricing,
  which reads its net support, among its consumers.
- The Incapacity Benefit SPI leaf (FRS benefit code 17) was "a structural
  zero in the audited 2023-24 FRS". A private check of the FRS 2024-25
  benefits table found no code-17 row, so the note now names both vintages.
  The text is in sources.yaml, its source_stages.json projection and the
  contract's expected string.

Both move stage contract digests, of frs_hmrc_spine_leaves, lcfs_consumption
and etb_services. So tools/graph_uk_spine_fixture.py regenerated the
charter-H2 fixture (only those three digests and the leaf note change), and
the release input coverage manifest takes the new source manifest digest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
8e3394b gave LedgerTargetReference an observation_period field. The
monetary binding serialises the reference with asdict, so every activated
and source reference receipt now carries the key (None unless declared), and
microcosm-calibrate's integrity check refused it as an unknown key: the full
engine-free run failed test_monetary_targets' two binding tests. The
verifier's reference keys gain observation_period, as #834 added the operand
keys, and the calibrate fixture's hand-built receipt carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fff37ea held the three OBR council tax nation rows (ruling D14e) and
ae1edf7 held SLC's Scottish and Northern Irish repayment rows (ruling R4),
which takes the packaged calibration measure exclusions from 40 to 45 and
the census's slc. marker from 3 to 5. test_uk_measure_simulation still
said 40 and 3; the full engine-free run caught it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
microcosm#1095 R4. ae1edf7 split SLC's devolved repayments into one row
per nation; each resolves SLC's 2024-25 outturn, the latest each nation
publishes, at the 2025 target period and measures the frame's own year
(measurement_period 2024). The compile needs a signed matched_period hold
for that, and test_every_committed_hold_is_signed refuses a proposed one,
so the hold waited for María's signature. She signed it on 2026-10-29
(approved_by juaristi22, expires 2027-01-29). The shared regeneration can
now compile the three rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-29 on microcosm#1095 (A1): "move everything to 3".
CD171's absolute floor is three; the build used ten, CD171's working
minimum against secondary disclosure across outputs. Her ~/.claude/CLAUDE.md
now sets three for what people read too.

- receipt_counts gains UK_DISCLOSURE_FLOOR (3), UK_WITHHELD ("<3") and
  UK_BELOW_TEN_BAND ("3-9"). small_cell_share withholds one or two records;
  support_band and extreme_count print "<3" and "3-9" below ten;
  rounded_count prints three to nine exactly.
- Every stage receipt that withheld on fewer than ten sources now withholds
  on fewer than three, through the shared names: annual income, the
  apprentice draw, Carer's Allowance, the Move to UC and UC take-up draws,
  pension coherence, property purchases, the SPI band mean, the NTS fare
  mix, the gambling fits, the high-value home gates and the single years of
  age (diagnostics.UK_DIAGNOSTIC_MIN_SOURCE_HOUSEHOLDS). stage_health reads
  the new label and bands, and its detail key becomes
  regions_rewritten_under_floor.
- Tools: the private-rent receipt (its output key becomes
  cells_under_floor_source_households), the childcare realised take-up
  report and, through the diagnostics constant, the council tax liability
  comparison.
- Kept on purpose: the zero-support floor (1); local_small_cell_rule.json,
  whose three signed rules defer published cells too small to fit, not
  survey cells, so they need no re-signing; the private-rent tool's
  ten-row group rule (no summary describes one district); and the SPI
  coherence tool's cell floor for correlations, medians and quantiles,
  since a median of a few records is one record's own value.
- Six gate notes and four stage notes state the new floor; the battery and
  spine-part digests, the coverage manifest's source digest and the
  charter-H2 fixture's data bundle follow (its graph is unchanged).
- About 86 test expectations across 21 files: a withholding test keeps its
  purpose on one or two sources and expects "<3"; an incidental "<10" now
  reads the exact figure, "<3" or "3-9".

1,270 tests across the touched files, the gate and contract pins and the
coverage manifest pass; the charter-H2 parity test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 this is an early look at 5a944e449, not a full review, since the body says it isn't ready. It covers must and should items only, and nothing on your own pending list. I'll do a full pass after main merges in and the references regenerate.

  1. must: an engine-uk failure the body doesn't list. tests/engine/uk/test_uk_graph.py:84 still asserts set(nonzero_share) == set(stage.outputs). But _graph_stage_records now adds stage.rewrites to shares (Carry manifest rewrites into the shared Stage/StageRecord producer graph #721). So age_tail fails in engine-uk on both Pythons (run 40917095981, at this head). Regenerating won't fix it. The assertion needs set(stage.outputs) | set(stage.rewrites). The body's engine-tier line covers only the charter-H2 parity test, so this one was missed.
  2. should: rounded_count now prints exact counts from three to nine (receipt_counts.py:103). The module docstring still says "no exact count that ... narrows a small group by difference". extreme_count and support_band print "3-9" there instead. Printing exact counts just above the floor is where differencing bites. That matters more now that the floor is three rather than ten, CD171's working minimum against secondary disclosure across outputs, as the docstring says. I'd keep "3-9" in rounded_count too. I'm not reopening the ruling. Please link the UK: every open data item: uk-data parity, engine locks, the local surface, consumption and high-value homes #1095 comment that records it from the module and the changelog, and say there that the SN 9422 conditions (HMRC sees SPI-based material ten working days ahead) are unchanged.
  3. should: UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426 adds new "<10" sites. The SPI pay-floor receipt in UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426 hardcodes ten in several places. Whichever of the two lands second should move them onto UK_DISCLOSURE_FLOOR. Otherwise one receipt stays on the old floor without anyone noticing. I've raised it on UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426 too.
  4. should: ship the Promotion for the constant-id national line: pointer and certified-loader extension #823 default-load flip on its own. Pointing load("uk") at latest-national.json and retiring variant="national" affects every consumer. It doesn't depend on Budget 2026 slower data fixes, PR C of #1418: electricity at the ONS level, pension income by area #1420, Budget 2026 ISA data, PR B of #1418: ISA subscriptions per adult, the Lifetime ISA anchor, additive ISA income, policyengine-uk 2.164.2 #1421 or the regeneration. As its own small PR it could land now, and it wouldn't be held behind this one. The old 823-national-variant-until-promoted.changed.md entry, which describes the opposite state, should go in the same change.

The D14f main-residence share (0.968) is covered on #1423: #1423 (comment).

What I ran. I read the diff at 5a944e449 against its base fddbc7073 and the failing CI logs. I didn't run anything locally. I didn't review the target, gate or ledger changes in detail, and I'll leave those for the full pass.

🤖 Generated with Claude Code

juaristi22 and others added 3 commits October 30, 2026 15:44
Main now carries #1420 (electricity at the ONS level, pension income by
area, Rural Fuel Duty Relief areas, the balanced Carer's Allowance draw)
and #1421 (policyengine-uk 2.164.2, ISA subscriptions, the Lifetime ISA
anchor, the additive ISA contract), with the feed at Chronicle 53815b2.

Hand-merged:
- uprating_holds.json: main's holds, re-pointed to microcosm#1095, less
  the three DfE in_year_snapshot holds this branch retires (D5), plus the
  signed SLC nation hold (R4) and main's ONS electricity hold.
- gates.json: the export surface keeps main's tax_free_savings_income
  exclusion (microcosm#1360) and drops the all-zero incapacity benefit
  layer as this branch does; its notes name both.
- sources.yaml: main's Carer's Allowance stage note, with its receipt
  floor restated at three source households.
- weight_year.py keeps the docstring that names no lock's growth figure.
- calibration_run.py, test_country_spec.py, contract.py and
  test_contract.py list both sides' new gates in gates.json's order.
- The tests take the merged counts: 458 targets (registry 396, profile
  62; mapped 185, unmapped 211), 44 measure exclusions, and 34 support
  floor rows over 61 area entries ({E06000053: 27, E09000001: 34}).
- The Carer's Allowance receipt test takes main's balanced placement at
  the floor of three; two stale comments restate that floor.

Regenerated on the merged tree with policyengine-uk 2.164.2:
source_stages.json, the release input coverage manifest (143 required,
2 reviewed exclusions), the gate battery, spine-part and release-cut
digests, and the charter-H2 fixture. The national and local references,
membership and compile-parity receipts regenerate next, on feed 53815b2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK lock moves from 2.164.2, which #1421 brought, to 2.165.1, the
newest release (microcosm#1095, Phase C's first re-bump):

- 2.164.3 (policyengine-uk#2736) counts other investment income in
  market income and in HBAI household net income, before and after the
  reset to zero. The LCFS, NEED and NTS stages band recipients on that
  HBAI income with its deductions added back, so their gross income now
  counts it, as the surveys' gross incomes do.
- 2.164.4 (policyengine-uk#2746) counts it in capital_income too. Its
  changelog says nothing reads capital_income; the WAS stage does, as a
  predictor, and the recipients' figure now matches the donor's
  DVGIINVR8, WAS's gross income from all financial investments.
- 2.165.0 (policyengine-uk#2750) adds the company car and car fuel
  benefits in kind and the Class 1A NICs on them, read from nine person
  inputs. The build writes none, so no one has a company car and both
  benefits are zero, as before. The inputs are signed at their default
  in UK_ENGINE_DEFAULT_INPUTS for María's signature; the company car PR
  for the Autumn Budget dashboard's #316 supplies six of them.
- 2.165.1 (policyengine-uk#2754) lets a family that would claim UC keep
  Tax-Free Childcare instead when the top-up is worth more than the
  award (forgoes_universal_credit_for_tax_free_childcare), and takes no
  debt deductions from an award only a reform creates. Every new
  variable is a formula.

Two lockstep tests follow the engine: Tax-Free Childcare's top-up is
one helper from 2.165.1, which reads the spend the build routes; and the
person-level GBP stocks gain the company car's list price and the
capital contributions towards it, which are no one's capital.

No economic assumption moves: the vendored uprating pins and the
bonus-sacrifice anchor change only their version stamps, and no gate
digest moves. The three pyproject floors, uv.lock (policyengine-uk
alone), the approved lock digest, the engine-core pin test, the
concept coverage report, the version mentions in code, notes and
tests, source_stages.json, the coverage manifest and the charter-H2
fixture follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
610a1bc records a stage's declared rewrites beside its outputs in the
graph build's stage records, as d2a1f9c does in the plan build
(microcosm#721). The engine-lane test of the record projection still
expected the outputs alone, so it failed on age_tail, which rewrites
age. It now expects both, and the record's rewrote field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's 821e342 answers its third review: restamp failures raise and
name the row, the reference period is one constant
(UK_TARGET_REFERENCE_PERIOD), the spine gates read the assembled
boundary from graph_evidence, and the stage-health count bounds and the
floor-of-ten guard are tighter. It merges without conflicts, and
company_cars stays in the transferred phase, after frs_brma.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion base

- A restamp, and the authoring tools, read an observation period under the
  selector's period_type when it declares one, as the exact-match path does:
  under fiscal_year "2011-12" moves to "2012-13", untyped or under month to
  "2012-12", and a spelling the type cannot read refuses. A target period
  must give a four-digit year, so "FY25" refuses (1).
- The floor guard reads "10." at a sentence's end and 10.0 as ten, still
  skipping 10.5, 10% and 10_000; prose matches across a line break, a
  comment marker or a string's quotes; the constant pattern is
  case-sensitive and skips MINUTE, so minutes=10 and {count:>9} no longer
  match. The rent receipt tool's docstring still said fewer than ten: it
  gives the floor of three it uses. The pay ratio's wrapped "one to nine
  households" (statistical) and the SPI tape's "at least ten working days"
  (licence) are kept by phrase (2).
- UK_OBSERVATION_PERIOD_BASE holds the contract's base apart from
  UK_TARGET_REFERENCE_PERIOD, the generators' default period. Tests check
  that the contract's observation periods are the January after the base,
  and that each national and local reference reads the January after its
  own period; the changelog says the period is read under the selector's
  type (3).
- Round-3 gaps: a test replays the tool's reasons over both committed
  parity receipts; the local compile's restamp error names a local
  reference, with a test; a test pins every printed count's bounds up to
  20,000, the 1,000 edge included; UK_SPINE_ASSEMBLED_FINAL_STAGE moves to
  uk_runtime/graph.py, which builds the checkpoint from it, and the
  stage-gate test imports it from there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Thanks for the fourth review, of 821e342d1. Everything below is at 0da33d0cd. Noted on the engine jobs: 8f7d69204 and bd9839bf2 fixed both failures.

New in the round-3 commits

  • 1. Periods under a typed selector: the restamp and the authoring tools now read an observation period under the selector's period_type when it declares one, as the exact-match path does.

    • Under fiscal_year, "2011-12" moves to "2012-13". Under month, or with no type, it moves to "2012-12".
    • "2026-01" spells no fiscal year, so a fiscal-year selector refuses to move it.
    • A target period has to give a four-digit year, so "FY25" refuses rather than moving "2026-01" to "26-01".
    • Tests cover each case.
  • 2. The floor guard:

    • A ten is now 10, 10. at the end of a sentence, or 10.0. The guard still skips 10.5, 10% and 10_000.
    • Prose matches across a line break, a comment marker or a string's quotes. That found three wrapped tens:
      • the rent receipt's "fewer than ten". It was stale: the tool already withholds below the floor of three (MIN_SOURCE_HOUSEHOLDS), and its docstring now says so;
      • the pay ratio's "one to nine households", which is a statistical floor;
      • the SPI tape's "at least ten working days" for HMRC's review, which is a licence term.
    • The last two are kept by their exact phrases.
    • {count:>9} and minutes=10 no longer match: the pattern for constants is case-sensitive now and skips MINUTE.
  • 3. The base: UK_OBSERVATION_PERIOD_BASE (2025) now holds the contract's base, separate from UK_TARGET_REFERENCE_PERIOD, the generators' default --period. Two tests replace the old one:

    • the contract's observation periods are each the January after the base;
    • for the national and the local references, each reference whose contract target declares an observation period reads the January after its own period, and no other reference declares one.

    So authoring at 2026 gives "2027-01", which passes once the default moves with it, and moving the default leaves the base alone. The changelog now says the period is read under the selector's type.

Round-3 gaps

  • Local parity receipt: a new test replays the tool's reason notes over both committed parity receipts. It fails on any reason the tool would now write differently, and changing one of the tool's texts marks its rows stale. The national replay needs the feed, so CI didn't check either receipt's reasons before.
  • Contract errors: the local compile's message now says "UK local reference". A test covers that branch with a real fiscal-year reference that can't move.
  • Count bounds: a test checks every count up to 20,000. Each printed count's bounds equal the lowest and highest counts that print as it, and putting the 1,000 edge back to v - 50 fails it.
  • Graph boundary: UK_SPINE_ASSEMBLED_FINAL_STAGE now lives in uk_runtime/graph.py, which builds the checkpoint node from it. graph_evidence.py, spine_build.py and the stage-gate test import it from there. Each touched module imports on its own in a fresh interpreter.
  • Pay ratio: a receipt from before 821e342d1 lacks the flag, so the gate doesn't report its ratio as held. A thin ratio still reads as unmeasured. No change.

The touched test files pass locally, and so do the H2 fixture check, the H2 parity test, the coverage manifest check and the gate digests. CI runs the full tiers.

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Fifth review of #1422 at 0da33d0cd

This re-reviews 0da33d0cd against my fourth review at 821e342d1. As before:

  • one reviewer re-ran each earlier finding;
  • a second read the commit for new defects.

Paths are relative to packages/microcosm-build/src/microcosm/build/ unless they start with packages/ or tools/.

Verdict: the round-4 items are fixed, CI is green, and nothing blocks a build. Two of the fixes regress latent cases:

  1. under the default match policy, typed selectors now refuse to restamp. My round-4 suggestion pointed there, again;
  2. the floor guard's case-sensitive constant pattern drops lowercase floor names.

Round-4 items

# Status
1. Typed periods Fixed for exact. A fiscal "2011-12" now moves to "2012-13", and "FY25" refuses. But see 1 below.
2. Floor guard Fixed. Sentence-final tens, 10.0 and wrapped prose are caught, and the three wrapped tens are real and counted. But see 2 below.
3. Observation base Fixed. Every mutation I tried fails the new tests, including the old trap with both constants at 2026.
Parity-reason replay Partly. It runs feed-free in CI and catches changed tool texts and stale ruled reasons. Not caught: rows that keep the classifier's generic text (318 national, 288 local), and a removed national rule. So "fails on any reason the tool would now write differently" overstates it.
Local message Fixed, but its test pins 1's regression (below).
Count bounds Fixed. Every edge mutation fails the test.
Graph boundary Fixed. There is one definition, in uk_runtime/graph.py, with no import cycle, and #1430's head merges with it cleanly.

New in 0da33d0cd

1. Typed restamps now refuse what the default policy reads (ledger_targets.py:4691-4695; target_reference_authoring.py:151).

  • Mechanism:
    • The restamp reads the observation period under the selector's type for every policy. Only exact reads it that way.
    • The default latest_not_after, which every committed DfE row uses, reads it untyped as a cutoff (:4766, :4853-4854, :4890).
  • Example: an academic_year or fiscal_year selector with observation "2026-01" and the default policy selects AY2025-26 at 2025.
    • Round 4 moved it to "2027-01", which selects AY2026-27.
    • Now it raises "cannot move the period '2026-01' by 1 year(s)". That stops the whole compile (uk_runtime/ledger_targets.py:508-516, :1453-1461) and the generators.
    • A month selector with an annual cutoff fails the same way.
  • Cause and fix: my round-4 suggestion didn't distinguish match policies. The fix is to read the period the way the policy that consumes it does: typed for exact; for latest_not_after, typed where the type can read the spelling and untyped otherwise.
  • The new local test (test_uk_ledger_targets.py:2812-2846) builds exactly this reference and calls it unmovable, so it pins the regression.
  • Impact: latent, since no committed reference has both a typed selector and an observation period.
  • Residuals:
    • the int fast path ignores the type (:4727-4728);
    • the constructor still validates the observation period untyped (:356-360), so a valid fiscal "2099-00" is refused;
    • with an untyped exact selector the match falls back to the fact's own type (:5092), so an untyped "2011-12" still restamps to "2012-12". The changelog's "as the exact-match path does" holds only for typed selectors.

2. The floor guard's case-sensitive constant pattern drops lowercase floors (packages/microcosm-build/tests/engine_free/uk/test_uk_receipt_counts.py:225).

  • Now missed: rounds 3 and 4 caught lowercase names at the start of a line. This head misses:

    • floor = 10, support_floor=10, and small_cell_floor = 10;
    • sdc_floor = 10 and disclosure_floor: int = 10;
    • row_threshold = 10.

    No live instance is affected.

  • Fix: the two false positives the reply credits to case sensitivity go away because of the new lookbehind (:222) and the narrowed key pattern (:218). So IGNORECASE can stay, with the words matched as snake-case tokens, e.g. (?:^|_)(?:min|minimum|floor|sdc|small|threshold)(?:_|\b).

  • Docstring claims it doesn't meet:

    • #: comment continuations. One live, harmless wrap sits at uk_runtime/target_weights.py:101-102.
    • Float-, Optional- and Final-annotated defaults.
    • Single-quoted dict keys.
  • False positives: fill-character format specs such as {n:0>9} are flagged, though they fail loudly.

  • Brittleness: the kept phrases embed indentation and line breaks, so reflowing those comments fails the guard, again loudly.

3. Small: the observation-period tests require each period to be the January after the reference period, and call int() on that period. A fiscal or prefixed observation period, or a string target period like "2025-26", would fail them. That's fine for D5, but it deserves a comment in the test.

Checked and found true

  • Typed shifts: every spelling each type can read moves correctly, with its start and range end both shifted. Untyped results are identical to round 4.
  • Generators: both use the default --period of 2025 and the base of 2025. The DfE rows at 2024, 2025, 2026 and 2030 give "2025-01", "2026-01", "2027-01" and "2031-01".
  • Graph: imports are clean, and the graph's loud failures on a missing boundary are intact.
  • Guard: it scans the tree in 1.3 s with no catastrophic backtracking.
  • Cross-PR: neither SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428 nor UK: company cars from the FRS job table (dashboard #316) #1430 declares observation periods, and the guard flags none of their added lines.
  • CI on 0da33d0cd: every job passes.

Disclosure-control items went to the author separately.

🤖 Generated with Claude Code

… parity reasons

- Read an observation period as its match policy does: under the
  selector's period_type for an exact match, untyped for the
  latest_not_after cutoff. A spelling read two ways refuses to move:
  an untyped exact "2011-12", and a cutoff the selector's type reads
  otherwise. The constructor checks an exact period under its type,
  and a bare year is checked under the type before it moves.
- The floor guard matches the floor words as snake-case words of a
  name in any case, defaults under any annotation, single-quoted keys
  and #: continuations, skips format specs with a fill, and matches
  kept phrases across whitespace. target_weights.py's count of homes
  is kept.
- The parity receipt replay starts each row from the classifier's
  reason for its kind (PARITY_DIFFERENCE_REASONS, now one constant in
  gates.py), so a changed classifier text or a removed rule fails.
- Note the observation-period test's assumption about D5's spellings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Thanks for the fifth review, of 0da33d0cd. Everything below is at 4af602d81.

New in 0da33d0cd

  • 1. Restamps under the default policy: fixed. An observation period is now read as its match policy reads it: under the selector's period_type for exact, and untyped for the latest_not_after cutoff.
    • A fiscal- or academic-year selector's "2026-01" moves to "2027-01" again, and a month selector's "2026" to "2027".
    • The suggested rule for the cutoff, typed wherever the type can read the spelling, still picks wrong for one spelling. Take "2011-12" under the default policy:
      • the cutoff reads it as December 2011, and it reads the facts' own spellings untyped too;
      • under a fiscal-year selector it has to become "2012-13" to keep selecting the year's own fact;
      • under a calendar-year selector it has to become "2012-12".
    • So where the selector's type reads a spelling differently from the cutoff, the restamp now refuses. An untyped exact match refuses "2011-12" the same way, since it reads the period under each fact's own type. A prefixed or four-digit spelling reads one way.
    • The local compile test now uses "FY26". The parser reads it as the year 26, so it can't move under any policy.
    • Residuals:
      • the int path checks the year under the type, so a month type refuses a bare year;
      • the constructor checks an exact period under the selector's type, so a fiscal "2099-00" stands, and an exact fiscal "2026-01" is refused when the reference is built;
      • the changelog and docstrings no longer say "as the exact-match path does". They describe each policy's reading and the two refusals.
  • 2. Floor guard: IGNORECASE is back, and the floor words (min, minimum, floor, sdc, small, threshold) are matched as snake-case words of a name.
  • 3. Observation-period tests: a comment now says D5's periods are untyped Januaries after an integer year, and that a fiscal, prefixed or range-spelt period needs its own expectation.

Parity-reason replay: my round-4 reply overstated it.

  • The classifier's four reasons now live in one constant, PARITY_DIFFERENCE_REASONS in gates.py, with unchanged texts.
  • The test starts each row from its kind's reason, then replays the tool's rules, so a changed classifier text and a removed rule fail too.
    • Removing the PIP rule fails the national receipt.
    • Changing the fixture-only text fails the national receipt too. The local receipt's generic rows are all calibration drift, so a change to that text fails both.

The touched test files pass locally, and each fix fails its tests under mutation. The gate digests, the coverage manifest check and the H2 fixture check pass too. CI runs the full tiers.

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Sixth review of #1422 at 4af602d81

This re-reviews 4af602d81 against my fifth review at 0da33d0cd. As before:

  • one reviewer re-ran each earlier finding;
  • a second read the commit for new defects.

Paths are relative to packages/microcosm-build/src/microcosm/build/ unless they start with packages/ or tools/.

Verdict: the round-5 items are fixed, CI is green, and nothing blocks a build. What's left is latent:

  1. nothing tests the guard's new name matching;
  2. a few period spellings from before 2013;
  3. two cross-PR notes.

Once 1 is pinned, this looks ready to me.

Round-5 items

# Status
1. Restamps by match policy Fixed. Under the default policy, a fiscal- or academic-year "2026-01" moves to "2027-01" again, and a month selector's "2026" moves to "2027". Exact typed periods are checked at construction. Every committed reference still builds: 1,409 national, 27,168 local, and the NZ, AM, BE and US ones. The 616,875 restamps from 2018 to 2035 all match round 5. Residuals are in 2.
2. Floor guard Fixed. It now catches lowercase snake-case floors, annotated defaults, single-quoted keys and #: wraps. admin_count=10 and minutes=10 stay quiet, and kept phrases survive re-indenting. But see 1.
3. Observation-period tests Fixed (the comment).
Parity-reason replay Fixed. The four texts live in one constant and are byte-identical. A changed classifier text, a removed rule and a stale generic row each fail now.

Remaining

1. Nothing tests the guard's new name matching (packages/microcosm-build/tests/engine_free/uk/test_uk_receipt_counts.py:225-234).

  • Why it can regress: the guard scans only the live tree, and there every name-style floor is an upper-case constant that is already kept. So narrowing the pattern again, as 0da33d0cd did, passes CI. Making name upper-case only passes, and so does deleting the annotated-default or the quoted-key alternative.
  • Fix: a parametrized self-test of the regex would pin it.
    • It should match support_floor=10, sdc_floor: Final[int] = 10, 'min_rows': 10 and MIN_CELL = 10.
    • It should not match admin_count=10, minutes=10 or {n:0>9}.
  • Shapes it now misses that round 5 caught (none of them live):
    • nested or quoted annotations on constants, such as MIN_CELL: Final[Optional[int]] = 10 and 'Final[int]', though the docstring says "annotated or not";
    • glued names such as MINROWS and thresholdRows;
    • comparisons right after a colon, such as lambda r:r<10 and a[i:]<=10.
  • False positive: space-fill format specs such as f"{n: >9}" are still flagged, loudly.

2. Period spellings before 2013 or after 2098 (latent; DfE's "2026-01" is unaffected).

  • Untyped selectors still move "2011-12" to "2012-12" silently. When the facts are range-spelt, that re-selects FY2011-12. The same happens for "2010-11", "2005-06" and "2000-01". The root cause predates this PR: the cutoff and _period_key read range spellings up to "2011-12" as months (ledger_targets.py:5281-5282).
  • Kind change: "2012-13" moved back a year becomes "2011-12", which the cutoff reads as a month.
  • Over-strict: typed selectors refuse some spellings that would move correctly, such as "2010-11" +1.
  • On the reply's reasoning: every committed fiscal-year fact is an opening-year integer. For those, "2011-12" (a December 2011 cutoff) selects FY2010-11, so "2012-12" is the right move. Refusing is still the safe outcome, given range-spelt facts.
  • "FY26": the constructor accepts it as the year 26. So the local test's case compiles as an unsupported row at its authored period, and refuses only when restamped.
  • Prefixed periods: an exact typed selector accepts a prefixed period it can never match, such as "AY2026-27" under fiscal_year, while "2026-01" is refused.
  • Unnamed rows: the authoring and tool refusals still don't name the row (target_reference_authoring.py:135-156).

3. Cross-PR:

Checked and found true

  • Restamps: identical to round 5 for every committed reference, and the constructor accepts every committed reference.
  • Tests and texts: the touched tests fail under their mutations, and the parity texts are byte-identical.
  • Guard: it scans the tree in under 2 s.
  • Merges: the forward merge with UK: company cars from the FRS job table (dashboard #316) #1430 (40968ff53) is clean and its tests pass.
  • CI on 4af602d81: every job passes.

Disclosure-control items went to the author separately.

🤖 Generated with Claude Code

@juaristi22
juaristi22 merged commit 11c1f4e into main Nov 1, 2026
10 checks passed
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's 0da33d0 answers its fourth review: periods are read under
their declared type, a period's year needs four digits, the authored
observation period reads the selector's period type, the assembled
boundary moves to graph.py, and the floor-of-ten guard reads more
spellings of ten. It merges without conflicts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422's 4af602d answers its fifth review: an observation period is
read and moved as its reference's match policy reads it, a spelling
read two ways refuses to move, the compile-parity reasons are one
constant the receipt test replays, and the floor guard reads floor
words in more names and spellings. It merges without conflicts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 1, 2026
#1422 merged as 11c1f4e, whose head 4af602d this branch already
merged. Main brings US, New Zealand and Belgian changes only; no UK file
moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Nov 1, 2026
Main adds S3, w3a, #1422 and #1436. Git conflicts in one place, the import
block of transport/cli.py: S1 widens the population_kernels import, and
#1436 adds the entitlement import above it. Both stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
Main brings #1422 (policyengine-uk 2.165.1 and the small-cell floor of
three) and #1430 (the company_cars stage and the Chronicle feed re-pin
to 08962f1). Following #1430's merge order for this PR:

- Both PRs add a stage in the same places, so the stage lists, kernels,
  transforms and coverage contracts keep both: company_cars after the
  SPI income stage, trading_allowance_receipts still last. The stage
  count is 52 in the spec tests, the graph test and the fixture tool.
- The UC take-up test keeps this PR's gap passing with #1422's source
  households and floor: band 0's pool withholds its flag (<3), band 2's,
  on four sources, says the count exceeds every entitled unit.
- The Table 3.10 employment count reads
  employment_income_with_benefits_in_kind_annual, the term every HMRC
  employment income row reads since #1430, with its notes, cross-grain
  reason, test and changelog.
- The support floor's adjudication keeps #1422's floor of three and this
  PR's extension to the employment count.
- Counts add both sides: 460 population targets and 398 in the registry
  scope, 45 held measure names, 185 mapped and 213 unmapped declarations,
  1,418 active references and 37 signed exclusions.

Regenerated on Chronicle 08962f1 (the artifact María supplied, digests
verified): the Table 3.7 savings rows (feed identity only; the values
are unchanged), the national references and membership and the national
compile-parity receipt from the national cut, then the coverage
manifest, the H2 fixture on policyengine-uk 2.165.1, and the gate-battery
pins and certification part digests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit to PolicyEngine/policyengine.py that referenced this pull request Nov 2, 2026
… 2.165.1

The bundle moves from the national cut 0a4c8e2c (a8e25870c, 2.164.2) to
f6e33b93, built from PolicyEngine/microcosm#1428 at 329a99e71: the
independent review's fixes, recalibrated trading allowance claimant
rates, and main with PolicyEngine/microcosm#1422 and
PolicyEngine/microcosm#1430 merged. It is certified on its own build,
policyengine-uk 2.165.1.

The local-area cut stays 6528c865 (Max's call between it and c9df4b86),
re-audited for 2.165.1: no dropped column, and fourteen inputs added
after 2.162.12 take their defaults, now including the nine company car
inputs, so local-area results carry no company cars.

The lock, release lock, TROs and the household calculator snapshots are
regenerated on 2.165.1, and the bundle checks pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
#1426 now builds on #1422, which moves every UK disclosure floor to three
(receipt_counts.UK_DISCLOSURE_FLOOR) and regenerates the national surfaces
on Chronicle 53815b2 with policyengine-uk 2.165.1.

- The SPI pay-floor receipt withholds through UK_DISCLOSURE_FLOOR and
  UK_WITHHELD: a figure on one or two source households reads "<3" where it
  read "<10" (Vahid's review of #1426). Its description in spi_spine.py,
  sources.yaml and source_stages.json, the hourly-pay gate's note and
  docstring, and its tests follow. Two test cases shrink their small side to
  two households, so they still test the withholding.
- Conflicts: the SPI income imports (both sides kept), the stage-1 fold key
  (#1426's age band) beside #1422's savings-interest method text, and the UC
  take-up gate test.
- Regenerated: the release input coverage manifest. The gate battery and
  spine-part digests are re-pinned.
- The H2 parity fixture takes #1422's side here; a later commit regenerates
  it on the merged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
Only the gate digests conflict; they are re-pinned on the merged gates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
tools/graph_uk_spine_fixture.py on #1426's items over #1422's branch, after
the review fixes; --check regenerates it byte-identically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
Only the gate digests conflict; they are re-pinned on the merged gates.
#1422's new floor-of-ten guard passes on #1426's SPI pay floor and age
band code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
The gate digests and the coverage manifest's source digest conflict; they
are re-pinned and regenerated on the merged gates and spec. #1422's
broader floor-of-ten guard passes on #1426's SPI code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
The fixture conflicts, as #1426's SPI stages differ from #1422's; it is
regenerated on the merged tree, --check reproduces it byte-identically, and
the H2 parity test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
No conflicts. The exact-count floor guard passes on #1426's SPI and HRAD
code, as do the stage, ledger and reference tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
No conflicts. The stricter floor guard passes on #1426's SPI and HRAD
code, as do the ledger, reference, stage and graph tests. The gate
digests and the coverage manifest are unchanged, and the H2 fixture
regenerates byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
No conflicts. The broader floor guard passes on #1426's SPI and HRAD
code, as do the ledger, gate, reference, stage and graph tests. The
gate digests and the coverage manifest are unchanged, and the H2
fixture regenerates byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Nov 2, 2026
#1422 landed (11c1f4e), followed by #1430's company cars and #1432's
default load. Only pinned and generated files conflicted. The gate
battery and spine-part digests are re-pinned on the merged gates spec
(five move), and the release input coverage manifest and the charter-H2
fixture are regenerated; the fixture checks byte-identical.
policyengine-uk stays at 2.165.1, the lock main and #1428 share, and the
feed pin moves to main's 08962f1.

The H2 parity test and 86 related engine-free files pass (5,682 tests,
53 skipped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants