E-G fix round: feed-free label tests, the stand-in rename and R2C-26

README.md

@@ -51,23 +51,40 @@ immutable publication before Hugging Face artifact staging, and preserves
 durable retry jobs and receipt snapshots. See
 [graph publication](docs/orrery-publication.md) for configuration and retry.
 
-Supported US and UK build commands start one local emitter service with separate
-telemetry and graph-publication workers. Microcosm supplies build metadata and
-graph files; the required `microcosm-emitter` package owns authentication,
-durable queues and delivery. Telemetry and graph publication are internal
-libraries in that single package, not separate installations.
-This draft migration requires a published emitter release before merge; see the
-[migration prerequisites](docs/orrery-publication.md#deferred-migration-prerequisites).
-The telemetry worker reports live progress to the hosted collector when the operator's
-existing Hugging Face login is accepted; otherwise it retains the events
+Supported US and UK build commands start the local emitter as the build's
+child process, with its telemetry, graph-publication and Hugging Face
+hand-over modules. Microcosm depends only on `microcosm-emitter-interface`,
+the typed calls a build makes; the emitter itself is installed on its own,
+outside the workspace environment, so an emitter release needs no Microcosm
+change:
+
+```bash
+uv tool install microcosm-emitter   # once per machine
+uv tool upgrade microcosm-emitter   # to take a new release
+```
+
+The build finds the `microcosm-emitter` command through
+`MICROCOSM_EMITTER_COMMAND`, then `PATH`. The emitter inherits the build's
+environment variables and user, so it uses the build's Hugging Face
+credential (`HF_TOKEN`, then `HUGGINGFACE_TOKEN`, then the `hf auth login`
+token). Without the command the build continues with no hosted telemetry and
+no remote staging. Before it starts the emitter, the build reads its
+version from `microcosm-emitter version` and does not start one older than
+0.6.0, the first release that takes staging run files; it prints the
+installed and required versions and the upgrade command, and continues as
+it does without the command. Microcosm supplies build metadata, graph files and staging
+run files; the emitter owns authentication, durable queues and delivery.
+The telemetry module reports live progress to the hosted collector when the
+operator's Hugging Face login is accepted; otherwise it retains the events
 locally and the build continues. This live event path is independent of the
 staging run files described below.
 
 US fiscal refresh builds also write pre-release staging run files **by
-default**. Progress JSON is uploaded to `policyengine/populace-us-staging`
-while the build runs (best-effort — a missing token or failed upload never
-fails the build), so every candidate shows up on the staging dashboard before
-it is published. Disable these files with `--no-staging`, or point them
+default**. The build hands its progress JSON to the emitter, which commits it
+to `policyengine/populace-us-staging` while the build runs, one commit per
+upload interval (best-effort: no emitter, a missing token or a failed commit
+never fails the build), so every candidate shows up on the staging dashboard
+before it is published. Disable these files with `--no-staging`, or point them
 elsewhere with `--staging-repo-id` / `POPULACE_STAGING_REPO_ID`. An *empty*
 `POPULACE_STAGING_REPO_ID` is ignored rather than read as off, and staging with
 no destination at all is an argparse error. `--no-staging` does not disable

changelog.d/1019-acs-local-take-up.fixed.md

@@ -0,0 +1 @@
+Seed SNAP and TANF take-up on the ACS rows of the ACS local-area release (microcosm#1019). Every ACS SPM unit previously reached both engine passes with `takes_up_snap_if_eligible` and `takes_up_tanf_if_eligible` missing, and the reviewed-null fill gave them the engine default, so every eligible ACS unit took both programs up. `microcosm.build.us_runtime.acs_local_take_up` now fills only missing ACS-row cells before either fill: `receives_snap` reporters take SNAP up and non-reporters draw to the `snap_take_up` stage's national FNS rate, and TANF uses the take-up contract's seeded rate. Materialize records the seed and an assignment digest, and the consumer export must reproduce it. The reviewed-null fill refuses to default-fill either column, finalize hard-fails on a new per-spine `acs_local_take_up_signal` gate (missing, constant, unanchored or out-of-band take-up), and packaging requires that gate passing and bound to the packaged H5. The other ACS take-up flags stay a reviewed limitation (microcosm#1022). Two informational measures are recorded and not graded: the take-up receipt cross-tabs transferred `receives_tanf` against the independent TANF draw, and the QA probe reports a December-eligible SNAP take-up snapshot, unit- and person-weighted, by spine, state and household type. The snapshot is not the FNS fiscal-year participation rate, which is among eligible people over a federal fiscal year.

changelog.d/1418-spi-donor-savings-rebase.changed.md

@@ -0,0 +1 @@
+The SPI income stage now rebases the 2022-23 tape's interest and other income to HMRC's published 2023-24 Table 3.7 amounts band by band (the published rows are vendored as `uk/hmrc_spi_table_3_7_savings_rows.json`), then carries both to the frame's year on the OBR's savings income growth (October 2026 EFO, Table 4.5), the path the Table 3.7 band targets take (#1418 savings follow-up, #1428). Before, each column moved by its engine index from 2022 to 2024: other income by GDP per head (x1.09) where SPI's own other income grew x1.65 in the single year to 2023-24, and interest by ONS household interest (x2.27) where SPI's grew x2.69 in that year. The frame then carried about 70 percent of the Table 3.7 savings rows at design weights, and the solve met them by stretching the weights of the few households that carry them. Each band's factor is the published amount over the FACT-weighted amount of the tape's taxpayers in the band, as Table 3.7 counts taxpayers only. The tape is banded once on its income moved to 2023-24, each income leaf at its own factor and each rebased column at its factor over all bands, so the band-edge migration its consumers re-band on uprated income is counted once; the income is the published total plus the leaves' growth, net of employment expenses and without the gift leaves, which SPI's total income doesn't hold, and the factor over all bands counts the same taxpayers as the band factors; a record whose band's factor differs from that of all bands can still sit across an edge on its rebased income. A taxpayer is a record the tape flags (TAXPAYER 1) or one whose income crosses the Personal Allowance by 2023-24, so a saver who becomes a taxpayer counts once; a flagged taxpayer below the first band sits in it (note 8), and any other record takes the factor of all bands. A band too thin to gross (under a tenth of its published amount) takes the factor of all bands, a column whose tape does not gross to the published amounts is left as it is and grows on its engine index from 2022-23, and a tape whose TAXPAYER codes are not 1 or 2 is refused. On the tape, the £1m-and-over band's other income outweighs its interest, as in the published table, so the rebase also undoes the split the two indices had flipped.

changelog.d/1428-dwp-jsa-income-based.added.md

@@ -0,0 +1 @@
+The national calibration binds DWP's income-based Jobseeker's Allowance spending, `dwp.jsa.amount_income_based` (benefit expenditure and caseload tables, Spring Forecast 2026), at the calendar-2025 window of the fiscal years, as the pension-age Housing Benefit row is (#1428; María's ruling of 2026-10-31). Income-based JSA closes with the move to Universal Credit: £91.0m in FY2024-25 and under £1m from FY2025-26. Without the split, the calibration meets OBR's total JSA by scaling income-based reporters together with contributory claimants, far past DWP's income-based spending.

changelog.d/1428-frs-other-income-definitions.changed.md

@@ -0,0 +1 @@
+FRS rows now book NS&I income on SPI's definitions (#1428). NS&I Direct Saver and the other NS&I and bank products go to `savings_interest_income`, because SPI has counted all interest paid gross or net as bank interest since 2016-17 (Table 3.7 note 4); gilts stay in `other_investment_income`. Income from an annuity not bought with pension funds or from a trust or covenant (FRS PENTYPE 4 and 5) stays in `private_pension_income`, where policyengine-uk's means tests (Pension Credit, Universal Credit, Housing Benefit, Council Tax Reduction, Income Support), household income and tax read it. Its part is stored in a new calibration-only descriptor, `annuity_and_trust_income`, and SPI Table 3.7's other-income rows read `other_investment_income_spi_concept`: the engine's other income plus that share of the engine's private pension, as the table counts both payments as other income (note 6). A row the SPI stage draws holds the tape's pension, which excludes them, so its descriptor is zero. Its other income is the tape's OTHERINV, Table 3.7's other income, which holds those payments, and policyengine-uk taxes it as savings income: on drawn rows they still take the Personal Savings Allowance, the starting rate and UK rates in Scotland. That limit predates this change; the tape doesn't split OTHERINV. The FRS spine now refuses a pension tab without PENTYPE.

changelog.d/1428-isa-fit-weight-records.fixed.md

@@ -0,0 +1 @@
+The ISA subscriptions stage no longer declares an empty list of fit-weight records (#1428). It fits no model, but the release-cut weights audit read its empty list as a fitting stage that recorded nothing, so every national certified after #1421 failed `uk_weights_audit`.

changelog.d/1428-spi-band-donor-total-income.fixed.md

@@ -0,0 +1 @@
+The SPI band donors place each tape record by the total income it realises on the frame as SPI's total income holds it: its income leaves at their factors net of employment expenses, without the gift leaves, which are reliefs, plus the rest of the published total, no longer floored at zero (#1428; Vahid's review). Before, a record's gifts and employment expenses added to its income, so a donor with large gifts of shares, or with expenses, sat in too high a band. With every factor at one the realised total is the published one. The savings rebase bands the tape the same way.

changelog.d/1428-spi-employment-by-self-employment-band.added.md

@@ -0,0 +1 @@
+The national calibration binds SPI 2023-24 Table 3.10's individuals with employment income by self-employment band, `hmrc.spi.employment_income.count_by_self_employment_income_band`, over the eight bands from £1 to £10,000 that the self-employment counts and amounts already bind (#1418; María's choice of 2026-10-30). With the self-employment counts alone, the solve could meet each band with traders who have no pay, while in Table 3.10 more than half of the traders below £3,000 also have employment income; an employee's side profit is taxed at their marginal rate, so the mix of small traders decides what the Budget's trading allowance measures cost. Each person with pay or company car benefits over the year (`employment_income_with_benefits_in_kind_annual`, the term every HMRC employment income row reads since #1430) counts in their own self-employment band, taxpayer or not. Table 3.10's column is wider, income assessed under ITEPA 2003 plus other miscellaneous income, so like Table 3.6's employment rows the binding leaves out miscellaneous income and the contributory benefits; benefits in kind count on SPI rows, whose pay is the tape's PAY + EPB + TAXTERM, and the company car and car fuel benefits on every row. Counts grow to 2025 with HMRC's projected growth of all taxpayers (Income Tax liabilities statistics Table 2.5), as the self-employment counts do, and the zero band and the bands from £10,000 are signed out row by row.

changelog.d/1428-spi-gift-anchor.changed.md

@@ -0,0 +1 @@
+The SPI income stage anchors the drawn copies' Gift Aid and charitable investment gifts to the SPI tape's own weighted totals (#1428; María's ruling of 2026-10-31). Both are sparse leaves that the stage-1 chain draws after, and conditioned on, the income leaves, so a change to the savings draws reshuffles which few donors' gifts the copies draw. Each column's total over the drawn copies, at the stage's household weights, is scaled to the tape's FACT-weighted total. A column that the tape or the copies carry none of is left as drawn. The scaling fixes the totals, not how concentrated each column is. The stage evidence records each column's factor, never a total or a count.

changelog.d/1428-spi-other-income-analogue.changed.md

@@ -0,0 +1 @@
+The SPI stage-1 transport now locates each copy's other-income draw at its FRS twin's own other income (gilt interest), as it does for the other investment leaves, instead of an identity uniform that ignored the twin's holdings (#1428).

changelog.d/1428-trading-allowance-receipts.added.md

@@ -0,0 +1 @@
+A new last spine stage, `trading_allowance_receipts`, stores the self-employed's gross receipts (`self_employment_gross_receipts`), so policyengine-uk applies the trading allowance against receipts as the law does (#1428; María's choice of 2026-10-31). Neither the FRS nor the SPI tape records turnover. Without receipts the engine relieved profits up to the allowance in full and nothing above it, so a rise in the allowance missed its main base: traders who already claim the £1,000 allowance against larger receipts. Under current law a claimant's receipts are their profit plus £1,000 exactly. Each trader with a positive profit is flagged as a claimant with probability 0.72 for profits up to £4,500, 0.14 from £4,500 to £9,000 and none above, drawn on their own `person_id`. A claimant's receipts are their profit plus £1,000; any other trader's are their profit plus £3,000, expenses at the raised allowance, since the costing books traders who switch to the allowance as behaviour. The rates are calibrated to HMRC's policy paper (about 400,000 taxpayers with a lower bill, by about £380 on average), measured on policyengine-uk for the national from 9f70c71e7, until HMRC publishes claimant counts; they drift with each national, so #1439 re-measures them after each build. Every trader with a positive profit is treated as filing, as the Table 3.10 counts the calibration binds are of filers, so a profit of £1,000 or less is now taxed where the engine used to relieve it in full. Two limits remain until policyengine-uk#2765 lets the stage store the claim status itself (#1439). A claimant's £1,000 grows with the mixed income index while the allowance is nominal, so the raise relieves £1,932 more in 2027-28, falling to £1,758 by 2031-32. The other traders' stored £3,000 grows the same way, so allowances above £3,000 are mis-scored. `self_employment_gross_receipts` leaves the register of engine inputs left at their default and joins the export surface's allowed columns. The stage spec and schema, the graph and kernel wiring, the spine build, the coverage manifest, the H2 fixture and the data contract's gate digests follow.

changelog.d/1428-uc-take-up-passes-unmet-gaps.changed.md

@@ -0,0 +1 @@
+The Universal Credit take-up stage now passes the gap of a pool that cannot meet DWP's count to its neighbouring pools, so each family type's caseload lands on DWP's wherever its entitled units allow (#1428; María's rulings of 2026-10-31 and 2026-11-01). The stage pools adjacent award bands within a family type until they rest on enough sources. A pool can't meet DWP's count when its reporters' and other undrawn units' expected claims already exceed it, or when the count exceeds every entitled unit. Such a pool used to keep its clipped rate, which left the family type's caseload off DWP's by the gap, and which bands fell short moved with small upstream changes. Now such a pool is held at its clip and its gap passes on. A shortfall and an excess elsewhere in the family type offset first. What remains goes to the other pools, nearest first, the one with more room first at equal distance and the lower band on a tie, each taking what its room allows. Every band keeps its own rate, so no band's drawn units claim against its own count to meet another's. The stage spec, the coverage manifest, the H2 fixture and the data contract's gate digests follow.

changelog.d/823-national-line-default-load.changed.md

@@ -0,0 +1 @@
+The UK 2024-25 national line is the default UK load: `microcosm.data.load("uk")` and `load("uk", 2025)` follow `latest-national.json`, which has named the certified cut microcosm-uk-2024-25-national-20261025T091414Z-a6299504 (built on policyengine-uk 2.151.0) since 26 October 2026, instead of the June 2023 `populace_uk_2023` artifact, which stays at `load("uk", 2023)`. The line's own `national` variant key retires (microcosm#823, Max's ruling).

changelog.d/823-uk-2025-registry.changed.md

@@ -1 +1 @@
-Registered the UK 2024-25 national line in the data registry as variant `national` on `microcosm_uk_2024_25.h5`, followed through `latest-national.json`, and taught the certified loader to follow per-line pointers at immutable revisions; non-default local-area manifests load only through their line pointers. `available()` now lists default-variant `(country, year)` pairs only (`available_variants()` lists every registered variant, the dense line included). The UK default stays on the June 2023 line until the national line's first promotion and the follow-up flip of its entry to the default variant.
+Registered the UK 2024-25 national line in the data registry as variant `national` on `microcosm_uk_2024_25.h5`, followed through `latest-national.json`, and taught the certified loader to follow per-line pointers at immutable revisions; non-default local-area manifests load only through their line pointers. `available()` now lists default-variant `(country, year)` pairs only (`available_variants()` lists every registered variant, the dense line included).

changelog.d/be-quantile-surfaces.added.md

@@ -0,0 +1 @@
+Add Belgium's wage and legal-pension quantile surfaces (B4a), still inactive. `tools/generate_be_target_references.py` now writes 31 more pre-calibration references. Five sum RSZ/ONSS T17's employed workers at the end of 2026-Q1 in each known-pay daily-wage class over its 24 age and sex cells. Twenty-six read SFPD's January 2025 legal-pension amount bands, parked (`requires_pension_route_d1902`) until shared rank-only support lands. A new resource, `quantile_surfaces.json`, holds each surface's band document, the bands parameter `transport.quantile_map@1` takes, beside its unit, period, cohort and coverage notes. The wage edges are euros a day, so nothing may map the donor's annual wages onto them until d1902 picks a conversion. The pension edges are monthly and not annualized. `reference_inventory.json` records the counts behind both surfaces: known plus unknown pay equals T17's quarter total, T7 agrees class by class, and the 26 bands sum to SFPD's published total. The package tests now accept a summed row whose rosters span several record sets, check that its declared member count and its listed reads match the roster product, and expand every roster when they look for a holdout that reads the same cell. Every method choice remains proposed pending d1902, and Belgium still refuses to activate.

changelog.d/be-reference-documents.added.md

@@ -0,0 +1 @@
+Add Belgium's calibration, pre-calibration and holdout reference documents, generated by `tools/generate_be_target_references.py` from a pinned Chronicle `be` feed (`chronicle_feed.json`). References carry selectors only. Demography targets Statbel's 1 January 2026 population by region, sex and age. Statbel A.1's eight broad stand-in income bands (the d1952 measure), B.4's joint and individual declarations, ONSS contributions and FPB benefit and mixed-income aggregates roll 2023 to 2025 facts to 2026 by declared FPB June 2026 ratio pairs. Commune population is the pre-calibration geography support. Eurostat D51A_C1, NBB and Eurostat household accounts, and FPB's gross wages are holdouts. B.1 assessed tax stays inactive because its tax-definition, income-ranking and policy-year checks fail. Rows without a producer or ruling stay parked with a named reason. `reference_inventory.json` records the stand-in gaps, ageing ratios, B.1 checks and every exclusion, and `reference_aging.json` holds G8-r2 diagnostic factors from RSZ wage totals. The catalog declares two euro money columns. Belgium stays inactive, and every method-card choice remains proposed pending d1902.