[anth-volk]

Fixes #749
Fixes #618

Summary

This certifies the 2036 US bundle: policyengine-us 2.2.2 with spm-calculator 1.1.0, whose SPM forecast covers 2022–2036. To keep state and district results, it extends certify-data --accept-regional-model-mismatch and its input-surface audit to the US, and accepts the ACS-local release on an audited mismatch.

Why a mismatch rather than a claim

• The national release populace-us-2024-spm-20260915 already claims policyengine-us >=2.0.0,<2.3, so it certifies for 2.2.2 on that claim.
• The ACS-local release populace-us-2024-buildo-acs-local-766b7c7-20261017T030317Z was built with 2.2.1 and claims nothing else. policyengine/populace-us now publishes builds in their own release folders, which are never changed, so no claim can be added to it.
• The alternatives were a new Microcosm build, a local-area recertification (Microcosm's tool accepts only national parents today), or dropping the regional release, which would send state and district runs to the national file.

US support for the accepted mismatch (#749)

• scripts/audit_regional_input_surface.py and regional_input_surface.py read the country model the regional manifest's build model names: policyengine-uk or policyengine-us. The target is read in process, and the build model in the throwaway uv environment with its own package.

• US files are read in either layout .py's US loader reads, by names only, never values:

  • entity tables, one per US entity, with _time_period naming the year (the ACS-local release's layout);
  • PolicyEngine core's variable/period groups.

  A missing entity table, a missing year, a US entity-table file stored by year (which .py's US loader and policyengine-us's Microsimulation(dataset=<path>) cannot read), or a file in neither layout refuses.

• US input aliases are .py's pending legacy input renames (LEGACY_INPUT_RENAMES), read from legacy_inputs.py without loading .py's US model. A rename counts only where .py applies it: the file stores the legacy column and not the live input (in any period, for the variable/period layout). There the live input counts as supplied and the legacy column is neither dropped nor never modelled; elsewhere the legacy column is classified like any other. UK aliases keep their shape (a move between two variables the model defines).

• Certification drops its UK-only refusal, and adds the UK VAT limitation only for policyengine-uk. .py writes execution receipts for UK results only, so for the US the record is data_releases.us.regional_certification (evidence included) and the US TRO.

• docs/engineering/skills/data-certification.md documents the US route.

The audit of the ACS-local H5

Run by Anthony Volk on the release's H5 (populace_us_2024_acs_local.h5, 9.8 GB, SHA-256 3ce781c4…, matching its manifest) at commit 28f355b8:

Measure	Result
Year	2024
Stored columns	420, across the six entity tables
Dropped columns	none
Inputs newly defaulted	none
Never-modelled columns	228, defined by neither model: the raw ACS and CPS source fields Microcosm keeps (AGEP, A_AGE, ADJINC…), and 39 Microcosm identifiers and derived fields (filing_status_input, tax_unit_role_input, puma_geoid…)
policyengine-us 2.2.1 and 2.2.2 surfaces	identical: 6,167 variables, 925 inputs, so no input can newly default

The evidence and the accepted reason are recorded in data_releases.us.regional_certification, with the limitation: "The ACS-local release was built with policyengine-us 2.2.1; 2.2.2 adds only the spm-calculator 1.1 pin and CBO's 2036 CPI-U and chained CPI-U points (policyengine-us#10765), with no variable, input or loader changes. State and district results rest on this accepted mismatch, not on the release's own build or a claim."

The certification

• Pins: policyengine-us==2.2.2 and spm-calculator==1.1.0 in the models, us and dev extras. Only those two packages move in uv.lock, checked with uv 0.13.4.

• SPM: set-spm stages forecast 335e6755dc8e…, the one spm-calculator 1.1.0 installs (scenario ce_trend, county geography, vintage 2020), and the published wheel (c528ec0f…). A new test checks the bundle's digest against the installed forecast.

• certify-data was run with the folder addresses:

  • national: hf://dataset/policyengine/populace-us@0ff35303e4e5c10bf1cfb2da8f65c83d7dfa9ab3/releases/microcosm-us-2024-national/populace-us-2024-spm-20260915/release_manifest.json;
  • regional: hf://dataset/policyengine/populace-us@28f355b8931d14fd9705ebe818554f578f1719d6/releases/microcosm-us-2024-local/populace-us-2024-buildo-acs-local-766b7c7-20261017T030317Z/release_manifest.json.

  Its warnings:

  • the national release is certified on its publisher claim (legacy_compatible_model_package);
  • the regional release is routed on the accepted mismatch;
  • the bundle's core 3.33.3 is not in either release's core claims, and the certificate follows the core pin, as before.

• TROs: generate --include-tros --strict-tros regenerated them, and check --published-spm --include-tros --strict-tros passes. The UK TRO changes only its bundle-manifest digest and fingerprint, because that digest covers the whole bundle file.

What changes for consumers

• Dataset addresses move to the build folders, at the publishing commits, because populace-us switched layouts:

  • the national dataset's URI is hf://policyengine/populace-us/releases/microcosm-us-2024-national/populace-us-2024-spm-20260915/populace_us_2024.h5@0ff35303…, not …/populace_us_2024.h5@populace-us-2024-spm-20260915;
  • the state and district path templates gain the releases/microcosm-us-2024-local/<build>/ folder.

  The bytes are unchanged (6496cc43… and 3ce781c4…). Consumers that key results or caches on the dataset URI will see new ones.

• US runs: a New York household calculates for 2026 and 2036 on this bundle, and 2037 is refused ("Forecast has no entry for 2037"). US runs carry no execution receipt, so the accepted mismatch shows in the bundle manifest and the US TRO, not per run.

#619 is included

Neither PR passed alone: on main, #619's 2036 household case fails, and on this bundle the 2036 refusal case fails. So #619's single commit (95d7d59, David's) is cherry-picked onto this branch as 658a452e, unchanged:

• docs/households.md and docs/microsim.md give the 2022–2036 horizon;
• tests/test_spm_household.py adds 2036 to the supported household years and uses 2037 as the out-of-horizon year;
• changelog.d/618.changed.md.

#619 can close as included here.

Review

Round 1 (a fresh reviewer, given the diff and the issues) found, all fixed test-first:

1. Critical: the bundle pinned forecast digest 335e6755bc8e…, one character off the installed 335e6755dc8e…, so policyengine-us refused every US SPM selection and every US calculation failed. validate_bundle_measurements only checks the digest's format, so the bundle check passed. A test now compares the digest with the installed forecast.
2. Docs and fragments said US area receipts are uncertified, but .py writes receipts for UK results only; they now say where the US record is.
3. The rename accounting counted a stored legacy column as read even where the live input is also stored, which .py does not do; that could hide a dropped column.
4. Reading the renames loaded .py's whole US model, which checks the installed bundle's certification and would fail for a target the bundle does not yet cover.
5. Lower-confidence: a multi-year US entity-table file was accepted though .py's US loaders cannot read it (now refused); stale versions in docs/households.md and tag addresses in the US certification example; two inaccurate test comments.

Round 2 (another fresh reviewer, over the full diff at a12158ba) found no substantive bug. It checked the bundle, both TROs and the test constants against the published releases and the installed packages; ran US households for 2026, 2035 and 2036, the national managed microsimulation for 2026 and the Simulation path for 2026 and 2036; and fuzzed the old and new UK comparison on 20,000 inputs with no difference. Its lower-confidence notes, now corrected in comments and docs:

• A known limitation: for entity-table files .py applies a rename only where the live input is absent from its own entity's table, but the evidence keeps one column list per file, so the audit checks across all tables. A legacy column stored in another entity's table would count as mapped. Neither 2.2.1 nor 2.2.2 defines would_claim_wic, and the ACS-local file stores neither name in the wrong table, so this changes nothing here.
• A file stored by year is refused because .py's US loader and policyengine-us's Microsimulation(dataset=<path>) read single-year files; policyengine-us's USMultiYearDataset does read them.
• A stale docstring.

Not verified: the 2036 national managed microsimulation, which the reviewer stopped after 20 minutes as its memory grew; it showed no error.

David's review at 39f3eb0 found one test that still expected the ACS-local file's old address: test__given_local_area_dataset__then_registered_with_sha_but_not_default asserted the bare file name at the release tag. It now asserts US_LOCAL_AREA_PATH at US_LOCAL_AREA_REVISION (bc5049f9), and the neighbouring resolution test is renamed …_then_resolves_to_its_build_folder, since the dataset no longer resolves to a tag.

Testing

• Tests first. The new tests/test_us_regional_input_surface.py (27 cases) failed on main before the change, and round 1's five new tests failed on the round's base:
  • the audit refused a release built with policyengine-us;
  • compare_input_surface took no model package;
  • certify-data refused a US mismatch as UK-only;
  • the entity-table cases failed before the reader existed.
• What it covers: both US layouts, the build-model probe's pins, dropped, never-modelled and defaulted columns, the legacy rename, alias shapes for each country, and certification of a US accepted mismatch.
• Changed tests, run individually by node ID: the UK test stand-ins that now take model_package, and the tests whose constants name the certified versions and addresses: test_release_manifests.py, test_models.py, test_bundle.py, test_bundle_metadata.py, test_spm_bundle_tooling.py, test_us_regions.py. The full suite was not run locally.
• After review: the two local-area manifest tests pass, and so do all six cases of Document and test the 2022–2036 SPM forecast horizon #619's two changed tests (test_explicit_national_amounts_and_component_provenance for 2024, 2026, 2035 and 2036; test_year_outside_canonical_horizon_is_not_silently_extrapolated for 2021 and 2037).
• ruff format and ruff check pass.

🤖 Generated with Claude Code

[DTrim99]

Thanks Anthony. I reviewed the full diff at 39f3eb0 from the US side, checked the certification against a run of my own, and ran the state and district route on Modal. Two things before merge, both small.

1. One test still expects the old ACS-local address. CI fails two tests, the same on all four Pythons (run 43264259222). The first is the expected test_spm_household.py::test_year_outside_canonical_horizon_is_not_silently_extrapolated[2036], which #619 fixes. The second is test_release_manifests.py::TestReleaseManifests::test__given_local_area_dataset__then_registered_with_sha_but_not_default: lines 444 and 446 still expect the bare populace_us_2024_acs_local.h5 at the release tag. The manifest is right; the two asserts need US_LOCAL_AREA_PATH and US_LOCAL_AREA_REVISION, which the PR already defines.

2. #619. It's one commit, 95d7d59, and it applies to this branch without conflicts. Neither PR passes alone: on main, #619's 2036 household case fails, and here the 2036 refusal case fails. So rather than merge them back to back, I'd cherry-pick 95d7d59 onto this branch and close #619 as included. I can push it if you're OK with me adding to your branch.

The certification reproduces. Yesterday I ran the claim-independent half on Modal from main 5b6d790: set-spm with the same forecast, then the 2.2.2 pin and uv lock on uv 0.13.4. Your pyproject.toml and uv.lock are byte-identical to mine, and so are measurements.spm and the spm-calculator package entry (wheel c528ec0f…). Both folder addresses, their commits and the two H5 hashes (6496cc43…, 3ce781c4…) match the folders I checked on the hub.

State and district runs work on this bundle. I ran them on Modal at 39f3eb0, installed with uv sync --locked --extra models (policyengine-us 2.2.2, spm-calculator 1.1.0). populace_us_2024_acs_local resolves to the new folder URI at 28f355b8, the same URI state/vt and congressional_district/NH-01 carry. ensure_datasets downloaded and verified the file and cut the 2026 and 2036 year files, and each scoped Simulation calculated:

Run	People	Net income	SPM poverty
state/vt 2026	653,576	$31.5B	8.3%
state/vt 2036	677,138	$45.5B	8.6%
congressional_district/NH-01 2026	701,134	$41.0B	6.0%
congressional_district/NH-01 2036	726,410	$58.8B	6.9%

Each run kept only its own area (district geoids 5000 and 3301), and every SPM receipt names forecast 335e6755dc8e…, ce_trend, at county geography.

The US extension reads right.

• For the UK, the new alias accounting is the old one: a UK move's source is a variable the target defines, so subtracting mapped changes nothing.
• For the US, a rename counts only where the live input isn't stored, as .py's loader applies it, and a legacy column stored beside its live input is classified like any other column.
• Loading legacy_inputs.py from its file is safe, since it imports only numpy and pandas.
• The address move doesn't break the legacy SPM entry for the ACS-local release, because legacy_spm_source keys on the SHA-256, not the URI.

Not blocking: the manifest grows from 18 KB to 657 KB and the US TRO from 6 KB to 706 KB, mostly the target and build surfaces stored in full, which are identical here. The UK route's evidence schema works the same way, so that's fine for this PR. Storing identical surfaces once could be a follow-up.

I'll approve once the test fix and #619 are on the branch.

[anth-volk]

@DTrim99 Thanks for the review. Both changes are on the branch now:

1. The test fix (bc5049f9). test__given_local_area_dataset__then_registered_with_sha_but_not_default now asserts US_LOCAL_AREA_PATH at US_LOCAL_AREA_REVISION. I also renamed the neighbouring test__given_local_area_name__then_resolves_to_its_immutable_tag to …_then_resolves_to_its_build_folder, since the dataset no longer resolves to a tag.
2. Document and test the 2022–2036 SPM forecast horizon #619 (658a452e). This is your 95d7d59, cherry-picked unchanged, with you as author. Document and test the 2022–2036 SPM forecast horizon #619 can close as included here, and the PR description now says so.

At 658a452e, these pass when run by node ID:

• both local-area manifest tests;
• all six cases of Document and test the 2022–2036 SPM forecast horizon #619's two changed tests: households calculate for 2024, 2026, 2035 and 2036, and 2021 and 2037 are refused.

#750 is still a draft.