Skip to content

Impute US humanitarian immigration statuses to cited stocks (H.R.1 channels) - #779

Open
MaxGhenis wants to merge 65 commits into
mainfrom
us-humanitarian-immigration-statuses-767
Open

MaxGhenis wants to merge 65 commits into
mainfrom
us-humanitarian-immigration-statuses-767

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Aug 25, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #767.

Implements the design proposed in the issue thread, plus one follow-on redesign of the spill controls (Aug 29, "Align immigration controls with Pew methodology") that this description now reflects. The US immigration_status source stage draws humanitarian/temporary-protection statuses to externally cited stocks, so the enacted H.R.1 eligibility channels stop computing ≈$0 on the next build.

What changes

Humanitarian draws (us_runtime/immigration.py). Between the ASEC-UA legal-status indicators and the EAD spill, thirteen seeded draws run — sequential, mutually exclusive, in decreasing order of target precision:

  • PAROLED_ONE_YEAR per program origin: OAW Afghans (73,566), U4U Ukrainians (158,000), CHNV Nicaraguans (93,070) and Venezuelans (117,330), with program-specific arrival windows (U4U/CHNV cannot draw from the 2020–21 bin);
  • REFUGEE (160k ≈ FY2023+FY2024 OHSS admissions) and ASYLEE (155k ≈ FY2022–24 OHSS grants as revised) from the indicator-documented pool only;
  • TPS per country from the CRS RS20844 table (Venezuela 605,015; El Salvador 170,125 with the 2001 continuous-residence cutoff binding at PEINUSYR granularity; Honduras/Nicaragua/Nepal/other-designated likewise);
  • DEPORTATION_WITHHELD carries an explicit cited zero (EOIR grants ~2.2k/year and publishes no stock).

Every draw excludes the Cuba/Haiti-born (the CUBAN_HAITIAN_ENTRANT class is the statutorily favorable label H.R.1 preserves — Haiti TPS rides it) and the DACA statutory cohort, and the parole draw carries Ukraine/Afghanistan TPS registrants. Residual-pool draws (parole/TPS only) flip ssn_card_type NONE → NON_CITIZEN_VALID_EAD, preserving the NONE ⇔ UNDOCUMENTED gate invariant.

Spill controls re-founded on Pew's universe. Pew's 2025 report supersedes the earlier series with a coherent 2023 pair — 14.0M unauthorized immigrants, 9.7M in the labor force — whose modeled universe deliberately retains DACA, parolees, TPS holders and similar temporary protections (and excludes refugees and granted asylees). The worker/student EAD spills now bind that broad universe rather than the bare residual pool: residual undocumented rows, DACA and residual-cohort Cuban/Haitian rows that keep protected engine labels after an EAD, and the Pew-included humanitarian marks. The worker margin uses measured current labor-force status (A_LFSR, age 16+) restored from the pinned Census ASEC archives by exact PERIDNUM join (checkpoint schema v4) instead of prior-year earnings. The composition gate anchors the same broad universe against 14.0M — identifying residual-derived entrants by the (CUBAN_HAITIAN_ENTRANT, NON_CITIZEN_VALID_EAD) pair, which is exactly the set the spill counts, since indicator-documented entrants carry OTHER_NON_CITIZEN.

Manifest, spec, attestations. Every target lives in source_stages.json with one citation per number (URLs verified live, the DHS OAW/U4U reports by exact byte match); the sources schema gains the required humanitarian_status_stocks block; thirteen _stable_site declarations join the seed protocol with owner-ledger bindings; every closed-world census the fail-closed machinery keeps (site counts, owner links, field universe, pointer claims, inventory digests, envelope digests, coverage evidence) is re-pinned on the merged tree.

Gates & probes. The composition gate bands each category's emitted mass against its cited target (explicit-zero categories must emit zero) and records achieved-vs-target for certification notes. Four release-blocking reform-coverage probes pin the channels end-to-end — restoring the pre-H.R.1 Medicaid list (§71109, 2027), the below-FPL PTC exception (§71302, 2026), the ACA lawful-presence list (§71301, 2027), and the pre-§10108 SNAP list (2026) must each move their measure on the export, binding only through immigration_status_str. The Medicaid probe counts the people it enrolls (medicaid_enrolled, at least 10,000); the other three score dollars.

Evidence

The stage receipts and the script that writes them live in docs/evidence/us-immigration-statuses-767/. The script runs the immigration stage alone on main's default pool, the 2024, 2025 and 2026 ASEC, which d2603 gives the next certified build. It runs at this PR's 4f94e489a and at main 764ae3aa7, at seed 0 and period 2024, then runs each tree's own composition gate. Later commits change only that folder. Weighted persons, in millions:

Status Main This PR Cited target
Citizen 300.244 300.244
Lawful permanent resident 11.680 10.427
Undocumented 13.071 12.891
DACA 0.376 0.277
Cuban/Haitian entrant 0.581 0.539
Paroled, one year 0 0.398 0.442
Refugee 0 0.160 0.160
Asylee 0 0.155 0.155
TPS 0 0.860 0.858
All 325.952 325.952
  • Each tree's composition gate passes with no failures. This PR's gate also bands each humanitarian draw, and it anchors Pew's broad universe at 14.0 million where main's anchors the undocumented rows at 11.0 million.
  • Ten of the twelve positive-target draws emit within 3% of their targets. Nicaraguan TPS emits 3,193 against 2,910. U4U parole exhausts its candidates, Ukraine-born non-citizens in arrival codes 28 and 29, at 112,934 against 158,000. The gate's band runs from 0.35 to 1.5 of each target.
  • Pew's broad unauthorized universe totals 14.574 million against its 14.0 million anchor, and its labor force 9.699 million against the 9.7 million control.
  • ssn_card_type is NONE exactly on the undocumented rows in both trees.
  • Before the tenth sync the stage refused this pool (receipt.pre_sync_42bb166b.json); see "The 2026 ASEC's arrival code 29" below.

The two trees label different rows, for two reasons. A fourth receipt, receipt.main_full_keys.json, runs main's method on this PR's draws and so separates them.

  • The draw key. Main keys each draw on the income year and PERIDNUM; this PR adds the household's PH_SEQ. In these files the old key already tells all 421,119 rows apart, but the new key changes every hashed string and so every row's draw, and the EAD spill takes candidates in draw order. On this PR's key, main's own method moves 2.400 million people from undocumented to LPR and 2.392 million back, and 0.220 million from undocumented to DACA and 0.244 million back.
  • The method. This PR draws the humanitarian statuses before the spill, counts a worker by A_LFSR at age 16 and over instead of prior-year earnings (WSAL_VAL, SEMP_VAL), and spills until Pew's broad universe matches the controls instead of the undocumented rows left in the residual pool. With both runs on one key, the method moves 0.319 million people from undocumented to LPR and 0.706 million back, and 0.031 million from undocumented to DACA and 0.105 million back.

The two effects don't add, since both reshuffle one pool. Together they move 2.021 million of main's undocumented rows to LPR and 2.415 million the other way, and 0.172 million of main's undocumented rows to DACA and 0.271 million of its DACA rows to undocumented. The README has a row-by-row table for each comparison.

The checks tab is authoritative for CI. Locally, the four read-only pin checks pass after the tenth and eleventh merges (spec 49d7b804), after the first 2026-11-10 review fixes (spec c414c985) and after the second (spec 783ff9a4): coverage 42,781 of 42,781 configuration fields and 41 of 41 inventory checks, bundle, spec contracts and pool contracts.

Independent review round (2026-09-19)

A second-reader audit of head 9622beb5 requested three changes; all are on the current head.

  • Spill order. _spill_pew_unauthorized_excess entered the other controlled margin before exhausting rows outside it, so the student control could remove a worker the worker pass cannot restore (reproduced in 4 of 6 seeds). Each tier is now exhausted before the overlapping one; test_student_spill_exhausts_nonworkers_before_touching_a_worker pins it.
  • Attested schema-9 pools. The branch imposed the new contract on historical pools: it rebuilt the legacy schedule and v11 authority from the current registry (payload 2f61b55e… instead of the sealed 02e618cc…), and it required the 12-step operator order and immigration reconciliation evidence of pools that predate them. History is now data: a frozen us/legacy_schema16_late_producer_schedule.json verified against its sealed hash, a literal 11-step order, and ordinary immigration transfer receipts — all only under legacy_worker_authentication. The legacy digests now equal main's byte for byte (test_us_legacy_schema9_contract.py), the positive fixture is genuinely historical, and two layers refuse a version-downgraded current artifact. (Main has since moved its legacy path; the tenth sync narrows the frozen schedule's role to US: humanitarian immigration statuses absent from the imputation — H.R.1 §71109/§71302 eligibility channels compute ≈$0 #767's two producers, below.)
  • Docstring. The module's stated draw order now matches the implemented precedence (parole first).

Second review round and invariant tests (2026-09-19 to 2026-09-29)

  • Historical execution rows. The round-2 review found that legacy scoring still checked every execution row against today's producer registry, which refused a real schema-9 pool: its immigration source row declares 57 inputs against today's 56, and its transfer row 93 against 100. legacy_us_late_producer_contracts() now rebuilds the schema-16 contracts from the frozen schedule's own input inventories. It refuses unless the rebuild reproduces both the frozen schedule_sha256 and the sealed payload hash. (Main's later contract changes broke that rebuild; the tenth sync replaces it, below.) The positive legacy fixture now seals those rows with the production readiness and evidence builders under the historical contracts. Two negative tests pin both directions: neither a legacy pool that carries today's surfaces nor a current pool that carries historical surfaces passes. A control shows the re-seal itself is faithful.
  • Two defects the invariant tests found, both fixed.
    • us_immigration_composition_gate re-derived the DACA cohort at a hard-coded 2024. It now takes the build's time_period as a required argument.
    • The Pew spill compared its excess to zero exactly, so about 1e-15 of summation residue could start the next tier and spill a row inside the other controlled margin. Closure now allows a billionth of the in-scope weight.
  • Syncs. The branch has merged main sixteen times, including main's test-tier reorganization. The tenth sync kept main's values for two stacked-authority test pins, which the 2026-11-10 review found (below).

Tenth main sync (2026-11-09)

This sync merges main 1acd884fe. It ports main's ACS-local immigration lane onto this PR's kernel, keeps main's design for the attested schema-9 legacy path while isolating #767's two producers on it, fixes the stage for the 2026 ASEC's arrival code 29, moves the ACS-transfer contract pins, and settles a conflict git merged silently in the ACS loader's column lists.

ACS-local immigration on this kernel

Main's ACS-local immigration lane (#1052, for #1020) runs this module's kernel on ACS rows. It aims each national control at the residual the donor spine leaves, measured at pooled weights. Taking either side's version would have broken the other, so this merge ports main's lane onto the #767 kernel:

  • Delivered counts use the Pew universe. The donor's delivered worker and student mass counts _pew_unauthorized_rows, the universe the kernel's spill binds, instead of bare UNDOCUMENTED rows. A donor DACA, TPS or parole worker now counts toward the 9.7M control, as it does nationally.
  • Each humanitarian draw targets its own donor residual. For each of the thirteen draws, the ACS rows get max(0, national target − donor-emitted mass). The kernel's own emission mask (_humanitarian_emitted_mask) measures the donor side. The receipt records target, donor-delivered, residual, ACS-assigned, pooled total and excess for each draw. acs_local_immigration_receipt_failures refuses a receipt whose draw set differs from the packaged draws, or whose residual or pooled identities fail. The method string moves to national_control_minus_donor_pooled_delivered_v2, so the release tool refuses a staging run from before this change.
  • The ACS side measures labor force. ACS ESR maps onto the CPS A_LFSR codebook (1→1, 2→2, 3→3, Armed Forces 4 and 5→0, 6→7, blank→0; the mapping refuses any other code). The kernel's worker test (age 16+, A_LFSR 1–4) then reads ACS rows directly.
  • The donor side falls back to earnings, and this PR says so. _donor_status_masks uses A_LFSR whenever the donor carries it. In production the donor doesn't: tools/build_us_puf_support_base.py gives the immigration stage A_LFSR as an ephemeral input and drops it from its output. So the donor's delivered worker mass keeps main's earnings-pair test, and the receipt names that test in donor_status_columns.worker. ACS-local donor worker count uses the earnings proxy because the support base drops A_LFSR #1554 tracks restoring A_LFSR on donor rows.
  • The signal gate grades the build's period. acs_local_immigration_signal_gate takes a required time_period (the composition gate has required it since the round-2 fix) and optional controls. Both callers pass their period: the release tool and the legacy multispine builder.
  • CI fixes from the old head. The engine-free post-export scoring counts now include this PR's probes: 47 probes, 23 baseline keys, 113 requests. All four probes are new. They add three baseline keys, because the SNAP probe shares its snap 2026 key with the Texas vehicle-exemption probe. The engine-US drift test now accepts the list-valued restored-eligibility probes.

Property tests (Hypothesis, in the same module) cover the port's invariants:

  • the ESR mapping is total on the codebook and refuses anything else, infinities included;
  • mapped codes pass the kernel's worker test exactly for the civilian labor force (ESR 1-3);
  • every receipt the producers can write passes the checker;
  • each draw's residual is max(0, target - donor) and the emitted ACS rows weigh exactly acs_assigned;
  • a covered stock assigns nothing, and the draw reaches a feasible residual with less than one row of overshoot;
  • a larger stock selects a superset of a smaller one's rows.

A mutant that draws the ACS rows toward the full national stock fails the residual property at once.

The attested schema-9 legacy path

Since the last sync, main's #1239 and #1258 changed the primary-PUF S-corporation outputs and moved the PUF itemization transfer targets between batches. Main rebuilds its legacy schedule and v11 authority from today's registry and surfaces and relabels them at schema 16, so at main 1acd884fe they hash to 2868139b… and 3860e1c3…, not the sealed 02e618cc… and e660a8ce…. This branch had rebuilt every legacy contract from the frozen schema-16 inventories and refused unless the rebuild reproduced the sealed hashes, so on the merged tree it refused every legacy load. The merge keeps main's design and isolates only what #767 changed:

  • the legacy contracts take the immigration source stage's and its transfer group's inventories from the frozen schedule, and each of those two contracts must hash to the digest the schema-16 registry derived (measured at 7c4a5ac0, whose registry reproduces the sealed schedule);
  • the legacy receipt carries the frozen schedule's schema version and execution receipt contract;
  • every other legacy contract is today's.

On the merged tree the legacy schedule payload, the legacy authority and all 38 legacy contracts equal main's exactly. The parameterized payload builder leaves the live schedule payload byte-identical (3f1daff0…). test_us_legacy_schema9_contract.py pins this in 16 tests, including a Hypothesis round trip of the inventory payloads. Two #767 changes still reach the legacy path, through two bindings it takes from today's code, as main's does. Late-transfer rows bind today's acs_transfer_execution_contract_identity, which main's own post-transfer changes since 7c4a5ac0 already moved for all nineteen groups. Post-clone source rows bind today's source-stage binding, which hashes the whole packaged us/source_stages.json and carries each bound stage's resolved spec. #767 rewrote the immigration_status stage, so its change reaches the fifteen source rows that bind a stage spec (direct housing assistance binds none), and main has changed that file in fourteen commits since 7c4a5ac0. Restoring main's whole legacy path to sealed history is separate work.

The 2026 ASEC's arrival code 29

On 2026-09-27 (e518fb8), main pinned the 2026 ASEC and set the default US pool to the newest three income years (asec_sources.ASEC_DEFAULT_POOL_INCOME_YEARS: the 2024, 2025 and 2026 ASEC). An earlier sync brought that onto this branch. Census re-bins the newest PEINUSYR codes each survey year. Code 27 is 2020-2021 in all three files. Code 28 is 2022-2024, 2022-2025 and 2022-2023 in turn, and the 2026 ASEC adds code 29 for 2024-2026. Foreign-born infants (age 0-1) sit only in the newest code of each file (28, 28 and 29). This branch had removed main's 29: 2024 midpoint and refused code 29 (3c3bdbf, Aug 28), which held while the stage ran on the 2024 ASEC alone. Since that sync, the stage has refused the default pool outright: PEINUSYR carries value(s) outside the 2024 ASEC domain 0..28: [29.0] (docs/evidence/us-immigration-statuses-767/receipt.pre_sync_42bb166b.json). The single-file receipts never saw it. Main's ACS-local _asec_entry_year also refuses any foreign-born donor whose code is missing from that table, so on the merged tree it refused every code-29 donor. The fix:

  • restores main's 29: 2024 and widens the domain check to 0..29;
  • names each program's codes:
    • Operation Allies Welcome (2021-2022 arrivals) draws from 27 and 28.
    • U4U, CHNV and FY2023-FY2024 refugees draw from 28 and 29 (_POST_2021_ARRIVAL_CODES), never 27.
    • Asylum grants (FY2022-FY2024) and TPS leave 29 out. Its arrivals come after all but the last months of those grants, and no TPS residence date falls after 2024.
  • adds tests:
    • test_2026_asec_peinusyr_29_is_in_the_domain, test_peinusyr_past_the_2026_asec_domain_raises and test_2026_asec_code_29_draws_only_programs_running_in_2024;
    • the property strategies now draw codes up to 29, with new window properties for refugees and for asylee and TPS rows;
    • a code-29 donor in the ACS-local entry-clock test.

ACS-transfer contract pins

Main's #1056 review pinned the shared ACS-transfer execution contract with no person predictor extensions. #767 moves that contract to schema 3, adding its immigration predictors, the baseline-excluded statuses and the humanitarian post-transfer block to every group, so the three pins move: default 4403aefd…, declared plan ce27d251…, pregnancy 061d8f10….

ACS loader column lists

Git merged two edits to the ACS loader's column lists without a conflict. This PR had moved CIT, POBP and YOEP into _PERSON_REQUIRED, and main's #1020 added the same three columns to _PERSON_OPTIONAL. The merge keeps main's design: the loader leaves an absent column absent, and the immigration consumers refuse a row with no citizenship source. Two new tests in test_us_acs_pums.py check that refusal through the kernel's evidence profile and through us_immigration_evidence_features, which the ACS transfer reads. The late-producer scope test names the three columns itself, and the transport battery's binding closure adds us_runtime.acs_pums, whose ACS_2024_1YR_VINTAGE the transfer imports.

Eleventh main sync (2026-11-09)

This sync merges main 764ae3aa7, 98 commits past 1acd884fe. One file conflicted: the transport battery's binding closure list (BINDING_MODULES_ADDED_AFTER_BASE in test_transport_evidence_kernels.py), where main's #1405 added the microcosm.data modules and the tenth sync added us_runtime.acs_pums. The list keeps both. Git merged main's SCF wealth gate record (#1216) beside this PR's humanitarian target materializer in tools/build_us_fiscal_refresh_release.py and its tests without a conflict, and those tests pass. The four pin checks pass unchanged. The stage receipts above run at 4f94e489a, four commits past this merge, and at 764ae3aa7.

Local runs (pytest -n 3; the checks tab is authoritative for the full tiers):

  • At the eleventh merge 4b3b6a7a6: the immigration, immigration pool-scaling, immigration property, ACS-local immigration and legacy schema-9 contract files (270 passed), and the transport evidence-kernel, fiscal-refresh builder and plan files (569 passed, 2 skipped).
  • At the tenth merge f5491c5f9: the ACS loader, ACS-local hours, income, receipt-anchor and work-disability, ACS multispine, ACS multispine legacy builder, ACS release-predictor, ACS transfer and late-producer files (452 passed).
  • ruff check . passes, and the four pin checks pass after each merge.

Review fixes (2026-11-10)

An independent review of bad5cb667 (a native Opus 5.5 workflow) found one blocking defect, one major and three minor. The current head answers all five:

  • Blocking: two stale pins. The tenth merge kept main's values for the stacked-authority receipt and late-producer schedule component pins in test_spec_engine_stacked_authority_semantics.py. Engine-US failed in a contract batch, which stops at its first failure (--maxfail=1); the runner then admits no new task and skips every workflow file, so most of the tier never ran. Both pins now hold the merged tree's live values (7cdedd5b…, bdc3ab40…).
  • Major: the receipts misreported main's draw key. Every receipt named this PR's key, although main keys its draws on the income year and PERIDNUM alone, and the README credited the method with every row the two trees label differently. The script now records each draw call's key and takes --draw-keys full|legacy. The receipts are new, and a fourth runs main's method on this PR's key: the key alone moves about as many rows as the full comparison (Evidence, above).
  • Minor: a second legacy-path reach. Post-clone source rows on the legacy path bind today's whole-file source_stages.json hash, which US: humanitarian immigration statuses absent from the imputation — H.R.1 §71109/§71302 eligibility channels compute ≈$0 #767 changed. docs/us-multispine-operator-ordering.md and the legacy-path paragraph above now name it.
  • Minor, kept at the time with a reason: the Operation Allies Welcome window had no end year on exact-year (ACS YOEP) rows. A kernel comment above _PAROLE_ACS_MIN_ARRIVAL_YEAR said the stacked transfer asks the ACS arm for its own mass share of each national target and refuses a candidate shortfall, so the window also took 2023-2024 arrivals. The comment moved the attested kernel identities (US spec 49d7b804 → c414c985). The second review found that the comment misstated the transfer's rule, and the window now ends in 2022 (below).
  • Minor: the probe count. All four channel probes are new (tenth sync, above).

Local runs at e3b2c8ce9 (the checks tab is authoritative):

  • the files the fixes touch, at pytest -n 3 (598 passed);
  • the whole engine-US tier as CI groups it, without --maxfail: the 67 contract and scenario files in one pytest -n 3 --dist loadfile session (485 passed, 6 skipped), then each of the 22 workflow files in its own process: 20 passed, test_us_puf_support_base_equivalence.py skipped both its tests, and test_us_post_export_scoring.py failed on the Medicaid probe, the second review's blocking finding (below);
  • tools/repin_us_spec_engine_contracts.py --check, tools/repin_us_pool_engine_contracts.py --check, coverage --check and bundle --check all exit 0.

Second review fixes (2026-11-10)

A native Opus 5.5 delta review of e3b2c8ce9 and 9d164367f (workflow wf_5bb29ab5-2eb, one finder and one skeptic per dimension) left six findings standing, which name four defects: the two blocking findings name one, and four minor findings name the other three. 4f94e489a and 2f70d5c3c answer all four, and the work on one of them found a fifth defect, in the stacked transfer.

  • Blocking: the post-export scorer could not score the Medicaid probe. PolicyEngine-US prices each Medicaid enrollee through medicaid_slcsp_state_denominator, an aggregate over the whole file. The household-batched scorer (microcosm#956) refuses it, and a reform engine refuses it even in one batch; main's own guard test pins both refusals. Engine-US failed at test_shipped_baseline_plans_match_whole_file_policyengine_us, and a release build would have aborted at the reform-coverage smoke. The probe now counts weighted people in medicaid_enrolled, with a 10,000-person floor. Probes carry a measure_unit (usd by default, or persons), the smoke gate states effects and floors in it, and the manifest check refuses a probe whose budget measure is not a live variable of that unit: a currency-USD variable, or a person-level boolean. On the default-pool stage receipt, 0.391 million of the 0.714 million people in the restored statuses report Medicaid at interview (receipt.head_medicaid_reporters.json), so the floor is 2.6% of the reporters.
  • Minor: the receipt script said main's tree has no composition gate. Main has one, without the humanitarian bands or Pew's universe. The script now runs each tree's own gate, so main's two receipts carry gate results, and both pass.
  • Minor, twice: the OAW kernel comment misstated the transfer's rule. The reconciliation asks the ACS rows for each draw's target less the ASEC rows' labelled mass, not for the ACS arm's mass share. The two differ whenever the ASEC arm misses its target, as U4U's does.
  • Minor: the changelog fragment still said attested schema-9 pools authenticate against the contract they sealed. It now says what the legacy path does.

The stacked transfer refused U4U in every stacked build. The --exact-k arm, the only one that runs the improvement gate, takes a stacked pool. Checking the OAW finding against the ACS 2024 1-year showed the U4U draw short of candidates under the real rule. The ASEC arm carries 56,467 of its 158,000 target, the reconciliation asks the ACS rows for 101,533, and at the ACS arm's weights (scale 0.478) the Ukraine-born non-citizens who arrived from 2022 on hold about 57,046. The old reconciliation refused any such shortfall, so every stacked build would have stopped at U4U, the second draw in manifest order. The ASEC stage already handles the same shortfall: it takes every candidate (U4U ends at 0.715 of its target there), and the composition gate's band, 0.35 to 1.5, decides. The reconciliation now follows that rule:

  • a draw whose candidates hold less weight than its residual takes every candidate, and its receipt records candidates_exhausted and candidate_shortfall_population;
  • the reconciliation refuses a draw only when it then ends below 0.35 of its target, the band's floor;
  • the stacked validator replays the rule and refuses a receipt that hides a shortfall, claims one its candidates cover, leaves a candidate unselected or ends below the floor;
  • the transfer contract's candidate_shortfall field now reads take_every_candidate_error_below_composition_band_floor, which moves the three transfer-contract identities.

With that rule, the OAW window ends where the program does: exact-year rows take Afghan arrivals from 2021 and 2022 only. ASEC rows keep their coarser codes.

scripts/stacked_supply.py projects each draw's ACS supply from the PUMS without building a pool (README, "The stacked pool's ACS residuals"). Five draws run short. U4U, Nicaraguan parole, Honduran TPS and OAW run short even if no earlier draw takes their candidates, and end at 0.718 to 0.971 of their targets; no earlier draw that takes any rows shares a candidate with them. TPS Venezuela's supply exceeds its residual by 38,923, but Venezuelan parole, the fourth draw, takes an expected 49,300 of its candidates, so TPS Venezuela ends at an expected 0.983 of its target before refugee and asylee draws take any of them. In the worst case, where every earlier draw takes as much of a later draw's candidates as its own residual allows, no draw ends below half its target: TPS Venezuela ends at 0.710, and the lowest, refugee and asylee, at 0.505. The ASEC arm alone carries 0.357 of U4U's target, above the floor even with no ACS rows.

A third delta review, native Opus 5.5 workflow wf_06fd9b1f-5b7 at 5610bde70 (it continued two runs that a usage limit stopped), found no code defect at 4f94e489a and three prose defects, which a5a1dda0b and this body answer. Honduran TPS's candidates all fall inside the zero-target deportation-withheld draw's, so the four short draws share no candidate with an earlier draw that takes rows, rather than with none. TPS Venezuela runs short in every stacked build on these inputs, not just in the worst case, so stacked_supply.py now projects an expected share for each parole and TPS draw. And the eleventh-sync section still named the receipts' old commit.

The new rule is this PR's call, not main's: main has no humanitarian draws, and the reconciliation, its shortfall refusal and the band all arrive here. Apart from the OAW window, a draw that does not exhaust its candidates runs the old statements and selects the same rows, and its receipt gains only the two new fields. The property tests check the old bounds on those draws, that an exhausted draw takes exactly the candidates no earlier draw took, that a draw left below the floor refuses by name, and that the replay agrees with the reconciliation. The kernel edit moves the attested identities: seed protocol and map digests, the US spec (c414c985 → 783ff9a4), the coverage evidence and the minimal-loader golden.

Local runs at 4f94e489a (the checks tab is authoritative):

  • the 19 engine-free files whose pins or behaviour the commit moves, at pytest -n 3 (1,021 passed);
  • tools/repin_us_spec_engine_contracts.py --check, tools/repin_us_pool_engine_contracts.py --check, coverage --check (42,781 of 42,781 configuration fields, 41 of 41 inventory checks) and bundle --check (spec 783ff9a4) all exit 0;
  • the engine-US tier as CI groups it, without --maxfail: the 67 contract and scenario files in one pytest -n 3 --dist loadfile session (489 passed, 6 skipped), then 11 of the 22 workflow files, each in its own process, before a session restart stopped the run. Ten passed, among them test_us_post_export_scoring.py, which failed at e3b2c8ce9, and test_us_puf_support_base_equivalence.py skipped both its tests. CI's engine-US jobs ran the whole tier at 2f70d5c3c and pass.

Release-build rule

docs/us-release-build-rule.md asks a pull request "large enough to change how the dataset is built" to carry a from-scratch build and certification receipt from its own tree before it merges. This PR changes how the build draws immigration statuses: the ASEC source stage, the transfer of those statuses onto ACS-transferred rows, and the ACS-local lane. In place of a from-scratch run it carries stage receipts on the inputs the next certified build uses, the first on main's default pool (Max's ruling d2603). The receipts cover the source stage (Evidence, above). The transfer and the ACS-local lane run only in a full build, where the property tests below and the release gates check them: the release tool refuses a base whose composition gate fails and grades that gate again at the delivered weights, and the four channel probes must each move their measure on the export. For the stacked transfer, the evidence folder also projects each draw's ACS supply against its residual ("The stacked pool's ACS residuals" in its README). Whether the receipts may stand in for the from-scratch run is Max's call. d2729 asks it on the terms of d2647 for #880: the next certified US build on main becomes this PR's from-scratch run.

Invariants (property-tested)

engine_free/us/test_us_immigration_properties.py holds 30 Hypothesis properties. Each one quotes the code comment or docstring that promises it. Every property below holds against the current code. The two example regression tests fail with their fix reverted.

  • _select_weight_to_target:
    • the selection is a draw-threshold subset of the candidates and never splits a tie group;
    • it reaches the amount and overshoots by less than its last tie group, or takes every candidate;
    • it matches an independent group walk (differential).
  • _stable_person_draws: draws lie in [0, 1), key only on the most complete source lineage, and reproduce exactly.
  • _spill_pew_unauthorized_excess:
    • it moves only in-scope residual non-citizens, and only to EAD;
    • the Pew universe never grows;
    • it closes every closable excess and otherwise exhausts the pool;
    • it undershoots by less than one spilled row;
    • it leaves a margin already within tolerance untouched;
    • rows inside the other margin move only after every outside row, and rounding residue never opens that tier.
  • Stage output:
    • ssn_card_type NONE ⇔ status UNDOCUMENTED;
    • status is the documented precedence function of citizenship, SSN, marks and cohort evidence;
    • humanitarian rows are non-citizens, never Cuba/Haiti-born or in the DACA cohort, and sit inside the ASEC windows the comments state;
    • draws are sequential and mutually exclusive, and each reaches its target or exhausts its pool;
    • output is bit-reproducible, and support clones share labels;
    • checked at periods 2024, 2025 and 2030.
  • Differentials:
    • the frame entry point matches the manifest handler and is idempotent;
    • stage output meets the composition gate's structural contract at the stage's own period;
    • the spill's projection mask equals the gate's Pew universe;
    • the transfer reconciliation satisfies the gate, its receipts obey their accounting identities, and its selection replay reproduces the reconciled rows;
    • a draw whose candidates hold less than its residual takes every candidate no earlier draw took, reaches its immutable mass plus theirs, and records the shortfall; one that ends below the band floor refuses.

engine_free/us/test_us_acs_local_immigration.py adds six Hypothesis properties for the ACS-local port, listed in the tenth-sync section above.

Independence limits, stated in the test module: the humanitarian candidate pools and some receipt identities are only checked against the module's own helpers or guards. The property tests do not independently establish them.

axiom: n/a: microsimulation data imputation (immigration-status inputs to existing policyengine-us eligibility rules), no policy encoding.

Deliberately out of scope (filed)

This PR touches no certified release; the imputation activates in the next build lane, where the smoke gate enforces the four channels permanently.

🤖 Generated with Claude Code

MaxGhenis and others added 16 commits August 25, 2026 15:36
REFUGEE / ASYLEE / PAROLED_ONE_YEAR (per program origin) / TPS (per
country) are now selected between the ASEC-UA indicators and the EAD
spill, each to a manifest-cited target, so the H.R.1 $71109/$71301/
$71302 and SNAP $10108 eligibility channels stop computing $0 on the
release. DEPORTATION_WITHHELD carries an explicit cited zero (no
published stock). Residual-pool parole/TPS draws receive EAD SSN cards,
keeping the NONE-undocumented invariant; the composition gate now bands
each category against its target.

Refs #767

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Thirteen stable blake2b sites (per-origin parole, refugee, asylee,
deportation-withheld, per-country TPS) join the seed protocol; the
seed-owner ledger binds them to the immigration_status stage; the
sources schema gains the required humanitarian_status_stocks block; the
bundle regenerates with the new source_stages attestation.

Refs #767

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…h tests

Refs #767

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restorations of the pre-H.R.1 Medicaid, SNAP, and ACA lawful-presence
lists and the below-FPL PTC exception must each move their budget
measure on the export; a ~$0 score means the humanitarian statuses
regressed and blocks publication.

Refs #767

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n digests

test_us_hr1_immigration_channels builds each probe's reform against the
locked engine and shows nonzero, sign-correct movement per channel
(medicaid, aca_ptc twice, snap) so the release smoke gate's probes have
a PR-CI early warning. The thirteen humanitarian draw sites join
EXPECTED_LEGACY_V1_SITES and the audited-source manifest; BE/UK envelope
digests re-pin because the shared seed-protocol source is attested.

Refs #767

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refs #767

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ration-statuses-767

# Conflicts:
#	packages/microcosm-build/tests/test_spec_engine_country_bundles.py
Main's #743 moved the UK bundle while this branch moves the shared seed
protocol; the merged UK digest combines both, BE carries only the
protocol move.

Refs #767

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… draws

The thirteen humanitarian sites move every closed-world census the
fail-closed machinery keeps: protocol site counts (53 -> 66), the
owner-link sum (112 -> 125, one owner each, distinct owners unchanged),
the configuration field universe (42,120 -> 42,419: +13 legacy-behavior
target fields, +286 attestation-only resolved seed fields), four pointer
claims (source_stages, spine/resolved seed-site bindings, resolved seed
protocol), and the source_stages byte attestation in the bundle tests.

Refs #767

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ration-statuses-767

# Conflicts:
#	packages/microcosm-build/tests/test_spec_engine_country_bundles.py
…ration-statuses-767

# Conflicts:
#	packages/microcosm-build/tests/test_spec_engine_country_bundles.py
Add source-aware ASEC and ACS immigration evidence, pooled control scaling, deterministic humanitarian reconciliation, calibration targets, terminal gates, authenticated receipts, and release regressions.

Refs #767
@MaxGhenis
MaxGhenis marked this pull request as ready for review August 28, 2026 21:18
@MaxGhenis
MaxGhenis marked this pull request as draft August 28, 2026 23:36
@MaxGhenis
MaxGhenis marked this pull request as ready for review August 29, 2026 01:39
@MaxGhenis
MaxGhenis marked this pull request as draft August 29, 2026 07:49
MaxGhenis and others added 2 commits September 4, 2026 08:12
…ration-statuses-767

# Conflicts:
#	docs/evidence/spec-engine/us-f0-coverage.json
#	packages/microcosm-build/src/microcosm/build/spec_engine/inventory_coverage.py
#	packages/microcosm-build/tests/test_spec_engine_country_bundles.py
#	packages/microcosm-build/tests/test_spec_engine_loader.py
#	packages/microcosm-build/tests/test_us_multispine_pool_h5_io.py
#	packages/microcosm-build/tests/test_us_multispine_pool_tool.py
Resolves the six pin collisions against current main: BE/UK envelope
digests, the loader golden vector, the US spec digest in the multispine
tool test, the nine inventory digests plus the producer-input count the
A_LFSR column adds, and the regenerated coverage evidence. The h5_io
test is a real three-way merge (branch kept its stacked-pool immigration
receipt tests; main's failed-gate fixture helper replaces the inlined
block). Re-applies the branch's seed-group entries and site counts on
main's inventory module.

Refs #767

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis and others added 3 commits September 4, 2026 08:28
The branch's scoring loader authenticates the canonical stacked terminal
gates, so main's minimal fixture gate is refused; the helper keeps its
API and adopts the branch's construction (real gates, completeness
flipped). The denied-pool test now derives the fixture's household count
from the positive canonical humanitarian draws instead of main's
three-row literal.

Refs #767

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main added the am country bundle this week; the envelope digests move
with the shared seed protocol, so all three pins are recomputed by
country rather than by position.

Refs #767

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ration-statuses-767

# Conflicts:
#	docs/evidence/spec-engine/us-f0-coverage.json
#	docs/us-multispine-operator-ordering.md
#	packages/microcosm-build/src/microcosm/build/spec_engine/field_usage.py
#	packages/microcosm-build/src/microcosm/build/spec_engine/inventory_coverage.py
#	packages/microcosm-build/src/microcosm/build/us/spec/sources.yaml
#	packages/microcosm-build/src/microcosm/build/us_runtime/h5_io.py
#	packages/microcosm-build/src/microcosm/build/us_runtime/stacked_spine.py
#	packages/microcosm-build/src/microcosm/build/us_runtime/us_late_producer_registry.py
#	packages/microcosm-build/tests/test_spec_engine_country_bundles.py
#	packages/microcosm-build/tests/test_spec_engine_coverage_tool.py
#	packages/microcosm-build/tests/test_spec_engine_field_usage.py
#	packages/microcosm-build/tests/test_spec_engine_legacy_adapter.py
#	packages/microcosm-build/tests/test_spec_engine_loader.py
#	packages/microcosm-build/tests/test_spec_engine_stacked_authority_semantics.py
#	packages/microcosm-build/tests/test_us_bundle_core_contracts.py
#	packages/microcosm-build/tests/test_us_multispine_pool_h5_io.py
#	packages/microcosm-build/tests/test_us_multispine_pool_tool.py
#	packages/microcosm-build/tests/test_us_spec_bundle.py
#	tools/build_us_multispine_pool.py
#	tools/generate_us_bundle_from_constants.py
#	tools/spec_engine_coverage.py
MaxGhenis and others added 3 commits September 22, 2026 17:44
Main's two new US package resources and this branch's frozen schedule
together move the configuration universe to 42,549 fields; the seed
protocol/map digests, pointer inventory, coverage evidence, loader golden
vector, and US spec sha mirror are recomputed on the merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sealed

Round-2 review: legacy scoring accepted the frozen schema-16 schedule but
still checked every execution row against today's registry, so a real
schema-9 pool's immigration source row (57 inputs) and transfer row (93)
were refused against today's 56 and 100. The registry builder now takes its
source/transfer inventories as arguments; legacy_us_late_producer_contracts
rebuilds the historical contracts from the frozen schedule's own inventories
and refuses unless that derivation reproduces both the frozen schedule_sha256
(which hashes every contract's full inputs and outputs) and the sealed
payload hash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ration-statuses-767

Sixth sync. Pin literals take the branch side and are recomputed next;
the fiscal-refresh tool and its test layer main's newer changes onto the
branch's humanitarian stock targets (materializer 13 -> 14 stays this
branch's bump); sources.yaml is regenerated from the merged stages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Sep 26, 2026
Native commit 071f563 ported #779's paired immigration transfer, which
requires exactly one row-level citizenship source (ASEC PRCITSHP or ACS CIT)
with its birth-country and entry-year partners on every donor and recipient
row. It did not port #779's ACS loader change (3c3bdbf) that carries
CIT/POBP/YOEP, so any multispine pool with ACS rows refused
(test_every_pool_transfer_family_accepts_its_produced_physical_dtype on
every lane: missing_rows=2, the two ACS-channel clones).

acs_pums now reads CIT/POBP/YOEP as optional person columns, like #765's
hours columns. Every ACS PUMS person file carries them. A spine loaded
without them stays loadable, and a paired immigration transfer onto it still
refuses; nothing is defaulted. The reviewed acs_pums.py fingerprint in
acs_native_coverage_binding is re-pinned with that review noted. The pool
fixture's ACS rows carry the evidence, as #779's did, and the transfer
gains its three evidence predictors (32 -> 35). A new loader test checks
carriage when supplied and absence otherwise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 9 commits September 27, 2026 15:42
…partial)

Observed on the merged tree: 42,567 configuration fields, 38 source stages,
and the source-stages, generated-authorities, predictor-tuple, resource
semantics, stacked checkpoint base identity, and seed digests. The ledger
mode counts, pointer inventory, coverage evidence, loader golden and spec
sha mirror follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ration-statuses-767

Seventh sync, across main's test-tier reorganization. Pin-only files are
main's versions with this branch's identities recomputed on the merged tree
(42,571 fields, spec 1e3125e9, source stages e2cda1e3, schedule 88bc9243,
66 seed sites / 125 owner bindings, 2,750 producer inputs); the frozen
schema-16 schedule resource and the 13 humanitarian seed sites are
re-declared. Branch test changes are ported onto main's engine_free/engine
split (helpers in test_support/microcosm_build), each port adversarially
verified; the three new test modules move into tiers, the H.R.1 channel
test to engine/us without its module-level engine guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ration-statuses-767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rged tests

The immigration-drift case now expects the SPM composition gate's
degraded-mode line main's cofailure contract appends on the fake export
frame; main's prepared-pool SPM case gets the immigration statuses every
post-#767 pool carries; the shipped reform-coverage baseline plans count
the three keys the four H.R.1 probes add (medicaid 2027, aca_ptc 2027,
snap 2026).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… within tolerance

Two defects the invariant tests surfaced (#767):

- us_immigration_composition_gate re-derived the source-aware cohort
  evidence at a hard-coded 2024 while the stage derives it at the build's
  time_period, so a non-2024 build would be gated against a different DACA
  cohort than it assigned. The gate now takes a required time_period, and
  both builders pass their own period (PERIOD, POOL_TIME_PERIOD).
- _spill_pew_unauthorized_excess compared the recomputed excess to zero
  exactly; a selection that closes the gap exactly can leave ~1e-15 of
  summation residue, which opened the next tier and spilled a row inside
  the other controlled margin — the ordering the round-1 fix promised to
  keep. Closure now allows a billionth of the in-scope weight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e for the immigration fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
28 Hypothesis properties over synthetic ASEC/ACS person tables (6-32 rows,
a third humanitarian-profile or DACA-cohort rows, periods 2024/2025/2030)
for every invariant immigration.py states: NONE <=> UNDOCUMENTED; mutually
exclusive humanitarian draws that never select an excluded cohort; the
weight-to-target selection contract; seed determinism and lineage keying;
the tiered spill (non-increasing Pew count, closure within tolerance,
never touching the other margin while an outside row remains); and a
differential between the spill's Pew projection and the composition
gate's, with the gate evaluated at the stage's own period. Two example
regression tests pin the period and closure-tolerance fixes of 5a89fed;
each fails with its fix reverted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Astra round 2 (second half): the positive schema-9 tests kept today's
declared-input surfaces, so nothing proved a real schema-9 pool's
historical rows authenticate. Every producer whose schema-16 contract
differs from today's now has its row sealed by the production readiness,
absence, input-evidence and output-evidence builders, run against the
historical contract on the fixture's own late-DAG input frame (with the
WSAL_VAL/SEMP_VAL inputs the schema-16 immigration source read). The
positive test asserts the historical surfaces (57 and 93 inputs, WSAL_VAL
and SEMP_VAL present, no A_LFSR); a legacy pool whose immigration source
row carries today's 56-input surface is refused, and a current pool whose
rows carry the historical surfaces is refused, with a control proving the
re-seal itself reproduces a loadable pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ration-statuses-767

Ninth sync, across main's engine test split (engine_contract / engine_scenario
/ engine_workflow). The H.R.1 channel test moves to engine_scenario/us (it runs
small household engine calculations); the loader golden follows main into
engine_free/shared. Pin hunks resolve to main's side and are then recomputed
on the merged tree: spec 4ae5e2e6, producer registry 2c71f35d, full checkpoint
6a6a3f74, loader golden 646929f6; coverage evidence regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 11 commits November 9, 2026 22:43
…ration-statuses-767

Tenth sync, onto main 1acd884. Main's ACS-local immigration lane
(#1052) now runs on the #767 kernel: the donor's delivered worker and
student mass counts the Pew universe the spill binds, each humanitarian
draw targets its own donor residual, ACS ESR maps onto the A_LFSR
codebook, and the signal gate grades the build's period.

The attested schema-9 legacy path keeps main's design, which rebuilds
the schema-16 schedule and v11 authority from today's registry. Only
#767's two immigration producers keep their schema-16 inventories: the
legacy contracts take them from the frozen schedule and check each
rebuilt contract against the digest the schema-16 registry derived. The
legacy schedule payload, authority and all 38 legacy contracts equal
main's, and the live schedule payload does not move.

Two fixes ride along. The 2026 ASEC adds PEINUSYR code 29, which the
default pool had started to refuse: this restores main's 29: 2024
midpoint, widens the domain to 0..29 and names each program's
arrival-code window. The ACS-transfer contract pins move with #767's
schema-3 contract.

Git merged two edits to the ACS loader's column lists without a
conflict: the PR had made CIT, POBP and YOEP required, and main's #1020
made them optional. Main's design wins. The loader leaves an absent
column absent, and the immigration consumers refuse a row with no
citizenship source; new tests check the refusal through the kernel's
evidence profile and through the evidence features the ACS transfer
reads. The late-producer scope test now names the three columns itself,
and the transport battery's binding closure counts us_runtime.acs_pums,
whose ACS_2024_1YR_VINTAGE the transfer imports.

Recomputed on the merged tree: spec 49d7b804, producer graph 2606b852,
full checkpoint d7829268, loader golden 7dd74e95, late schedule
46cea046; coverage evidence regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ration-statuses-767

Eleventh sync, onto main 764ae3a. One file conflicted: the transport
battery's binding closure list, where main's #1405 added the
microcosm.data modules and the tenth sync added us_runtime.acs_pums;
the list keeps both. Git merged main's SCF wealth gate record (#1216)
beside this PR's humanitarian target materializer in the fiscal
refresh release tool and its tests without a conflict; those tests
pass. The spec-engine, bundle, coverage and pool pin checks pass
unchanged (spec 49d7b804).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The release-build rule asks this PR for a from-scratch build from its
own tree; d2729 asks Max whether stage receipts stand in for it, on the
terms of d2647. The receipts run the immigration stage alone on main's
default pool (the 2024-2026 ASEC, the pool d2603 gives the next
certified build) at this PR's merge of main 764ae3a, at that main,
and at the pre-sync head 42bb166, then run the composition gate.

At the merge the gate passes with no failures: refugees, asylees and
TPS emit within 3% of their cited stocks, one-year parole reaches 0.90
and citizens match main. The pre-sync head refuses the 2026 ASEC's
PEINUSYR code 29. The receipts hold aggregates only; the record-level
labels behind the README's row-by-row table stay out of the repository,
and scripts/xtab_labels.py rebuilds the table from two runs' labels.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The independent review of bad5cb6 found that the tenth merge kept main's
stacked-authority pins (receipt and late-producer schedule component) in
test_spec_engine_stacked_authority_semantics.py, so engine-US failed. Pin the
merged tree's live values.

The review also found that the Operation Allies Welcome parole window on
exact-year (ACS YOEP) rows has no end year. Keep it open and say why above
_PAROLE_ACS_MIN_ARRIVAL_YEAR: the stacked transfer asks the ACS arm for its
own share of each national target and raises on a candidate shortfall, and
the ACS 2024 1-year holds 70,123 weighted Afghan-born non-citizens who
arrived in 2021-2022, fewer than the 73,566 target. The comment moves the
attested kernel identities: seed protocol and map digests, the US spec
(49d7b804 -> c414c985), the coverage evidence and the minimal-loader golden.

Name the legacy path's second reach in the multispine design doc: post-clone
source rows bind today's whole-file source_stages.json hash, which #767
changed for the fifteen source rows that bind a stage spec.

The stage receipt script now records the key each draw call uses instead of
a constant, and --draw-keys full|legacy forces one key so two trees' runs
share their draws.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…changes

The review of bad5cb6 found that every receipt reported this PR's draw
key, although main keys its draws on the income year and PERIDNUM alone.
The script now records each draw call's key. Regenerate the head, main and
pre-sync receipts with it, and add receipt.main_full_keys.json: main's
method run on this PR's key.

The README credited the method with all the rows the two trees label
differently. The key change alone moves about as many: on this PR's key,
main's own method moves 2.400 million people from undocumented to LPR and
2.392 million back. Holding the key fixed, the method moves 0.319 million
and 0.706 million. Rewrite the README to name both causes, with a
row-by-row table for each.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y candidate

The delta review of e3b2c8c and 9d16436 (wf_5bb29ab5-2eb) found that
the post-export scorer cannot score the H.R.1 Medicaid probe. PolicyEngine-US
prices each Medicaid enrollee through medicaid_slcsp_state_denominator, an
aggregate over the whole file, which the household-batched scorer refuses and
a reform engine refuses even in one batch (#956). Engine-US failed, and a
release build would have aborted at the reform-coverage smoke. The probe now
counts weighted people in medicaid_enrolled, with a 10,000-person floor.
Probes carry a measure_unit, usd by default or persons; the smoke gate states
effects and floors in it, and the manifest check refuses a budget measure that
is not a live variable of that unit. medicaid_reporters.py counts the people
in the restored statuses who report Medicaid at interview, from the head
run's labels.

The review also found that the receipt script said main's tree has no
composition gate. Main has one, so the script now runs each tree's own gate.
The changelog fragment no longer says schema-9 pools authenticate against the
contract they sealed.

The review found that the OAW kernel comment misstated the stacked
transfer's rule: the reconciliation asks the ACS rows for each draw's target
less the ASEC rows' labelled mass. Under that rule, U4U, Nicaraguan parole
and Honduran TPS hold too few ACS candidates at the stacked ACS arm's
weights, and the reconciliation refused any shortfall, so every stacked build
would have stopped at U4U. A draw short of candidates now takes every one, as
the ASEC stage's draws do, and its receipt records candidates_exhausted and
candidate_shortfall_population. The reconciliation refuses only a draw that
ends below 0.35 of its target, the composition band's floor, and the stacked
validator replays the rule. The transfer contract's candidate_shortfall value
changes, which moves its three identities. The Operation Allies Welcome window
on exact-year rows now ends in 2022. stacked_supply.py projects each draw's
ACS supply against its residual.

The kernel edits move the attested identities: seed protocol and map digests,
producer resource semantics, the stacked checkpoint base identity, the US spec
(c414c985 -> 783ff9a4), the coverage evidence and the minimal-loader golden.
The coverage manifest's ACA probe reasons now cite the default-pool receipt's
TPS and humanitarian totals.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… receipt

The receipt script reran each tree at the code commit. The head run labels
every row as before and its composition, gate and draw results equal the
previous head receipt's; only provenance moves. Main's two receipts now carry
the result of main's own composition gate, which passes. The Medicaid-reporter
receipt names the new head's labels file.

receipt.stacked_supply.json projects each humanitarian draw's ACS supply in a
stacked pool from receipt.head.json and the ACS 2024 1-year PUMS. Four draws
run short of ACS candidates and end at 0.718 to 0.971 of their targets; in the
worst case none ends below half its target. The README's receipts table,
stacked-pool section and rerun instructions describe it.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stacked-supply section listed four draws that run short of ACS
candidates without saying the projected share assumes no earlier draw
takes a draw's candidates. Define the projected share, note that none of
the four shares a candidate with an earlier draw, and add TPS Venezuela:
its supply exceeds its residual by 38,923, but earlier draws could take
up to 214,138 of its candidates, which would leave it at 0.710 of its
target (receipt.stacked_supply.json, worst_case_share_of_target).

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The third delta review of #779 (wf_06fd9b1f-5b7) confirmed two defects in
the stacked-supply prose. Honduran TPS does share candidates with an
earlier draw: every one also falls in the deportation-withheld draw's
pool, whose zero target takes none, so its worst case still equals its
projected share. And TPS Venezuela does not just "may" run short:
Venezuelan parole, the fourth draw, takes 58,115 from a pool that shares
0.848 of its mass with TPS Venezuela's, about 49,300 of it, more than TPS
Venezuela's 38,923 margin.

stacked_supply.py now projects an expected share for each parole and TPS
draw: each earlier parole or TPS draw takes its residual from the
candidates still free, each with the same chance, as the reconciliation's
person-keyed hash draws do. It leaves out refugee and asylee takes, which
PUMS cannot show. TPS Venezuela's expected share is 0.983; every other
draw's equals its projected share. The receipt gains that one field per
draw and the script's new hash; every other value is unchanged. The README
table gains the column, and its bullets count five short draws.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The delta review of a5a1dda (wf_78421e89-31e) found two docstring
claims in stacked_supply.py that its code contradicts. The expected
share sits between the worst case and the projected share, not between
the two lower bounds: for TPS Venezuela it is 0.983 against a worst case
of 0.710. And leaving refugee, asylee and withholding takes out does not
overstate a later draw's share by all they take of its candidates: those
takes can only lower the share, and only once the draw's free supply
falls below its residual, as TPS Venezuela's does.

The receipt changes only in script_sha256.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The confirmation of d12edbd (wf_abe4b4fc-c03) found that a draw that
does not exhaust its candidates also takes the tie group that crosses its
residual, and a refugee, asylee or withholding take can move that
overshoot either way, by less than one row's weight. The docstring now
takes the reviewer's wording: those takes can only lower a later draw's
share, up to the weight of that tie group, and lower it by more only once
the draw's free supply falls below its residual.

The receipt changes only in script_sha256.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

MaxGhenis commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Hand-off to the Microcosm hub (2026-11-10). The owner session hands this PR to the hub, which lands it.

  • Head: 1f0009cbeddb87c04389bec4dea01173e946ad1b. Code head 4f94e489a (US spec 783ff9a4). The five commits after it (2f70d5c3c, 5610bde70, a5a1dda0b, d12edbd05, 1f0009cbe) change only docs/evidence/us-immigration-statuses-767/.
  • Independent review (native Opus 5.5 workflows), agreed at the head:
    • wf_06fd9b1f-5b7, which continued wf_86849e93-d08 and wf_6361e255-615 after a usage limit stopped them: no code defect at 4f94e489a; three minor prose defects, fixed in a5a1dda0b and the PR body. Record: ~/reviews/microcosm-767-sync10/runs/review-c34-delta/ (MANIFEST.md, result-c35.json).
    • wf_78421e89-31e at a5a1dda0b: two docstring defects in stacked_supply.py, fixed in d12edbd05.
    • wf_abe4b4fc-c03 at d12edbd05: one wording point (the crossing tie group), fixed in 1f0009cbe with the reviewer's own wording. No finding remains open. Record: ~/reviews/microcosm-767-sync10/runs/review-c36-delta/ (MANIFEST.md, result.json, result-confirm.json).
  • Checks: every check passed at 2f70d5c3c. Pushes cancelled the runs at 5610bde70, a5a1dda0b and d12edbd05; the run at 1f0009cbe is pending and is the first against main c1eb707f2. Locally, the four read-only pin checks pass on this head's code merged with c1eb707f2, and the US spec stays 783ff9a4.
  • Hold: decision d2729 (waive the from-scratch PR-tree receipt and land on the default-pool stage receipts; deadline 2026-11-12; the default, if silent, holds this PR). Do not merge before a yes on d2729: the decision closes itself when this PR merges. The hub's own merge order also applies: this PR lands first among the pin-touching PRs held until 2026-11-11 18:15Z.
  • Depends on: nothing.
  • Owed after merge: finalize ~/reviews/microcosm-hub/notes/1085-sync-after-779.md with the merge commit; tell local_ccd7a569 (Carry SIPP/SCF assets on the declared household head #1221) if this PR closes unmerged or stops landing first. Out of scope and filed: US: store years_since_us_entry from PEINUSYR (blocked on engine ICHIA option) #776, Verification/redetermination churn flag surface (option 2 of #767, generalized) #777, Add the CHIP §71109 reform-coverage probe when the engine encodes it (pe-us#9346) #778.

The engine-free (3.14) leg of this head's run drew an ubuntu-26.04 runner
twice (jobs 130198769545 and 130281464672). uv took that image's Python
3.14.4 there, and the head-to-head scorer's worker-recycling test needs
3.14.7. Main pins every Tests job to ubuntu-24.04 since #1576, and a re-run
keeps the old workflow, so this empty commit starts a run that reads main's.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Hub note: I pushed 173766c97, an empty commit on 1f0009cbe with the same tree, to start a CI run on the pinned ubuntu-24.04 workflow from #1576. The engine-free (3.14) leg failed twice at 1f0009cbe on ubuntu-26.04 runners, where uv took the image's Python 3.14.4 and the head-to-head scorer's worker-recycling test needs 3.14.7. The reviews at 1f0009cbe cover this head, since no file changed. The PR stays held for d2729.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

US: humanitarian immigration statuses absent from the imputation — H.R.1 §71109/§71302 eligibility channels compute ≈$0

1 participant