Check the Table 2.5 EITC child splits against the real IRS rows in CI

[MaxGhenis]

Why

The app v3 Validation tab (PolicyEngine/policyengine-app-v2, report 4323) showed the US EITC at about 2× its SOI Table 2.5 child-split targets. The model was right and the targets were wrong.

The tab reads irs_soi.ty2023.table_2_5.eitc_by_agi_children.<split>.total.{eitc_returns,eitc_total}@2024 from release populace-us-2024-spm-20260915's calibration_diagnostics.json. Those 8 rows summed to 11.92M returns and $34.52B. IRS published "more than 23 million" and "about $64 billion" for TY2023.

Root cause: the compile, not the extraction

The extraction is correct. Chronicle's soi-table-2-5-eitc-agi-children-2023 package reads IRS 23in25ic.xls (sha256 c6a58f63…, byte-identical to the live IRS file on 2026-10-14). Checked against the workbook:

• Columns. It reads the "Total earned income credit" columns for each split: Z/AA (none), AP/AQ (one), BF/BG (two), BV/BW (three or more). Amounts are scaled ×1,000.
• Rows. Rows 10–37 map to the 28 AGI bins and row 9 to the total. Every row has a header guard.
• Totals. Each split's bins add to its total row within 3 returns and $1,000. The four splits add to the all-returns total, 24,439,936 returns and $66,270,000k (within $1,000).

The bug was in microcosm's compile. _uprate_cross_period_eitc_decompositions scales the TY2023 rows to the TY2024 filing-season EITC by AGI. For the all-AGI total rows, the group is (-inf, inf). _eitc_source_decomposition_total_for_bounds summed every row inside that group: each split's 28 bins and its total row. So the denominator was 2 × the source:

	Released factor	Its denominator	Correct factor
Returns	0.487667991438275	23,837,149 / 48,879,872 (= 2 × 24,439,936)	0.975335982876551
Amount	0.520911812001927	$69,041,650,000 / $132.54B (= 2 × $66.27B)	1.04182361614341

The AGI-band rows were not affected. For example, 10k–15k used 3,819,056 / 4,197,059 = 0.9099, the correct all-split sum. The code path dates from #157 (2026-06-20).

The fix is #1227, merged 2026-10-14 as de90c6c7e. It counts each split's outermost rows only. An independent Opus 5.5 review at its head 40bfb23ea approved it after the merge (Subfleet lanes were closed, so it ran in-session). That reviewer re-derived the effect with a whole-registry differential on the pinned feed: exactly 8 targets change, each ×2.

What this PR adds

Tests only, so the halving can't come back unnoticed in CI. The existing pinned-feed tests skip in CI because CI has no feed.

• Fixture. tests/fixtures/us_soi_eitc_table_2_5_feed_rows.jsonl holds the 274 rows of the pinned feed consumer_facts_us_51aa40f.jsonl (sha256 965a29ac…) for:

  • TY2023 Table 2.5: the all-returns total row and the four splits × 29 rows × 2 measures;
  • TY2024 filing-season week 47 EITC by AGI (25inweek47.xlsx): 16 bins and the totals.

  They are verbatim, sorted by record id, and loaded through load_ledger_consumer_artifact as the release is. test_fixture_rows_are_verbatim_pinned_feed_rows checks them byte for byte against the feed wherever it is present.

• Source checks. All are example tests on the real rows.

  • Each split's AGI bins tile AGI and add to its total row within SOI rounding (5 returns, $5,000).
  • The filing-season AGI bins add to their total.
  • The four splits add to the all-returns total.

• IRS reconciliation. These read the IRS EITC statistics pages, accessed 2026-10-14.

  • TY2023 split totals are within 10% of IRS-published TY2023 EITC: 24.44M vs "more than 23 million" (+6.3%), and $66.27B vs "about $64 billion" (+3.5%).
  • Child shares are within 1.5 points of IRS's 27/36/24/13%. Ours are 27.7/35.5/23.7/13.1%.
  • Why 10%: the two products have different documented bases. IRS's page reports TY2023 "returns processed in 2024 … as of December 2024". Table 2.5 is SOI's sample estimate for "Tax Year 2023 (Filing Year 2024)"; its footnote 1 defines EIC earned income "for prior-year returns", which implies they are included. The observed gap is +6.3% / +3.5%, and the tests don't attribute it. 10% covers it; a halving misses by about 48%.

• Compiled checks. These compile the real rows at target period 2024.

  • The four total rows add to the sum of the filing season's 16 AGI bins (23,837,149 returns; $69,041,650,000; the filing-season total row reads $69,041,649,000) to 1e-12, and so do the 112 bins.
  • Each total row's factor is active / source.
  • The compiled 2024 sum is within 10% of IRS-published TY2024 EITC (about 24M and $69.6B; it is 0.7% and 0.8% below).

• Property tests. These are Hypothesis tests on the real Table 2.5 layout, 100 examples each. They cover both grouping paths: bins inside one filing-season bin, and the $50k-and-over bin, which spans seven.

  1. For any nonnegative filing-season EITC by AGI, the four total rows add to its total and keep the TY2023 child mix. The 112 AGI bins add to the same total, and to the filing season's EITC within each uprating group.
  2. Under uniform growth, each split's 28 compiled bins add to its compiled total row, within the source's own rounding gap. So "AGI-band cells sum to the reported totals" holds per split.

  Under non-uniform growth it holds summed over splits, but not per split. That is by design since Count each EITC child split once in the Table 2.5 uprating source total #1227: a total row keeps the all-AGI child mix while each bin keeps its group's. Reconciling the two is the methodology question in Rake the EITC child-split bins to their total rows and AGI groups #1235.

On the pre-fix code (main at 21416b03e), 4 of the 15 tests fail: both compiled reconciliations (11,918,574.5 returns and $34,520,825,260 against 23,837,149 and $69,041,650,000) and both properties. Hypothesis minimizes the defect to a filing season with 1 return, in 10k–15k: the total rows compile to 0.5. The 11 source and fixture tests pass on both trees, because the extraction was never wrong.

Did the calibration weights use these targets?

Yes. Release populace-us-2024-spm-20260915 inherits its calibration from Build P (populace-us-2024-buildp-sparse-rmloss100-cae8640-20260728T011454Z). Its calibration_diagnostics.json lists 5,659 targets, which equals target_surface.n_targets. That surface includes all 232 Table 2.5 child-split rows, the 8 halved total rows among them. Loss weights were sqrt_value_concept_budget_weighted…_cap_100pct with relative error capped at 1.0.

The halved rows pulled against the other EITC targets and lost:

Split	Measure	Released target	Corrected (#1227)	Initial	Final	Final vs released	Final vs corrected
None	returns	3.31M	6.61M	3.97M	6.72M	+103.2% (initial +20.2%)	+1.6%
One	returns	4.23M	8.47M	5.68M	8.37M	+97.7% (initial +34.1%)	−1.2%
Two	returns	2.82M	5.64M	4.33M	5.51M	+95.4% (initial +53.5%)	−2.3%
Three+	returns	1.56M	3.11M	3.21M	3.09M	+98.2% (initial +106.1%)	−0.9%
None	amount	$1.28B	$2.56B	$1.38B	$2.42B	+89.1% (initial +7.5%)	−5.4%
One	amount	$12.20B	$24.41B	$15.26B	$24.02B	+96.8% (initial +25.1%)	−1.6%
Two	amount	$12.97B	$25.94B	$18.54B	$24.75B	+90.8% (initial +42.9%)	−4.6%
Three+	amount	$8.06B	$16.13B	$14.93B	$15.58B	+93.2% (initial +85.1%)	−3.4%

Seven of the 8 started inside the cap and finished at or near it. The fit followed the other EITC targets: the filing-season totals and AGI bins, the 224 Table 2.5 bins, and Historic Table 2 state EITC by children. The childless returns row was pushed past the cap (it is in past_cap_census.pushed_out_rows). The final weighted estimates are within −5.4% to +1.6% of the corrected values. These figures come from the published diagnostics; they don't predict a re-calibration.

What #1227 changes in the next calibrated build. The reviewer re-derived these on the pinned feed.

• The 8 total rows double, to the corrected column above, and now agree with the bins and the filing season instead of opposing them.
• No other target moves.
• Counts are unchanged: 33,250 compiled and 6,102 national_state.
• Registry versions: national_state e1f9906bf0e0 → 3fad14f85468; full f5bd921d6b21 → c3aaea9c1e8c.
• The 8 rows' loss weights rise by about √2, since weights are √|value| over the basis mean (target_loss_weights.py). Count each EITC child split once in the Table 2.5 uprating source total #1227's differential found every other national_state weight moving by at most 0.12% after renormalization.

Given where the final estimates already sit, these rows should go from near-cap error to within a few percent. Their pull against the other EITC targets ends. How much the weights move is only measurable by a calibration build.

This PR changes nothing in the build. It is tests and a fixture.

The Validation tab will keep showing the halved targets until a release is calibrated from main at or after de90c6c7e, published, and selected by the calibration-diagnostics dashboard manifest. The tab reads that release's diagnostics, and publication is a deliberate step (README, "Releasing & alerts").

The zero AGI-band cells

These are not a band-parsing misalignment; the row guards hold. They are two different things:

• Childless, 25k–30k (and above): real. For TY2023, policyengine-us has the childless EITC at $600 maximum, 7.65% phase-out and a $16,370 joint phase-out start. That ends it at $24,213 of AGI for joint filers, so IRS reports 0 there. At 2024 (end $25,511) and 2026 (about $26,820), a few joint filers are eligible above $25,000. The 0 then comes from fixed nominal bins and indexed thresholds, tracked in Period-aware transformation needed for SOI EITC-by-AGI calibration targets #116.
• Three or more, 30k–35k: an IRS publication anomaly, not fixed here. In 23in25ic.xls the three-or-more 25k–30k cell holds 635,160 returns and $3.81B. The 30k–35k cell is 0.
  • All-returns minus the other splits implies about 324,948 and 311,398 returns for those two cells. The all-returns column disagrees with the split sum by −310,212 and +311,398 in those rows, and by −1,187 in the 20k–25k row above them; every other row agrees. TY2022 has no such anomaly.
  • Each split still adds to its total, so these tests pass.
  • It distorts the uprating groups for every split: 25k–30k takes 0.832 instead of about 0.966, and 30k–40k takes 1.030 instead of about 0.948.
  • In the release, the calibration cut three-plus EITC returns at 30k–35k from 174,894 to 16,231.
  • The repair needs Chronicle to extract the all-returns AGI rows, or a methodology choice. It is split into its own task.

Invariants (load-bearing logic)

• Source. Each split's AGI bins tile AGI and add to its total row, and the four splits add to the all-returns total, within SOI rounding.
• Conservation. For any filing-season EITC by AGI, Σ total rows = Σ AGI bins = the filing-season total, and Σ bins in each uprating group = that group's filing-season EITC.
• Child mix. The total rows keep the TY2023 all-AGI child mix, and each group keeps its own.
• Uniform growth. Each split's bins add to its total row, within the source rounding gap.
• External. Split totals reconcile with IRS-published EITC within 10%: the TY2023 source rows with TY2023, and the compiled 2024 rows with TY2024.

Tests run

• uv run pytest packages/microcosm-build/tests/engine_free/us/test_us_eitc_table_2_5_irs_reconciliation.py passes on this branch, 15 of 15 including the local pinned-feed check.
• The related selection (test_us_fiscal_targets.py and test_us_eitc_decomposition_uprating.py, -k 'eitc or decomposition or pinned_feed', -n 4) passes 21 of 21 with the local feed, none skipped. That includes the national_state fence at 3fad14f85468.
• On 21416b03e (pre-Count each EITC child split once in the Table 2.5 uprating source total #1227), 4 fail and 11 pass.
• ruff check and ruff format --check are clean.

Review

Independent review by an Opus 5.5 agent (in-session; every Subfleet lane was closed until about 16:00Z): APPROVE at 4cbdfd1eb. It checked:

• Fixture. The rows are the feed's 274, verbatim. All 232 TY2023 values match 23in25ic.xls, with 0 mismatches. Compiling only the fixture gives the same values for every child-split and filing-season spec as compiling the full pinned feed the release way (age_targets=True, Medicaid substitutions).
• Failure before the fix. On 21416b03e, 4 fail and 11 pass.
• CI behaviour. With no feed present, 14 pass and 1 skips. With outbound network denied, the result is the same.
• Hypothesis stress. 1,500 examples on two seeds, and all 376 uniform-growth values: 0 failures.
• Mutants. It killed every non-equivalent mutant of fiscal_targets.py: total rows or bins ×0.75, ×1.02 and ×1.5; innermost rows; no-merge; drop-zero groups; one-split source; and the pre-fix sum. The one survivor, narrowest→widest group choice, is equivalent on this data, because filing-season bins never overlap.
• Body figures. It verified every number in the calibration table against the release diagnostics.

Its non-blocking findings are fixed in b9b08396f, which it re-reviewed and approved. Its one nit there (footnote 1 implies, rather than states, that prior-year returns are included) is fixed in the head commit, a docstring-only change.

1. The test docstring stated a cause for the Publication 1304 vs December-count gap that nothing verifies. It now states only the two documented bases.
2. The per-split rounding gap was misstated; it is $1,000, not $2,000.
3. The compiled 2024 rows were compared with TY2023 published figures as well. They are now compared with TY2024 only.
4. The source check's docstring said what a sum check cannot see. It now says so.
5. In this body, the row-31 gap was missing, and the filing-season sum was mislabeled; both are corrected.

axiom: n/a: calibration-target compile tests (no policy encoding)

🤖 Generated with Claude Code