CPS top-coding caps AGI at $6.26M — zero observations above $10M in any state

[PavelMakarchuk]

Summary

An audit of the Enhanced CPS state datasets reveals a hard top-coding ceiling at $6,263,051 AGI across all 51 state datasets. No state has a single observation above $10M, and only a handful have sparse records between $5M-$6.3M. This creates significant data quality issues for modeling policies that target high-income brackets (e.g., NY A05435 which raises rates on $5M-$25M and $25M+ brackets).

Impact

• Bills targeting income above $10M or $25M have zero observations to model against
• Revenue estimates for high-bracket policies are systematically understated
• Example: NY A05435 estimates $120M revenue from the $5M+ bracket change, but captures zero impact from the $25M+ bracket change — the real revenue would be substantially higher

Audit methodology

For each state, loaded the Enhanced CPS dataset via Microsimulation, calculated adjusted_gross_income at the tax unit level, and counted raw records and weighted totals above $1M, $5M, $10M, and $25M thresholds.

from huggingface_hub import hf_hub_download
from policyengine_us import Microsimulation

dataset_path = hf_hub_download(
    repo_id="policyengine/policyengine-us-data",
    filename=f"states/{state}.h5",
    repo_type="model",
)
sim = Microsimulation(dataset=dataset_path)
agi = sim.calculate("adjusted_gross_income", 2026).values
weight = sim.calculate("tax_unit_weight", 2026).values

Full results by state

State	$1M+ raw	$1M+ wtd	avg wt	$5M+ raw	$5M+ wtd	avg wt	$10M+ raw	$10M+ wtd	$25M+ raw	$25M+ wtd	Max AGI
AL	513	19,593	38	22	76	3	0	0	0	0	$6,263,051
AK	67	3,263	49	3	136	46	0	0	0	0	$5,575,013
AZ	864	40,390	47	31	758	24	0	0	0	0	$6,263,051
AR	325	11,072	34	12	366	31	0	0	0	0	$6,263,051
CA	10,353	344,096	33	304	5,552	18	0	0	0	0	$6,263,051
CO	1,251	45,100	36	37	866	23	0	0	0	0	$6,263,051
CT	839	42,203	50	26	463	18	0	0	0	0	$6,263,051
DE	97	5,182	53	2	71	36	0	0	0	0	$5,571,654
DC	311	9,054	29	9	240	27	0	0	0	0	$5,793,470
FL	1,971	208,750	106	89	8,724	98	0	0	0	0	$6,263,051
GA	1,620	58,722	36	42	611	15	0	0	0	0	$6,263,051
HI	286	5,341	19	10	81	8	0	0	0	0	$5,793,470
ID	178	9,561	54	5	112	22	0	0	0	0	$5,607,726
IL	1,818	93,636	52	57	1,313	23	0	0	0	0	$6,263,051
IN	781	31,170	40	24	181	8	0	0	0	0	$6,263,051
IA	269	15,641	58	7	106	15	0	0	0	0	$6,263,051
KS	427	14,788	35	11	142	13	0	0	0	0	$6,263,051
KY	475	15,236	32	21	133	6	0	0	0	0	$6,263,051
LA	444	19,520	44	9	97	11	0	0	0	0	$6,263,051
ME	158	6,247	40	4	125	31	0	0	0	0	$5,699,046
MD	1,528	36,862	24	48	358	7	0	0	0	0	$6,263,051
MA	1,839	78,200	43	53	1,321	25	0	0	0	0	$6,263,051
MI	944	52,221	55	28	287	10	0	0	0	0	$6,263,051
MN	1,116	35,256	32	33	403	12	0	0	0	0	$6,263,051
MS	263	8,166	31	7	13	2	0	0	0	0	$5,793,470
MO	604	30,117	50	23	157	7	0	0	0	0	$6,263,051
MT	224	6,538	29	3	172	57	0	0	0	0	$5,571,654
NE	189	11,803	62	9	62	7	0	0	0	0	$6,263,051
NV	274	23,761	87	10	577	58	0	0	0	0	$5,699,046
NH	156	11,986	77	5	144	29	0	0	0	0	$5,586,008
NJ	2,275	88,779	39	64	958	15	0	0	0	0	$6,263,051
NM	196	7,310	37	5	20	4	0	0	0	0	$6,263,051
NY	4,730	187,147	40	153	4,335	28	0	0	0	0	$6,263,051
NC	1,498	55,637	37	46	385	8	0	0	0	0	$6,263,051
ND	81	5,731	71	5	38	8	0	0	0	0	$6,263,051
OH	1,088	59,179	54	30	285	10	0	0	0	0	$6,263,051
OK	342	16,165	47	6	208	35	0	0	0	0	$5,686,040
OR	942	20,904	22	32	398	12	0	0	0	0	$6,263,051
PA	1,598	81,179	51	46	815	18	0	0	0	0	$6,263,051
RI	175	6,691	38	3	58	19	0	0	0	0	$5,675,536
SC	641	23,356	36	18	275	15	0	0	0	0	$6,263,051
SD	48	6,585	137	3	116	39	0	0	0	0	$5,686,040
TN	518	39,970	77	18	166	9	0	0	0	0	$6,263,051
TX	2,242	199,803	89	98	4,544	46	0	0	0	0	$6,263,051
UT	618	18,401	30	18	89	5	0	0	0	0	$6,263,051
VT	86	3,199	37	2	24	12	0	0	0	0	$5,586,008
VA	1,481	56,697	38	42	621	15	0	0	0	0	$6,263,051
WA	631	72,514	115	26	2,636	101	0	0	0	0	$6,263,051
WV	114	4,967	44	3	11	4	0	0	0	0	$5,607,726
WI	619	32,345	52	20	302	15	0	0	0	0	$6,263,051
WY	63	4,636	74	3	357	119	0	0	0	0	$5,572,205

Key observations

1. Hard ceiling at $6,263,051 — this is the CPS top-coding/swapping cap. Most states hit this exact number.
2. Zero observations above $10M in every state — policies targeting $10M+ or $25M+ brackets cannot be modeled.
3. Sparse $5M+ data — even large states have very few records (NY: 153 raw / 4,335 weighted; CA: 304 raw / 5,552 weighted). Many of these have tiny weights (1-5), meaning a single record can disproportionately drive results.
4. Weight instability — average weights for $5M+ records vary wildly (FL: 98, WA: 101 vs MS: 2, KY: 6), suggesting calibration noise at the tail.

Practical consequence

For the state legislative tracker, any bill targeting income above $5M will have unreliable estimates, and bills targeting $10M+ or $25M+ will show zero impact from those brackets. This needs to be disclosed as a data limitation or addressed via tail imputation (e.g., Pareto extrapolation from IRS SOI data).

[PavelMakarchuk]

Enhanced CPS vs IRS SOI comparison ($1M+ AGI filers, by state)

Compared our Enhanced CPS state datasets (2026 projection) against IRS SOI Historic Table 2 (TY2022, AGI_STUB=10 = "$1M or more").

Key findings

1. CPS over-counts $1M+ filers by ~3x — every state shows weighted counts at 250-400% of SOI actuals. This likely results from calibration compensating for missing ultra-high incomes by duplicating moderate-high-income records.
2. Total AGI is also over-represented (~2-3x) — but less than the count overcount, because CPS can't represent the true tail.
3. Average AGI is compressed — SOI averages range $2.5M-$5.7M across states; CPS averages are uniformly ~$2.7M due to the $6.3M ceiling. Worst case: WY has SOI avg $5.66M vs CPS avg $2.94M.

Full comparison table

State	SOI returns	CPS wtd	Count ratio	SOI AGI ($B)	CPS AGI ($B)	AGI ratio	SOI avg AGI	CPS avg AGI	CPS max AGI
AL	6,490	19,593	301.9%	$19.2	$52.5	273.8%	$2,957,171	$2,682,035	$6,263,051
AK	1,070	3,263	305.0%	$2.5	$9.0	355.9%	$2,361,368	$2,755,532	$5,575,013
AZ	14,020	40,390	288.1%	$43.8	$110.4	252.3%	$3,121,250	$2,733,640	$6,263,051
AR	3,520	11,072	314.5%	$15.1	$29.9	198.2%	$4,279,320	$2,696,117	$6,263,051
CA	128,900	344,096	266.9%	$412.6	$919.0	222.7%	$3,200,663	$2,670,674	$6,263,051
CO	16,050	45,100	281.0%	$47.7	$122.3	256.4%	$2,971,365	$2,710,939	$6,263,051
CT	16,320	42,203	258.6%	$57.4	$109.9	191.3%	$3,519,138	$2,603,664	$6,263,051
DE	1,660	5,182	312.2%	$4.3	$14.0	324.4%	$2,597,519	$2,699,139	$5,571,654
DC	3,320	9,054	272.7%	$12.3	$26.6	216.7%	$3,697,711	$2,938,179	$5,793,470
FL	77,670	208,750	268.8%	$346.6	$592.1	170.9%	$4,462,178	$2,836,611	$6,263,051
GA	21,070	58,722	278.7%	$62.3	$156.5	251.2%	$2,958,028	$2,665,708	$6,263,051
HI	1,970	5,341	271.1%	$6.9	$15.1	219.4%	$3,497,882	$2,831,186	$5,793,470
ID	3,420	9,561	279.6%	$10.7	$26.2	246.2%	$3,114,844	$2,743,100	$5,607,726
IL	30,220	93,636	309.8%	$100.1	$254.0	253.8%	$3,311,907	$2,712,506	$6,263,051
IN	9,310	31,170	334.8%	$28.2	$86.8	308.0%	$3,028,392	$2,786,194	$6,263,051
IA	4,340	15,641	360.4%	$13.0	$43.4	333.9%	$2,993,459	$2,773,622	$6,263,051
KS	4,880	14,788	303.0%	$14.4	$40.3	281.0%	$2,941,197	$2,727,433	$6,263,051
KY	4,690	15,236	324.9%	$13.4	$41.7	311.6%	$2,855,356	$2,738,801	$6,263,051
LA	6,370	19,520	306.4%	$17.8	$52.6	295.5%	$2,793,650	$2,693,904	$6,263,051
ME	1,970	6,247	317.1%	$5.0	$18.0	360.4%	$2,530,462	$2,875,988	$5,699,046
MD	13,070	36,862	282.0%	$35.5	$102.8	289.4%	$2,716,705	$2,787,870	$6,263,051
MA	27,270	78,200	286.8%	$92.3	$214.0	231.8%	$3,385,513	$2,737,148	$6,263,051
MI	15,370	52,221	339.8%	$47.4	$143.8	303.4%	$3,083,772	$2,753,977	$6,263,051
MN	11,390	35,256	309.5%	$32.4	$97.4	301.0%	$2,840,876	$2,762,328	$6,263,051
MS	2,560	8,166	319.0%	$8.2	$22.6	276.9%	$3,188,195	$2,767,994	$5,793,470
MO	9,110	30,117	330.6%	$29.2	$79.8	273.3%	$3,204,873	$2,649,334	$6,263,051
MT	2,320	6,538	281.8%	$8.8	$17.4	198.4%	$3,781,478	$2,662,105	$5,571,654
NE	3,230	11,803	365.4%	$10.9	$32.7	299.5%	$3,381,033	$2,770,941	$6,263,051
NV	8,420	23,761	282.2%	$39.1	$69.1	176.5%	$4,648,299	$2,908,000	$5,699,046
NH	3,640	11,986	329.3%	$12.9	$32.3	250.8%	$3,537,087	$2,694,278	$5,586,008
NJ	31,390	88,779	282.8%	$87.7	$228.6	260.6%	$2,794,569	$2,574,467	$6,263,051
NM	2,080	7,310	351.4%	$5.5	$19.3	348.2%	$2,666,159	$2,641,829	$6,263,051
NY	69,780	187,147	268.2%	$267.9	$521.9	194.8%	$3,839,669	$2,788,477	$6,263,051
NC	18,890	55,637	294.5%	$53.5	$148.4	277.3%	$2,831,917	$2,666,599	$6,263,051
ND	1,620	5,731	353.8%	$4.5	$15.6	350.8%	$2,753,560	$2,730,150	$6,263,051
OH	16,680	59,179	354.8%	$49.2	$161.1	327.4%	$2,950,053	$2,722,696	$6,263,051
OK	5,260	16,165	307.3%	$15.5	$44.0	283.7%	$2,945,781	$2,719,672	$5,686,040
OR	6,490	20,904	322.1%	$18.3	$57.8	316.3%	$2,816,999	$2,766,013	$6,263,051
PA	24,850	81,179	326.7%	$75.8	$219.4	289.4%	$3,051,622	$2,703,188	$6,263,051
RI	1,870	6,691	357.8%	$6.0	$18.1	303.1%	$3,201,907	$2,712,110	$5,675,536
SC	8,290	23,356	281.7%	$24.4	$63.5	260.5%	$2,942,554	$2,720,728	$6,263,051
SD	1,990	6,585	330.9%	$6.6	$18.4	279.0%	$3,322,817	$2,801,243	$5,686,040
TN	13,490	39,970	296.3%	$43.8	$108.7	248.3%	$3,245,298	$2,719,981	$6,263,051
TX	73,930	199,803	270.3%	$262.8	$541.6	206.1%	$3,554,250	$2,710,904	$6,263,051
UT	7,150	18,401	257.4%	$25.5	$49.2	192.6%	$3,572,829	$2,673,658	$6,263,051
VT	1,070	3,199	299.0%	$2.9	$9.1	311.5%	$2,723,311	$2,837,284	$5,586,008
VA	18,720	56,697	302.9%	$58.2	$151.3	259.9%	$3,110,417	$2,668,857	$6,263,051
WA	21,530	72,514	336.8%	$66.0	$201.7	305.9%	$3,063,483	$2,782,018	$6,263,051
WV	1,260	4,967	394.2%	$3.3	$13.2	402.1%	$2,596,671	$2,648,407	$5,607,726
WI	9,160	32,345	353.1%	$30.4	$90.2	296.5%	$3,321,893	$2,789,109	$6,263,051
WY	1,760	4,636	263.4%	$10.0	$13.6	137.0%	$5,658,187	$2,942,277	$5,572,205

Interpretation

The Enhanced CPS calibration appears to be overshooting on the count of $1M+ filers (likely to match total AGI or income targets) but can't represent the income distribution within that group due to the $6.3M ceiling. It compensates for missing ultra-high incomes by adding more moderate-high-income records.

States with high concentrations of ultra-wealthy filers (FL, NY, CT, NV, WY) show the worst AGI-average compression — their SOI averages are $3.5M-$5.7M vs CPS uniformly ~$2.7M.

[baogorek]

Great investigation, @PavelMakarchuk. It looks like subsample is specific mechanism causing the $6.26M ceiling. I'm bringing the functionality from PR #516 into the pipeline in batches. This fix will land in an upcoming round of updates.

Root cause: weight-proportional subsampling of the PUF

The pipeline brings IRS PUF (Public Use File) data into the CPS via Quantile Random Forest (QRF) imputation in extended_cps.py. The PUF has detailed tax return microdata with incomes well into the hundreds of millions — exactly the kind of tail data we need. The problem is how we subsample it before training the QRF.

The current code does:

puf_sim = Microsimulation(dataset=self.puf)
puf_sim.subsample(10_000)

The subsample() method in policyengine-core samples records proportional to their survey weights. This is the standard approach for creating representative population subsamples, but it's catastrophic for the income tail. High-income PUF records have very low weights because each one represents only a handful of actual tax filers in the population:

AGI threshold	Full PUF records	Avg weight per record	After subsample(10_000)
> $1M	49,593	12.9	46
> $5M	15,420	4.3	7
> $10M	11,390	2.2	0
> $25M	5,992	1.1	0

The full PUF has 11,390 records above $10M AGI (max: $231M), but their average weight is only 2.2, each represents roughly 2 tax filers out of 161 million. When subsample(10_000) draws proportional to weight, these records have near-zero probability of selection. The result: zero records above $10M survive into the QRF training data. There's then a secondary subsample to 5,000 within the QRF batching that compounds the problem.

The QRF itself is flexible. It's a quantile-based imputer that preserves the full conditional distribution of its training data. But it can't produce values it's never seen. With training data maxing out at $8M, the imputed income components max out around $3M per variable, giving a combined max AGI of ~$5.8M in the ExtendedCPS dataset. Uprated from 2024 to 2026, that becomes the $6,263,051 ceiling you're seeing.

The downstream calibration/reweighting can only adjust weights of existing records; it cannot create new income values. So even though we have SOI calibration targets for the $5M–$10M and $10M+ brackets, the optimizer has zero (or near-zero) records to assign weight to in those brackets. It compensates by massively upweighting the moderate-high-income records ($1M–$6M), producing the ~3x overcount of $1M+ filers you documented in your SOI comparison.

How PR #516 fixes this

@MaxGhenis's PR #516 (Reorder calibration pipeline: clone before PUF, geography before QRF) restructures the calibration pipeline and includes a new puf_impute.py module that replaces the PUF imputation step. The critical change:

puf_sim = Microsimulation(dataset=puf_dataset)
# Use all PUF records for QRF training (no subsample)

All 207,692 PUF records, including the 11,390 with AGI above $10M and the 5,992 above $25M, go directly into QRF training. The secondary sample(5,000) within batching is also removed. With the full PUF tail in the training data, the QRF can draw from the actual income distribution and produce the $10M, $25M, and $50M+ values that the reweighting optimizer needs to match SOI targets.

Additionally, state_fips is added as a QRF predictor, making imputations geographically aware, which should help with the state-level concentration issues (e.g., NY, CA, FL having disproportionate shares of ultra-high earners).