Audit the Tax Table against the rate schedule in the federal income tax references (evidence only; no score changes)

[MaxGhenis]

Audits one convention behind every federal income tax reference: the references apply the 26 U.S.C. 1 rate schedule exactly, where the Form 1040 instructions send a filer with taxable income under $100,000 to the Tax Table. Evidence only: no reference, exclusion record, score, payload, benchmark card, paper or frozen snapshot changes. Which convention PolicyBench uses is Max's call, queued as cos decision d1263 and not yet ruled. The full record is reference_audit/2026-10-10-tax-table/README.md.

The point was raised by the independent review of scenario_039 in #213 ($8,600 on the table against the $8,596.03 reference).

Findings

The law	The Tax Table is the tax the statute imposes, in place of the schedule tax, on a filer who does not itemize and is under the ceiling (26 U.S.C. 3(a)(1)). Its amounts must be "computed on the basis of the rates prescribed by section 1". The instructions set the ceiling at $100,000 and send line 16 and the capital gain worksheets' look-ups to the table under it, for itemizers too.
2026	No 2026 Tax Table dated or posted before the 2026-07-03 freeze was found. The earliest found is the IRS's early release draft, dated Aug 28, 2026 and posted 09/16/2026. Archived pages of the IRS's draft listing up to the freeze day show none, though they are a sample and do not cover every day. No final 2026 table was found. The 2026 rate schedule was announced on October 9, 2025 (Rev. Proc. 2025-32).
The construction	The rule inferred here (the schedule's tax on the band's midpoint, a half dollar rounding up) reproduces all 8,248 amounts of the published 2025 table and all 8,248 of the 2026 draft. No document read states the rule.
The engine	policyengine-us 2.38.6 has no Tax Table logic.
What moves	Of 84 scored federal income tax references, the table changes 30 and moves 21 by more than $1 (-$2.96 to +$3.97). It also moves two scored federal refundable credit references. No scored state output changes.
What models answer	On the 21 moved cells, 173 of 987 answers are within $1 of the schedule value and 18 within $1 of the table value, 16 of them within $1 of both. Neither of the two table-only answers mentions a table.

Leaderboard impact

Convention	Exact-rate change, all models	Models gaining / losing	Exact ranks changed	GPT-6 Sol
(a) rate schedule (published)				95.58%
(b) accept either value	0 to +0.23 points	2 / 0	2	95.58%
(c) table values as the reference	-3.25 to +0.21 points	1 / 21	17	92.33%
for comparison: stop scoring the 23 moved outputs	+1.05 to +4.22 points	47 / 0	24	96.64%

Each case is scored by the repo's scorer (policybench.cli analyze and weighted_hit_rate_scores_by_model) and by an independent re-aggregation that imports nothing from policybench. They agree to 2.8e-14 points, and the unchanged copy reproduces the published payload.

The decision is what the output means

The audit shows what the references compute, and what the same computation gives when each look-up under $100,000 uses the constructed table. It does not establish what any household's filed return would show: that also depends on two methods not read (Form 8615 and Schedule J), on whether the filer rounds the return, and on a 2026 table published only as a draft. Three readings of the output are open:

1. The tax on the return. Where a return under $100,000 is figured through line 16 or a capital gain worksheet, the instructions give the table amount. On that basis 23 scored references differ from it by more than $1. Those counts are sensitivities of the engine's computation, not verified filing amounts. Option (c) follows, or (b) while the IRS's 2026 table is a draft.
2. The tax computed from the section 1 rates, to the cent. The references are right as they stand, and the definition has to be stated. Option (a).
3. What a careful reader could compute from law published before the freeze. Both values are defensible. Option (b), or exclusion.

Recommendation

Reading 2: keep the rate schedule and state the definition in the card, the paper and the v2 prompt (#165). This is a judgment about what the benchmark should measure, not a finding of law. The schedule needs no convention beyond the published rates, where the table adds its bands, a midpoint rule no document read states, and a choice about rounding the looked-up amount (which moves scenario_042 by $6). PolicyBench's amounts are computations to the cent, which a return need not be. It changes no score.

Its cost: the current prompt does not say which computation is wanted. The line 16 instructions say a filer under $100,000 "must use the Tax Table", and where that governs a household's return, an answer that follows it is scored a miss (21 federal cells in the sweep). The README gives the case for each other option and what each would take.

Invariants

For the constructed table, at every taxable income under $100,000, filing status column and year (property tests, tests/test_tax_table_audit.py):

• it is a whole, non-negative number of dollars, and zero at zero;
• it never falls as taxable income rises, and is constant inside a band;
• it is the schedule's tax at the band's midpoint, a half dollar rounding up;
• it is within half a band-width times the band's top marginal rate, plus the half dollar of rounding, of the schedule's tax on the income, and so within one band-width times that rate;
• joint ≤ head of household ≤ single, and separate equals single;
• at $100,000 or more the instructions' method is the schedule exactly.

Differential tests:

• the constructed table equals the IRS's 2025 table and its 2026 draft in every amount, and no other simple rule does;
• the independent scorer agrees with the repo's scorer on generated runs (exclusions, missing answers, a second accepted value) and reproduces the published headline on the release's own files;
• the recorded sweep agrees with the construction on every return, and the recorded baseline is the release's reference on all 1,926 scored outputs.

No counterexample was found. The mutation run kills 34 of 34 deliberate faults. The first run left two standing, both now held: a wrong top rate, which a random property test misses in 13 of 40 seeded runs with the income generator first used (scripts/top_rate_detection.py), and a parser that read past the table.

States

A scoped follow-up, not an audit: 12 of the 21 states that hold a nonzero scored state income tax reference prescribe a tax table (8 make it mandatory), covering 16 of 31 such references. Every quotation was checked against the state's own document. Nothing is recomputed.

Review

An independent GPT-6.1 Sol review found the numbers correct and asked, over two rounds, for changes to the claims (commits b2db187 and 1d3d4f9): the return-level statements made conditional, with what the households' facts say about Form 8615 and Schedule J; "none found" in place of "none existed" for a pre-freeze 2026 table, with the search stated; the recommendation framed as a choice of definition; the instructions' other methods listed; the rounding variant fixed at the ceiling and relabelled; the states section's inference from cents removed.

Test plan

• uv run pytest tests/test_tax_table_audit.py: 54 passed locally.
• uv run ruff check . and uv run ruff format --check .: clean.
• CI runs the full suite.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policybench-site	Ready	Preview	Oct 10, 2026 8:08pm UTC

[MaxGhenis]

Review record before merge (head 334c5f2d)

Independent review: GPT-6.1 Sol through Subfleet (--task review --tier hard), five rounds. It reconciled every number in the README against verification/ and law/ in rounds 1 to 4 and asked for changes to claims, not to results.

Round	Head	Verdict	What it asked for
1	3fe3a115	request changes	"None existed" before the freeze was stated without the evidence for it (blocking); the recommendation leaned on Form 1040-ES and on how models answer; "every filer and every look-up" overstated the worksheets; the rounding variant kept the unrounded schedule tax at the ceiling; cents in state outputs proved nothing; five smaller points
2	b2db1871	request changes	All eleven resolved. The new decision section presented the sweep as filing liability; a match count was read as a model's interpretation; one summary field selected the wrong returns
3	1d3d4f9f	request changes	One sentence still inferred interpretation from matches; the living-parent condition lacked its excerpt
4	4d68b4d0	request changes	A summary sentence I had added misstated who files Form 8615
5	334c5f2d	approve	No findings. (A first attempt at this round could not read files in its lane and gave no verdict; it was rerun with the files in the brief.)

One point of round 1 was not accepted, and the reviewer agreed in round 2: line 5 of the Foreign Earned Income Tax Worksheet does send its look-up to the Tax Table, so the sweep's code stands. The passage is now in law/excerpts.md.

What the review could not check, in its own words: the source documents and archived pages are not in the repository, so the excerpts, the 251 listed drafts and the absence of an earlier 2026 table cannot be authenticated from the pull request alone; Form 8615, Schedule J and complete filed-return amounts remain unchecked; it ran no script or test.

Gates at this head: gh pr checks exits 0 (Vercel, app, lint, test), mergeable and clean, no review requesting changes. The pull request changes no reference, exclusion, score, payload, card, paper or snapshot. The choice of convention is cos decision d1263 and is not ruled.

The reviews and the author's responses are kept on the maintainer's machine under ~/reviews/policybench-tax-table-2026-10-10/lanes/.