Skip to content

Read IRS SOI district and Historic Table 2 measures from the columns their guides define - #345

Merged
MaxGhenis merged 4 commits into
mainfrom
fix/irs-cd-ht2-column-mappings
Oct 12, 2026
Merged

MaxGhenis merged 4 commits into
mainfrom
fix/irs-cd-ht2-column-mappings

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The IRS documentation guide for each SOI geographic file (Historic Table 2, congressional district, county) defines every CSV column. Four Chronicle measures read a column that the guide defines as a different item. This PR points each one at the column its guide defines, and adds a regression test that pins every measure_id in these packages to its guide definition.

# Packages Measure ids Old columns (guide definition) New columns (guide definition) TY2022 US, old → new
1 soi-congressional-district-2022 limited_state_local_taxes_returns / _amount N18425/A18425, "State and local income taxes", Schedule A:5a N18460/A18460, "Limited state and local taxes", Schedule A:5e 10,842,580 → 14,777,040 returns; $250.4B → $121.2B
2 same premium_tax_credit_returns / _amount N85530/A85530, "Additional Medicare tax", 8959:24 N85770/A85770, "Total premium tax credit", 8926:24 (8962:24 in the TY2023 guide) 6,848,330 → 7,713,000 returns; $14.2B → $53.1B
3 same interest_paid_deduction_* → mortgage_interest_paid_* N19300/A19300, "Mortgage interest paid", Schedule A:8a, under Table 2.1's line 10 ids same columns, under Table 2.1's line 8a ids, concepts and labels; plus new 8b, 8c and 9 measures values unchanged ($140.1B)
4 soi-historic-table-2, soi-historic-table-2-state-broad-2022 qbi_claims / qbi_amount N03270/A03270, "Self-employment health insurance deduction", Schedule 1:17 N04475/A04475, "Qualified business income deduction", 1040:13 3,711,990 → 25,430,690 returns; $31.3B → $215.7B
  • Items 1 and 2: Historic Table 2 already read N18460 and N85770 for the same measure ids. The column letters, source_column_id and expected_column_header guards all move (BX/BY → CF/CG, EX/EY → DV/DW).
  • Item 3: Table 2.1 uses interest_paid_deduction_* for the Schedule A line 10 total. The 2022 Schedule A computes line 10 as lines 8a, 8b, 8c and 9 combined (8d is reserved), and no geographic file publishes line 10. The district line 8a columns now carry Table 2.1's line 8a ids. The other three parts are added with Table 2.1's ids, concepts and labels:
    • home_mortgage_personal_seller_*: N19500/A19500, line 8b;
    • deductible_points_*: N19530/A19530, line 8c;
    • investment_interest_paid_*: N19570/A19570, line 9.
  • Item 4: Microcosm maps no Historic Table 2 qbi_* measure, so no target moves.

What the IRS files define

Guides. Each guide's section G table is extracted to tests/data/irs_soi_doc_guides/<guide>.tsv, with the irs.gov URL, sha256 and extraction method in its header. I downloaded each file from irs.gov today; the bytes match the copies in ~/PolicyEngine/_reviews/cg-returns-control-20260925/irs/.

Guide Documents sha256 N/A variables
22incddocguide.docx 22incd.csv (congressional district TY2022) 4e265d3c… 145
22incmdocguide.doc 22in55cmcsv.csv (Historic Table 2 TY2022) d99db44b… 145
23incmdocguide.doc 23in55cmcsv.csv (Historic Table 2 TY2023) 94279069… 147
22incydocguide.docx 22incyallnoagi.csv (county TY2022) a6acb748… 145
  • Every N/A column in those four CSVs is defined in its guide, except A00101 in 22incd.csv and 22incyallnoagi.csv, which no package reads.
  • The four TY2022 guides agree on every shared variable. The TY2023 guide changes the capitalization of N/A18425 and N/A18450, rewords N/A11070 ("additional child tax credit"), and prints the premium tax credit lines as 8962:24 and 8962:25 where the TY2022 guides print 8926.

Schedule A (2022) (f1040sa--2022.pdf from irs.gov, sha256 1e2e4ee7…):

  • line 5e: "Enter the smaller of line 5d or $10,000 ($5,000 if married filing separately)";
  • line 8e: "Add lines 8a through 8c" (8d is "Reserved for future use");
  • line 10: "Add lines 8e and 9".

Publisher cells, TY2022 US (district STATE=US, agi_stub=0; Historic Table 2 STATE=US, AGI_STUB=0; Table 2.1 row "All returns, total"):

Check Value
District A19300 vs Table 2.1 line 8a (mortgage_interest_paid_amount) $140.1B vs $142.4B (0.984)
District A19300 vs Table 2.1 line 10 (interest_paid_deduction_amount) $140.1B vs $170.5B (0.822)
District A19300 + A19500 + A19530 + A19570 vs Table 2.1 line 10 $163.7B vs $170.5B (0.960)
Historic Table 2, the same four parts, vs Table 2.1 line 10 $169.2B vs $170.5B (0.993)
District A18460 vs Table 2.1 limited SALT $121.2B vs $121.7B (0.996)
Limited SALT average per return, old district column A18425/N18425 $23,098, above the $10,000 line 5e cap
Highest limited SALT average in any district, state or AGI-band row (new columns) $9,820 (Historic Table 2, $1M+)
District / Historic Table 2 US ratio, all 44 shared measures (new columns) 0.92–0.99
The same ratio on the old columns SALT amount 2.04, SALT returns 0.72, PTC returns 0.87, PTC amount 0.26; district QBI deduction 6.65× HT2's old qbi_amount

Chronicle effects (measured)

Row-level diff. I built build-bundle --year 2023 over all 20 IRS SOI packages on main and on this branch (41,083 → 43,963 facts) and compared consumer rows on source_record_id:

  • Values change on 480 district limited_state_local_taxes_* rows per measure and 480 premium_tax_credit_amount rows. premium_tax_credit_returns changes on 479: in TN-05, N85530 and N85770 are both 20,780. Values also change on the 62 Historic Table 2 qbi_* rows per measure (11 national, 51 state). On all of these, layout.source_column_id and lineage.source_cell_keys change with the value.
  • The 960 district interest_paid_deduction_* rows become mortgage_interest_paid_* rows with the same values. Their source_record_id, measure_id, concept, label and keys change.
  • 2,880 rows are added: the six 8b, 8c and 9 measures at the district file's 480 rows.
  • layout.record_set_spec_hash changes on every row of the three edited packages, because it hashes the whole record-set spec. No other field changes.

Bundle counts. Measured with the same 20-package build:

  • +2,880 facts: irs_soi by source, tax_unit by entity, tax_year:2022 by period, and the "Congressional District Data 2022" table count (26,880 → 29,760).
  • +6 facts at each district-package geography.
  • No new geography, period or entity keys. Semantic duplicates stay at 1,671 and warnings at 3.
  • The default-bundle pins in tests/test_chronicle_bundle.py move by those amounts: fact and row count 1,004,254 → 1,007,134; US 2,110 → 2,116; California 249 → 255; CA-01 56 → 62. CI's bundle job checks them; the full default bundle doesn't fit on this host.

Package validation. validate_source_package is valid at --year 2022 and 2023 for all three edited packages: 29,760, 605 and 2,703 source records.

Consumers (Microcosm)

I read Microcosm origin/main 51a263726. Its SOI compiler selects facts by measure_id:

  • _measure_id (fiscal_targets.py:4074) reads observed_measure.source_measure_id.
  • _soi_reference_from_fact (:3069) looks that id up in SOI_AMOUNT_MEASURE_VARIABLES / SOI_RETURN_MEASURE_VARIABLES (:113, :199).
  • An id with no entry returns None at :3101, so the fact is skipped.

Nothing changes in Microcosm until its Chronicle feed is re-pinned (us/chronicle_feed.json). This PR does not re-pin it. On the next re-pin:

  • District interest paid (item 3). Today the district interest_paid_deduction_* rows map to interest_deduction (:125, :210). After the rename they are mortgage_interest_paid_*, which Microcosm leaves unmapped on purpose. Its comment at :176 reads: "the two leg measures (mortgage_interest_paid_, home_mortgage_personal_seller_) stay unmapped to avoid double-counting".
    • So on Microcosm main, the district interest targets would drop out.
    • The state_cd bridges for interest_paid_deduction_* (STATE_CD_LEVEL_BRIDGES, tools/us_acs_local_cd_surface.py:90-97) would stop matching.
    • The new 8b, 8c and 9 ids are unmapped too.
    • The share-or-drop decision belongs to the Microcosm session that owns it ("Decide share-or-drop for CD interest-paid targets"). It has said Microcosm will key district interest-paid rows on layout.source_column_id N19300/A19300, compiled as a share, so the rename is safe as long as those columns stay. They do: the 960 rows keep N19300/A19300 and their values.
    • This PR adds no constructed line 10 total. It adds the other three parts under their own columns, and the four amounts sum to line 10.
  • District SALT and PTC (items 1–2). The rows carry N18460/A18460 and N85770/A85770. microcosm#1224's US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS is keyed on (measure id, N18425/A18425/N85530/A85530), so it stops matching them, and the corrected rows compile.
    • Exception: on Microcosm main, _is_soi_cd_premium_tax_credit_amount_conflict (:2953) drops every district premium_tax_credit_amount row, whatever its column. #1224 removes that check.
    • #1224's pinned receipt test (test_pinned_feed_exclusion_receipt, 4 × 488 ids) and its 31,786-spec pin need updating on re-pin.
    • STATE_CD_DEFECTIVE_CD_FILE_MEASURES (tools/us_acs_local_cd_surface.py:53-68) is keyed on measure id alone, so it keeps the corrected rows off state_cd until that register is edited.
  • Historic Table 2 QBI (item 4). Neither measure map has a qbi_* entry, and test_soi_qbi_historic_table_2_measures_are_not_direct_targets guards that. No target moves.

TY2023 Historic Table 2 (#295)

#295 adds TY2023 national, state-broad and state-EITC packages. Its QBI measures already read N04475/A04475. Its test_packages_mirror_their_2022_twins carries a TY2022_QBI_CORRECTION shim for the wrong TY2022 twin, and once this PR lands the twin matches.

Simulation: this branch with #295's packages and tests overlaid.

Chronicle governance

  • Approved agent role: ledger-source-ingestor. Three paths sit outside that role's allowed_paths: docs/concept-migrations.md (the rename note docs/architecture.md requires), changelog.d/ and tests/data/irs_soi_doc_guides/.
  • Deterministic checks run:
    • validate_source_package for the three edited packages at 2022 and 2023: valid.
    • The 20-package SOI bundle on main and this branch: valid, 0 errors, 0 aggregate-duplicate keys, with the row diff above.
    • ruff check chronicle policyengine_chronicle db scripts tests (locked ruff 0.14.10): clean.
    • Local pytest: the new module (16 tests), plus test_chronicle_source_package.py, the SOI and all-package suites listed under Tests.
    • CI on a90ad379: Chronicle bundle passed (1h56m), confirming the default-bundle pins, and Chronicle surface passed (47m). CI reruns on the follow-up head.
  • LLM judge verdicts (independent GPT-6.1 Sol review via Subfleet, read-only lane, of a90ad379; posted below):
    • ledger-source-fidelity: PASS. It re-extracted all four guides with textutil (582/582 TSV rows match), verified the guide and CSV hashes and every changed declaration's header, and built the district facts in memory (29,760).
    • ledger-boundary: PASS. The production change only remaps or relabels publisher cells; the test-only sums create no Chronicle facts.
    • ledger-contract: no schema or consumer-contract change.
    • Overall APPROVE, with one P2 and two P3s, all fixed in d61c069a. The re-review of d61c069a was APPROVE, PASS/PASS, with three P3 coverage gaps, all fixed in c542be25. The confirmation review of c542be25, the current head, is APPROVE with no new findings. It ran all 42,720 compiled records across the seven packages and repeated every mutation.

Invariants (tests/test_chronicle_soi_doc_guide_columns.py)

Every invariant below is checked by enumerating every package, measure and row, not by sampling.

  1. Guide extracts are complete. Every N/A column in each data file a package reads is defined in that file's guide, except a listed set (A00101). Every guide variable is a column of its file.
  2. Definitions name one column. Within a guide, no two variables share a description (case-insensitive). Every GUIDE_DEFINITIONS entry names exactly one variable, and the same one in every guide.
  3. Every compiled selector reads the column its definition names. This covers every cell selector compiled by every irs_soi package that parses a delimited IRS-variable file, in any vintage. It reads the compiled source-record specs, so row-level column overrides, row ranges and row-level header guards count as the build reads them.
    • The header at its column letter must be the variable the measure's guide definition names.
    • source_column_id and expected_column_header, where declared, must equal that header.
    • The guide's form line must match the pinned line, after the one recorded erratum, 8926 → 8962.
    • The declared column id is taken from the row or measure declaration, row first, as the build resolves it. An explicit id equal to the measure id is checked like any other.
    • A header guard must sit on the file's header row, a row range must stay in one column, and a divisor column fails, because these packages publish levels.
    • A measure id must carry one concept and label within a package.
    • Fails closed: an unknown measure_id, an undocumented column, a data file with no guide, or a package without artifact_year each fail.
    • Mutation tests run on a temp copy of historic_table_2, and the check names each one. They cover a row moved to column T (A00100), an explicit column id equal to the measure id, a divisor, and a second record set with another qbi_amount concept. The reviewer's own mutations were caught the same way: an Autauga County row moved to column V with matching guards, a county divisor, and a duplicated district record set with the old line 10 concept.
  4. Shared measure ids read one column across packages.
  5. Line 10 ids stay off the geographic files. interest_paid_deduction_* may not be read from them and stays declared by Table 2.1.
  6. The district interest parts match Table 2.1. Concepts and labels equal Table 2.1's for the same ids.
  7. Differential: the district US row is a subset of Historic Table 2's. For all 44 shared measures (QBI paired by IRS variable), district / Historic Table 2 is in [0.90, 1.00].
  8. Differential: line 8a and line 10. District line 8a is 0.984 of Table 2.1's line 8a and under 0.85 of its line 10, and the four parts are 0.960 of line 10.
  9. Property: the line 5e cap. In every district, state, national and AGI-band row of the four packages that carry limited SALT (Table 2.1 at TY2022 and TY2023), amount ≤ $10,000 × (returns + 5) + $500. That allows for counts rounded to the nearest ten and amounts in thousands.

On main the module fails 12 of its 20 tests. Invariant 3 names all ten wrong declarations in the guide's wording, and the line 5e cap fails on the district package independently of the YAML pins.

Not changed here

  • Unguarded Historic Table 2 national measures. 13 measures in historic_table_2 declare only a column letter. Their consumer rows carry the measure id as layout.source_column_id (for example premium_tax_credit_returns). The new test resolves their letters against the file's header, so a column shift fails CI. Adding source_column_id guards would change those rows' layout.source_column_id, which microcosm#1224 reads, so it belongs with a Microcosm re-pin.
  • Concept splits on shared measure ids. chronicle#307 lists them. One more split: charitable_* (district irs_soi.charitable_contributions_deduction vs Table 2.1 irs_soi.contributions_deduction, both Schedule A line 14). Also ctc_* reads N07225, which the guides define as "nonrefundable child and other dependent tax credit", under the concept irs_soi.child_tax_credit. Noted on IRS SOI measure_id net_capital_gains_* names two different measures (Form 1040 line 7 vs Table 1.4 Schedule D gain) #307.
  • Microcosm. No feed re-pin and no mapping change; see Consumers.

Tests

  • New: tests/test_chronicle_soi_doc_guide_columns.py (20 tests) and tests/data/irs_soi_doc_guides/*.tsv (4 guide extracts).
  • Updated:
    • test_soi_congressional_district_2022_builds_all_return_facts: 29,760 facts, plus the corrected SALT, PTC and line 8a values and columns, the new parts, and no interest_paid_deduction_*.
    • The Historic Table 2 national and state-broad QBI values: $215,679,578k and CA $25,995,054k, from A04475.
    • The default-bundle count pins.
  • Local runs (load ~270, serial): the new module, 20 passed on c542be25. test_chronicle_source_package.py, test_source_package_alias_drift.py, test_chronicle_artifact_year_restamp.py, test_issue_265_non_uk_labels.py, test_chronicle_issue_272.py, test_chronicle_governance.py, test_chronicle_facts_only.py, test_chronicle_boundaries.py, test_chronicle_soi_targets.py, test_etl_soi.py, test_etl_soi_state.py, test_us_poverty_target_coverage.py, test_issue_261_dimension_labels.py and test_chronicle_source_label_authority.py: 323 passed. The default bundle and the full suite run in CI.

Review follow-ups (d61c069a)

  • P2, row-level overrides. The check read only measure.column, but the build reads row.column or measure.column and row-level guards. The test now walks the compiled source-record specs, and adds the mutation test above.
  • P3, SALT cap rounding. The bound now allows the guides' rounding.
  • P3, guide section letters. The Historic Table 2 guides call the variable table section F in their contents and G in their body. The extracts now name it by title.

Re-review follow-ups (c542be25)

  • P3, explicit column ids. An explicit source_column_id equal to the measure id looked like the layout fallback. The declared id now comes from the compiled record set.
  • P3, dedup. Concept and label are in the key, and conflicting concepts or labels for one measure id fail.
  • P3, divisors. Divisor selectors were not inspected. They now fail.

axiom: n/a: source-data column mapping, no policy rule change.

🤖 Generated with Claude Code

The IRS documentation guides define each geographic-file column. Four
mappings read a column the guide defines as something else:

- congressional_district_2022 limited_state_local_taxes_* read N18425/A18425
  (state and local income taxes, Schedule A line 5a); now N18460/A18460
  (limited state and local taxes, line 5e), as Historic Table 2 reads.
- premium_tax_credit_* read N85530/A85530 (additional Medicare tax, Form 8959
  line 24); now N85770/A85770 (total premium tax credit).
- interest_paid_deduction_* (Table 2.1's line 10 total) read N19300/A19300,
  line 8a mortgage interest. Renamed to Table 2.1's line 8a ids and concepts
  (mortgage_interest_paid_*), with new line 8b, 8c and 9 measures.
- historic_table_2 and historic_table_2_state_broad_2022 qbi_* read
  N03270/A03270 (self-employed health insurance); now N04475/A04475.

tests/test_chronicle_soi_doc_guide_columns.py pins every measure id these
packages declare to its guide definition (extracts in
tests/data/irs_soi_doc_guides), with differential checks against Historic
Table 2 and Table 2.1 and the $10,000 line 5e cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nsumers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit to PolicyEngine/microcosm that referenced this pull request Oct 12, 2026
PolicyEngine/chronicle#345 relabels the district file's line 8a column
to mortgage_interest_paid_* and adds lines 8b, 8c and 9 under Table
2.1's own ids. The column-keyed rule binds the relabeled 8a rows
unchanged and the new ids stay unmapped, so nothing double-binds.
Also drop an unused import.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-ups (GPT-6.1 Sol, APPROVE with one P2 and two P3s):

- The guide check read only measure.column. The build reads
  row.column or measure.column and row-level header guards, so the test now
  walks the compiled source-record specs. Row ranges must stay in one column,
  and a header guard must sit on the file's header row. A new mutation test
  moves one Historic Table 2 row to column T and expects the check to name it.
- The $10,000 line 5e bound now allows the guides' rounding: counts to the
  nearest ten, amounts to thousands.
- The guide extracts name the "Selected Income and Tax Items" table by title.
  The Historic Table 2 and county guides call it section F in their contents
  and G in their body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Independent review (GPT-6.1 Sol via Subfleet, read-only lane; job 20261012-103056-chronicle-345-review) of head a90ad379. The P2 and both P3 findings are addressed in d61c069a; see "Review follow-ups" in the description.

ledger-source-fidelity: PASS
ledger-boundary: PASS
Overall: APPROVE — no P0 or P1 findings.

Reviewed requested commit a90ad379 against a5280e18. The checkout’s later commit, da557c9c, changes only documentation; I excluded that amendment.

  1. P2 should-fix — Guide checks miss row-level column overrides.
    tests/test_chronicle_soi_doc_guide_columns.py:392 checks only measure["column"]. The compiler instead resolves row.column or measure.column and permits row-specific header guards.

    Evidence: An executed, in-memory mutation gave Autauga County column: V, source_column_id: A00100, and matching header guards. All nine structural guide/header/id checks still passed. Compiling that same mutated payload produced:

    county_01001.return_count
    reads V2, expects A00100
    concept irs_soi.individual_income_tax_returns
    

    Thus AGI can be selected as return counts without detection. The remaining numeric checks never build county facts. Current YAML contains no such override.

    Fix: Validate the effective selector and provenance fields for every row/measure pair, or explicitly reject row-level overrides in these packages.

  2. P3 nit — Exact SALT-cap assertion omits publisher rounding.
    tests/test_chronicle_soi_doc_guide_columns.py:707 requires amount <= 10_000 * returns, although the guides round return counts to tens.

    Evidence: Twenty-four actual claimants each deducting $10,000 produce $240,000, but their published count rounds to 20. The assertion then rejects valid underlying deductions. The legal cap applies per actual return. IRS Schedule A, TY2022

    Fix: Allow documented count and amount rounding bounds, or describe this as a regression check for these particular datasets. Current fixtures pass with sufficient margin.

  3. P3 nit — Three fixture headers identify the wrong guide section.
    22incmdocguide.tsv:1, 23incmdocguide.tsv:1, and 22incydocguide.tsv:1 identify “section G (Selected Income and Tax Items).”

    Evidence: All three original guides identify that section as F; G is Endnotes. Their extracted variable definitions are correct.

    Fix: Change those three headers to section F. The congressional-district header correctly names G.

Executed verification:

  • 16 passed in the new regression suite; 3 passed in the affected CD, national HT2, and state-broad source-package tests. Used the existing virtual environment with PYTHONDONTWRITEBYTECODE=1, -s, and -p no:cacheprovider to accommodate read-only execution.
  • Verified registered CSV hashes and every changed declaration’s actual column header against source_column_id and expected_column_header.
  • Verified all four original guide hashes. Re-extracted their text with textutil; 582/582 TSV rows matched, including 60 explicit spot checks covering every requested variable. No dropped, merged, or altered definitions were found.
  • Confirmed total PTC matches HT2’s N85770/A85770 and Form 8962 line 24; advance and net PTC are distinct lines 25 and 26. IRS Form 8962, TY2022
  • Built CD facts in memory: 4,791 source rows, 79,365 source cells, 29,760 facts. Confirmed every requested bundle-count edit follows +2,880, with +6 at each CD-package geography.
  • Searched tests for the specified stale counts, values, IDs, and columns; remaining occurrences are valid historical explanations or unaffected tests.
  • Read the cited Microcosm compiler and bridge code at 51a263726 and local origin/main; also inspected PR #1224’s actual diff. The consumer claims are supported.

Static assessment: unknown IDs, missing guides, new filenames/vintages, and undocumented columns fail closed for discovered delimited_text_full_rows packages, subject to finding 1. The ratio and 0.984/0.960 checks compare independent publisher tables and are not tautological. No encoding or resource-path failure occurred.

The full persisted bundle test was not executed. Duplicate-key and warning claims were assessed from identity construction and unchanged warning inputs. The production changes only remap or relabel publisher cells and normalize their units; test-only comparisons and sums do not create computed Chronicle facts.

Re-review of d61c069 (GPT-6.1 Sol, APPROVE, three P3s):

- An explicit source_column_id equal to the measure id looked like the
  layout's fallback. The check now takes the declared id from the compiled
  record set (row before measure), so it is checked against the header.
- Deduplication could drop a selector whose concept or label differed. Both
  are now in the key, and a measure id with more than one concept or label
  in a package fails.
- Divisor selectors were not inspected. These packages publish levels, so a
  divisor column now fails until the test checks divisors.

The mutation test is parametrized over all four cases on a temp copy of
historic_table_2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Re-review (GPT-6.1 Sol via Subfleet, read-only lane; job 20261012-142858-chronicle-345-rereview) of a90ad379..d61c069a. Its three P3s are addressed in c542be25; a confirmation review of that head is running.

ledger-source-fidelity: PASS
ledger-boundary: PASS
Overall: APPROVE — no P0 or P1 findings.

Reviewed a90ad379..d61c069a. All three prior findings are resolved: the county row override is detected, the SALT bound allows rounding, and the TSV headers identify the table by title.

Three nonblocking coverage issues remain:

  1. P3 — Explicit column IDs can be mistaken for compiler fallbacks.
    Line 418 discards any source_column_id equal to measure_id.

    Evidence: Setting county return_count’s explicit source_column_id: return_count produced zero failures, although its selector reads F2, whose header is N1. The former declaration-based check rejected this.

    Fix: Preserve whether the effective row/measure ID was explicitly declared; ignore only an actual compiler fallback.

  2. P3 — Deduplication hides conflicting concepts and labels.
    Lines 442–443 silently discard later reads with the same key, even when their concept or label differs.

    Evidence: An executed mutation appended a distinct district record set with the same interest columns but an incorrect mortgage-interest concept and label. 16 compiled records became eight reads; guide checks and the district concept/label test still passed. The former equality assertion caught this conflict.

    Fix: Assert that concept and label agree on dedup collisions. Adding them to the key alone is insufficient because the district consistency test also collapses reads by measure ID.

  3. P3 — “Every compiled cell selector” overstates divisor coverage.
    Line 404 inspects spec.selector but omits spec.divisor_selector.

    Evidence: Adding county return_count.divisor_column: V produced zero failures, while compiling return counts from F2 divided by AGI from V2. This gap predates the follow-up, but the new completeness claim is inaccurate.

    Fix: Assert that these level measures have no divisor selector, or validate divisors separately and adjust the documentation.

Executed verification:

  • 16 pytest tests passed; the temporary-file mutation test was separately executed and passed with its YAML held in memory because the checkout is read-only.
  • The original Autauga mutation compiled return_count from V2 with matching A00100 guards and was correctly rejected: the guide defines return counts as N1.
  • A header guard on row 2 was rejected. Same-column ranges work, and range-column checks occur before deduplication.
  • The mutation test is meaningful: it mutates YAML, uses production compilation, and compares against independent CSV headers and pinned guide definitions.
  • Consumer wording is supported; the compiled SALT maximum is $9,819.80, supporting the stated $9,820.

One wording correction: Historic Table 2’s contents and body disagree on F/G; the cached county guide uses F in both. The committed title-only headers remain accurate.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Confirmation review (GPT-6.1 Sol via Subfleet, read-only lane; job 20261012-144603-chronicle-345-confirm) of d61c069a..c542be25, the current head.

ledger-source-fidelity: PASS
ledger-boundary: PASS
Overall: APPROVE

Reviewed d61c069a..c542be25. All three P3s are resolved. No new findings.

  • Repeated county explicit-ID, divisor, and Autauga AGI-override mutations were rejected. District concept-only, label-only, and combined conflicts were also rejected.
  • Verified all 42,720 compiled records across seven packages: every row/measure lookup resolves, declaration precedence matches the compiler, and meanings are consistent. No new false positives or false negatives found.
  • Exact label equality restores the original audit’s package-specific contract. Contextual label variants permitted by the generic schema could trigger it and would require extending the check.
  • 20/20 tests passed: 16 normally; four mutation tests with memory-backed temporary YAML because the checkout is read-only.

@MaxGhenis
MaxGhenis marked this pull request as ready for review October 12, 2026 19:16
@MaxGhenis
MaxGhenis merged commit 8826247 into main Oct 12, 2026
3 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged as 8826247b (merge commit) at reviewed head c542be25. Gates at merge: gh pr checks exited 0 on run 38293122461 (bundle 1h38m, surface 44m, checks); MERGEABLE; no CHANGES_REQUESTED review. Three independent GPT-6.1 Sol reviews approved, the last at this head with no findings (ledger-source-fidelity PASS, ledger-boundary PASS). Follow-ups: #295 should drop its TY2022_QBI_CORRECTION shim (patch in #295 (comment)); #350 rebases onto main; Microcosm's feed re-pin picks up the corrected columns (see Consumers above).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant