Read IRS SOI district and Historic Table 2 measures from the columns their guides define

changelog.d/irs-soi-doc-guide-columns.fixed.md

@@ -0,0 +1,9 @@
+Read four IRS SOI measures from the columns the IRS documentation guides
+define. In the congressional-district package, limited SALT now reads
+`N18460`/`A18460` (Schedule A line 5e), not 5a's `N18425`/`A18425`. The
+premium tax credit now reads `N85770`/`A85770`, not the additional Medicare
+tax `N85530`/`A85530`. The line 8a mortgage-interest columns are renamed from
+`interest_paid_deduction_*` to Table 2.1's `mortgage_interest_paid_*`, next to
+new line 8b, 8c and 9 measures. Historic Table 2 `qbi_*` now reads the QBI
+deduction `N04475`/`A04475`, not `N03270`/`A03270`. A regression test pins
+every measure id in these packages to its guide definition.

docs/concept-migrations.md

@@ -74,3 +74,78 @@ and the congressional-district file still label with different concepts.
 concept, only Table 1.4 may declare the Schedule D gain concept, the retired
 ids may not reappear, and the consumer selector (`source_measure_id`) and
 published values are unchanged.
+
+## 2026-10-12: IRS SOI geographic-file columns follow their documentation guides
+
+Four IRS SOI mappings read a column that the file's documentation guide defines
+as a different item. Extracts of the guides are in
+`tests/data/irs_soi_doc_guides/`, each with its irs.gov URL and sha256.
+
+### Column corrections
+
+Measure ids, concepts and labels stay. Values, `layout.source_column_id` and
+the cell lineage change.
+
+| Packages | Measure ids | Old columns | New columns | TY2022 US, old → new |
+|---|---|---|---|---|
+| `soi-congressional-district-2022` | `limited_state_local_taxes_returns` / `_amount` | `N18425`/`A18425`, "State and local income taxes" (Schedule A line 5a) | `N18460`/`A18460`, "Limited state and local taxes" (line 5e) | 10,842,580 → 14,777,040 returns; $250.4B → $121.2B |
+| same | `premium_tax_credit_returns` / `_amount` | `N85530`/`A85530`, "Additional Medicare tax" (Form 8959 line 24) | `N85770`/`A85770`, "Total premium tax credit" | 6,848,330 → 7,713,000 returns; $14.2B → $53.1B |
+| `soi-historic-table-2`, `soi-historic-table-2-state-broad-2022` | `qbi_claims` / `qbi_amount` | `N03270`/`A03270`, "Self-employment health insurance deduction" (Schedule 1 line 17) | `N04475`/`A04475`, "Qualified business income deduction" (Form 1040 line 13) | 3,711,990 → 25,430,690 returns; $31.3B → $215.7B |
+
+Historic Table 2 already read `N18460` and `N85770` for the same measure ids.
+The TY2022 guides print the premium tax credit's form as "8926"; the TY2023
+Historic Table 2 guide prints 8962 for the same variables.
+
+### Schedule A line 8a relabel
+
+| Packages | Columns | Old measure id / concept | New measure id / concept |
+|---|---|---|---|
+| `soi-congressional-district-2022` | `N19300` | `interest_paid_deduction_returns` / `irs_soi.returns_with_interest_paid_deduction` | `mortgage_interest_paid_returns` / `irs_soi.returns_with_home_mortgage_interest_paid_to_financial_institutions` |
+| same | `A19300` | `interest_paid_deduction_amount` / `irs_soi.interest_paid_deduction` | `mortgage_interest_paid_amount` / `irs_soi.home_mortgage_interest_paid_to_financial_institutions` |
+
+Labels become Table 2.1's: "Returns with home mortgage interest paid to
+financial institutions" and "Home mortgage interest paid to financial
+institutions amount". Unlike the 2026-09-27 rename, `measure_id` and
+`source_record_id` move too, so the consumer selector moves.
+
+**Why.** The guide defines `N19300`/`A19300` as mortgage interest paid,
+Schedule A line 8a. `interest_paid_deduction_*` is Table 2.1's line 10 total,
+and the 2022 Schedule A computes line 10 as lines 8a, 8b, 8c and 9 combined
+(line 8d is reserved). No geographic file publishes line 10. For TY2022 the
+district file's US `A19300` is $140.1B, 0.984 of Table 2.1's line 8a and 0.822
+of its line 10 ($170.5B).
+
+**Added.** The line 8b, 8c and 9 columns, with Table 2.1's ids, concepts and
+labels: `home_mortgage_personal_seller_*` (`N19500`/`A19500`),
+`deductible_points_*` (`N19530`/`A19530`) and `investment_interest_paid_*`
+(`N19570`/`A19570`). The four amounts sum to line 10: $163.7B for the TY2022
+US row, 0.960 of Table 2.1's. Return counts do not add, because one return can
+report more than one line.
+
+**Consumer effect.** Microcosm's SOI compiler (main, 2026-10-12) selects facts
+by `measure_id` and skips ids it does not map. On its next Chronicle feed
+re-pin:
+
+- The district line 8a rows stop mapping to `interest_deduction`. Microcosm
+  leaves `mortgage_interest_paid_*` unmapped on purpose, so those district
+  targets drop, and its `state_cd` bridge entries for `interest_paid_deduction_*`
+  stop matching. The 8b, 8c and 9 ids are unmapped too. How to use the
+  district rows is Microcosm's decision; the line 8a rows keep
+  `layout.source_column_id` `N19300`/`A19300`, so a selector keyed on the IRS
+  column still finds them.
+- The district SALT and PTC rows carry the corrected columns.
+  microcosm#1224's register keyed on `N18425`/`A18425` and `N85530`/`A85530`
+  stops matching them, and its pinned receipt test needs updating.
+- Microcosm maps no Historic Table 2 `qbi_*` measure, so the QBI fix moves no
+  Microcosm target.
+
+**Pinned by** `tests/test_chronicle_soi_doc_guide_columns.py`. It maps every
+`measure_id` declared by a package that reads an IRS-variable file (Historic
+Table 2, congressional-district or county, any vintage) to one guide
+definition. It then checks that every compiled cell selector reads the column
+that definition names. Row-level column overrides and header guards count, as
+the build reads them. Measure ids shared across packages must read one column.
+The line 10 ids may not be read from a geographic file. Two differential checks
+compare published values: the district US row against Historic Table 2's, and
+the line 8a column and four-part sum against Table 2.1. A property check holds
+every limited SALT row to the $10,000 line 5e cap, within publisher rounding.

packages/irs_soi/congressional_district_2022/source_package.yaml

@@ -14322,21 +14322,21 @@ record_sets:
   - measure_id: limited_state_local_taxes_returns
     label: Returns with limited state and local taxes
     ordinal: 30
-    column: BX
-    source_column_id: N18425
+    column: CF
+    source_column_id: N18460
     expected_column_header_row: 1
-    expected_column_header: N18425
+    expected_column_header: N18460
     concept: irs_soi.returns_with_limited_state_local_taxes
     unit: count
     aggregation: sum
     expected_cell_type: number
   - measure_id: limited_state_local_taxes_amount
     label: Limited state and local taxes
     ordinal: 31
-    column: BY
-    source_column_id: A18425
+    column: CG
+    source_column_id: A18460
     expected_column_header_row: 1
-    expected_column_header: A18425
+    expected_column_header: A18460
     concept: irs_soi.limited_state_local_taxes
     unit: usd
     aggregation: sum
@@ -14388,25 +14388,25 @@ record_sets:
     aggregation: sum
     value_scale: 1000
     expected_cell_type: number
-  - measure_id: interest_paid_deduction_returns
-    label: Returns with deductible interest paid
+  - measure_id: mortgage_interest_paid_returns
+    label: Returns with home mortgage interest paid to financial institutions
     ordinal: 36
     column: CJ
     source_column_id: N19300
     expected_column_header_row: 1
     expected_column_header: N19300
-    concept: irs_soi.returns_with_interest_paid_deduction
+    concept: irs_soi.returns_with_home_mortgage_interest_paid_to_financial_institutions
     unit: count
     aggregation: sum
     expected_cell_type: number
-  - measure_id: interest_paid_deduction_amount
-    label: Deductible interest paid
+  - measure_id: mortgage_interest_paid_amount
+    label: Home mortgage interest paid to financial institutions amount
     ordinal: 37
     column: CK
     source_column_id: A19300
     expected_column_header_row: 1
     expected_column_header: A19300
-    concept: irs_soi.interest_paid_deduction
+    concept: irs_soi.home_mortgage_interest_paid_to_financial_institutions
     unit: usd
     aggregation: sum
     value_scale: 1000
@@ -14662,23 +14662,92 @@ record_sets:
   - measure_id: premium_tax_credit_returns
     label: Returns with premium tax credit
     ordinal: 54
-    column: EX
-    source_column_id: N85530
+    column: DV
+    source_column_id: N85770
     expected_column_header_row: 1
-    expected_column_header: N85530
+    expected_column_header: N85770
     concept: irs_soi.returns_with_premium_tax_credit
     unit: count
     aggregation: sum
     expected_cell_type: number
   - measure_id: premium_tax_credit_amount
     label: Premium tax credit
     ordinal: 55
-    column: EY
-    source_column_id: A85530
+    column: DW
+    source_column_id: A85770
     expected_column_header_row: 1
-    expected_column_header: A85530
+    expected_column_header: A85770
     concept: irs_soi.premium_tax_credit
     unit: usd
     aggregation: sum
     value_scale: 1000
     expected_cell_type: number
+  - measure_id: home_mortgage_personal_seller_returns
+    label: Returns with home mortgage interest paid to individuals
+    ordinal: 56
+    column: CL
+    source_column_id: N19500
+    expected_column_header_row: 1
+    expected_column_header: N19500
+    concept: irs_soi.returns_with_home_mortgage_interest_paid_to_individuals
+    unit: count
+    aggregation: sum
+    expected_cell_type: number
+  - measure_id: home_mortgage_personal_seller_amount
+    label: Home mortgage interest paid to individuals amount
+    ordinal: 57
+    column: CM
+    source_column_id: A19500
+    expected_column_header_row: 1
+    expected_column_header: A19500
+    concept: irs_soi.home_mortgage_interest_paid_to_individuals
+    unit: usd
+    aggregation: sum
+    value_scale: 1000
+    expected_cell_type: number
+  - measure_id: deductible_points_returns
+    label: Returns with deductible points
+    ordinal: 58
+    column: CN
+    source_column_id: N19530
+    expected_column_header_row: 1
+    expected_column_header: N19530
+    concept: irs_soi.returns_with_deductible_points
+    unit: count
+    aggregation: sum
+    expected_cell_type: number
+  - measure_id: deductible_points_amount
+    label: Deductible points amount
+    ordinal: 59
+    column: CO
+    source_column_id: A19530
+    expected_column_header_row: 1
+    expected_column_header: A19530
+    concept: irs_soi.deductible_points
+    unit: usd
+    aggregation: sum
+    value_scale: 1000
+    expected_cell_type: number
+  - measure_id: investment_interest_paid_returns
+    label: Returns with investment interest expense deduction
+    ordinal: 60
+    column: CP
+    source_column_id: N19570
+    expected_column_header_row: 1
+    expected_column_header: N19570
+    concept: irs_soi.returns_with_investment_interest_expense_deduction
+    unit: count
+    aggregation: sum
+    expected_cell_type: number
+  - measure_id: investment_interest_paid_amount
+    label: Investment interest expense deduction amount
+    ordinal: 61
+    column: CQ
+    source_column_id: A19570
+    expected_column_header_row: 1
+    expected_column_header: A19570
+    concept: irs_soi.investment_interest_expense_deduction
+    unit: usd
+    aggregation: sum
+    value_scale: 1000
+    expected_cell_type: number

packages/irs_soi/historic_table_2/source_package.yaml

@@ -810,25 +810,25 @@ record_sets:
       - measure_id: qbi_claims
         label: Returns with qualified business income deduction
         ordinal: 47
-        column: BF
-        source_column_id: N03270
+        column: CT
+        source_column_id: N04475
         concept: irs_soi.returns_with_qualified_business_income_deduction
         unit: count
         aggregation: sum
         expected_cell_type: number
         expected_column_header_row: 1
-        expected_column_header: N03270
+        expected_column_header: N04475
       - measure_id: qbi_amount
         label: Qualified business income deduction
         ordinal: 48
-        column: BG
-        source_column_id: A03270
+        column: CU
+        source_column_id: A04475
         concept: irs_soi.qualified_business_income_deduction
         unit: usd
         aggregation: sum
         expected_cell_type: number
         expected_column_header_row: 1
-        expected_column_header: A03270
+        expected_column_header: A04475
         value_scale: 1000
       - measure_id: rental_royalty_income_returns
         label: Returns with rental and royalty net income less loss

packages/irs_soi/historic_table_2_state_broad_2022/source_package.yaml

@@ -692,25 +692,25 @@ record_sets:
   - measure_id: qbi_claims
     label: Returns with qualified business income deduction
     ordinal: 45
-    column: BF
-    source_column_id: N03270
+    column: CT
+    source_column_id: N04475
     concept: irs_soi.returns_with_qualified_business_income_deduction
     unit: count
     aggregation: sum
     expected_cell_type: number
     expected_column_header_row: 1
-    expected_column_header: N03270
+    expected_column_header: N04475
   - measure_id: qbi_amount
     label: Qualified business income deduction
     ordinal: 46
-    column: BG
-    source_column_id: A03270
+    column: CU
+    source_column_id: A04475
     concept: irs_soi.qualified_business_income_deduction
     unit: usd
     aggregation: sum
     expected_cell_type: number
     expected_column_header_row: 1
-    expected_column_header: A03270
+    expected_column_header: A04475
     value_scale: 1000
   - measure_id: rental_royalty_income_returns
     label: Returns with rental and royalty net income less loss