Drop congressional-district SALT and PTC facts read from the wrong IRS column

changelog.d/1038-cd-source-column-exclusions.fixed.md

@@ -0,0 +1 @@
+The US fiscal-target compiler drops congressional-district SOI facts read from an IRS column their measure id does not mean, through a new reviewed register keyed by (measure id, IRS column), `US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS`: N18425/A18425 (state and local income taxes, Schedule A line 5a) published as the limited SALT deduction, and N85530/A85530 (additional Medicare tax, Form 8959 line 24) published as the premium tax credit. It replaces the hard-coded congressional-district PTC-amount check, refuses a listed district fact that names no column, and appears in the exclusion receipt and the release coverage report. On the pinned feed the full surface loses the 976 district SALT specs, which summed to 1.98x the national target, and the 488 district PTC-return specs (microcosm#1038).

docs/us-acs-local-soi-target-surface.md

@@ -79,6 +79,20 @@ their role is `soi_fiscal_distribution`, so `totals` is unchanged:
 | **Admin specs** | **4,380** | **760** | **31,474** |
 
 With the 487 population marginals `state` now calibrates to 4,867 targets.
+
+**Since microcosm#1038** the compiler drops the district file's facts read
+from the wrong IRS column (`US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS` in
+`us_runtime/fiscal_targets.py`): the SALT rows read from N18425/A18425
+(state and local income taxes) and the premium-tax-credit rows read from
+N85530/A85530 (additional Medicare tax). `state` and `state_cd` are
+unchanged. `totals` loses the 487 district-file premium-tax-credit return
+rows and `full` loses those and the 974 district-file SALT rows:
+
+| Family | `state` | `totals` | `full` |
+|---|---:|---:|---:|
+| `irs_soi` | 4,227 | 120 | 29,860 |
+| **Admin specs** | **4,380** | **273** | **30,013** |
+
 The rest of this section describes the 2026-09-22 measurement.
 
 The 487 population marginals (51 states and 436 congressional districts) are
@@ -104,8 +118,12 @@ congressional-district file. The feed-gated test
 
 ### What `totals` contains, and what it does not
 
-All 607 SOI specs that `totals` keeps are ACA premium tax credit measures (547
-returns, 60 amounts), 454 of them at congressional-district geography.
+All 607 SOI specs that `totals` kept on 2026-09-22 are ACA premium tax credit
+measures (547 returns, 60 amounts), 454 of them at congressional-district
+geography. Since microcosm#1038 it keeps 120 (60 returns and 60 amounts,
+all Historic Table 2: the 51 states and the 9 at-large district copies), 18
+of them at district geography: the 487 district-file return rows read
+N85530, additional Medicare tax, not the premium tax credit.
 `_soi_target_role` in `us_runtime/fiscal_targets.py` gives named roles only to
 country-level all-income-range facts, the all-income-range PTC measures and
 the W-2 tip items; every other SOI fact gets `soi_fiscal_distribution`,
@@ -292,21 +310,21 @@ bridged state row records its sibling and bridge factor.
 
 | Rows | Specs on the pinned feed | Why |
 |---|---:|---|
-| District-file `limited_state_local_taxes_*`, both geographies | 974 | Column A18425/N18425 is state and local income taxes, not the limited SALT deduction (microcosm#1038) |
-| District-file `premium_tax_credit_returns`, both geographies | 487 | Column N85530 is the additional Medicare tax, not the premium tax credit (microcosm#1038) |
+| District-file `limited_state_local_taxes_*`, both geographies | 974 | Column A18425/N18425 is state and local income taxes, not the limited SALT deduction (microcosm#1038). Since #1038 the compiler drops them first; `STATE_CD_DEFECTIVE_CD_FILE_MEASURES` stays as a second fence |
+| District-file `premium_tax_credit_returns`, both geographies | 487 | Column N85530 is the additional Medicare tax, not the premium tax credit (microcosm#1038); dropped first by the compiler, as above |
 | District `tax_filer_individual_count` | 436 | No state parent in either vintage, so it could not nest in a bound state target |
 | District rows of states with one district on the 117th plan (AK, DE, DC, MT, ND, SD, VT, WY) | 459 | The SOI file has no sub-state rows there. These rows are the state total copied, or for Montana split by population, so they carry no district information |
 | Historic Table 2 rows copied to at-large districts | 360 | The same copies from the other vintage |
 | District-file state rows for concepts Historic Table 2 carries | 2,295 | Second vintage of a state concept |
 | District rows of the four out-of-band rebase blocks | 34 | The factor band (above) |
 | Bridged state rows whose sibling ratio is out of band | 4 | The factor band (above) |
 
-PR #1040 (awaiting a ruling) excludes the same SALT and PTC columns in the
-compiler and rescales the district file's capital-gains rows by one national
-factor. `state_cd` already takes district capital gains as within-state
-shares of Historic Table 2, which a national factor does not move, so their
-level changes only when Historic Table 2's own capital-gains rows do
-(#1036).
+Since microcosm#1038 the compiler excludes the same SALT and PTC columns
+(`US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS`). PR #1040 (awaiting a ruling)
+rescales the district file's capital-gains rows by one national factor.
+`state_cd` already takes district capital gains as within-state shares of
+Historic Table 2, which a national factor does not move, so their level
+changes only when Historic Table 2's own capital-gains rows do (#1036).
 
 North Carolina's district rows are bound. Before #1043 the packaged
 117th→119th crosswalk was built from NC's 2016 plan rather than the 2019 plan
@@ -556,7 +574,10 @@ record one of the four modes, before any release directory exists.
   calibrate stage's peak is the artifact write.
 - A checkpoint written before the sparse matrix (measures as dense H5
   columns) is refused by `--stage calibrate`; re-run `--stage materialize`.
-- To reproduce the 2026-09-22 hours rebuild (#974), pass `--soi-mode totals`.
+- To reproduce the 2026-09-22 hours rebuild (#974), pass `--soi-mode totals`
+  on a compiler from before microcosm#1038; after it, `totals` no longer
+  carries the 487 district-file premium-tax-credit return rows that run
+  calibrated to.
   To reproduce a build that ran as `full` after `b7922b089`, pass
   `--soi-mode full`.
 - A checkpoint materialized under an earlier default records its own mode and

docs/us-fact-to-target.md

@@ -204,6 +204,19 @@ should calibrate, add its id to `US_FISCAL_TARGET_EXCLUSION_VINTAGE_BYPASSES`
 with a reason. `us_source_coverage.json`'s `fiscal_target_exclusion_receipt`
 lists the ids each rule dropped or allowed in a release.
 
+A fact whose package reads the wrong IRS column for its measure id is a
+different defect: every fact read from that column is wrong, at every
+geography and vintage, while the same measure read from the right column is
+fine. `US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS` is keyed by (measure id,
+`layout.source_column_id`) for these, so a package corrected to read the
+right column passes untouched; its receipt rule is
+`source_column_concept_exclusion` (microcosm#1038). Check the column against
+the IRS documentation guide before adding an entry, and list a column there
+only when it carries another concept. A column that is a component or a
+superset of the measure's concept (the district file's line 8a mortgage
+interest under `interest_paid_deduction_*`, its line 7 capital gains) needs a
+share-or-drop decision instead.
+
 ## 5. Add a reform-coverage probe
 
 Probes are **generated data**: author them in

packages/microcosm-build/src/microcosm/build/us_runtime/__init__.py

@@ -291,6 +291,7 @@
     US_FISCAL_TARGET_LEDGER_REFERENCES,
     US_FISCAL_TARGET_REFERENCES,
     US_FISCAL_TARGET_REGISTRY,
+    US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS,
     US_FISCAL_TARGET_SPECS,
     US_FISCAL_TARGET_SUPPORT_EXCLUSIONS,
     US_JCT_TAX_EXPENDITURE_REFORMS,
@@ -1165,6 +1166,7 @@
     "US_FISCAL_TARGET_SUPPORT_EXCLUSIONS",
     "US_FISCAL_TARGET_ALL_VINTAGE_SUPPORT_EXCLUSIONS",
     "US_FISCAL_TARGET_EXCLUSION_VINTAGE_BYPASSES",
+    "US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS",
     "US_FISCAL_TARGET_COVERAGE_REQUIREMENTS",
     "US_FISCAL_TARGET_LEDGER_REFERENCES",
     "US_JCT_TAX_EXPENDITURE_REFORMS",

packages/microcosm-build/src/microcosm/build/us_runtime/fiscal_targets.py

@@ -55,6 +55,7 @@
     "US_FISCAL_TARGET_SUPPORT_EXCLUSIONS",
     "US_FISCAL_TARGET_ALL_VINTAGE_SUPPORT_EXCLUSIONS",
     "US_FISCAL_TARGET_EXCLUSION_VINTAGE_BYPASSES",
+    "US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS",
     "US_FISCAL_LEDGER_PARITY_REGISTRY",
     "US_FISCAL_LEDGER_PARITY_REPORT",
     "US_JCT_TAX_EXPENDITURE_REFORMS",
@@ -877,6 +878,74 @@ def _us_hierarchy_seed(
 # every vintage.
 US_FISCAL_TARGET_EXCLUSION_VINTAGE_BYPASSES: dict[str, str] = {}
 
+# Reviewed source-column concept exclusions (microcosm#1038). Each entry is an
+# IRS column that a Chronicle package publishes under a measure id whose
+# concept the column does not carry, so no fact read from that column
+# compiles, whatever its package, geography or vintage, and a package
+# corrected to read the right column passes untouched. The key is (measure_id,
+# layout.source_column_id) because the IRS column is the definition. The
+# congressional-district documentation guide (22incddocguide.docx) defines:
+#
+# - N18425/A18425 as "State and local income taxes" (Schedule A line 5a) and
+#   N18460/A18460 as "Limited state and local taxes" (line 5e). salt_deduction,
+#   like the Historic Table 2 rows of these measure ids, is the limited line 5e
+#   deduction.
+# - N85530/A85530 as "Additional Medicare tax" (Form 8959 line 24) and
+#   N85770/A85770 as "Total premium tax credit" (Form 8962 line 24; the guide
+#   prints "8926:24"). N85770/A85770 is what the Historic Table 2 rows of
+#   these measure ids read and what assigned_aca_ptc measures. The amount was
+#   dropped before by a hard-coded congressional-district check; the returns
+#   row compiled as premium-tax-credit recipients. A package corrected to read
+#   N85770/A85770 compiles as assigned_aca_ptc targets with no further gate,
+#   which is intended: those columns are the premium tax credit.
+#
+# Chronicle's congressional_district_2022 package reads these four columns
+# under the limited_state_local_taxes_* and premium_tax_credit_* measure ids;
+# the 46 other measures it shares with a Historic Table 2 package read the
+# same column there. Of the six it has alone, interest_paid_deduction_* reads
+# N19300/A19300, "Mortgage interest paid" (Schedule A line 8a), while
+# interest_deduction is the line 10 total (lines 8a-8c plus investment
+# interest on line 9; 8a is 85.6% of it in the TY2022 US row). That is a
+# component of the measure's concept, not another concept, so it is not
+# listed here: like the line 7 capital-gains rows, it needs a share-or-drop
+# decision of its own, which also binds the state_cd surface's itemized
+# bridge.
+# A congressional-district fact of a listed measure that carries no
+# layout.source_column_id refuses the compile, so a feed schema change cannot
+# silently lift the exclusion.
+US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS: dict[tuple[str, str], str] = {
+    ("limited_state_local_taxes_returns", "N18425"): (
+        "N18425 counts returns with state and local income taxes (Schedule A "
+        "line 5a), not returns with the limited state and local tax deduction "
+        "(line 5e, N18460). Chronicle's congressional_district_2022 package "
+        "reads it under this measure id: TY2022 US 10,842,580 returns against "
+        "14,777,040 for N18460 in the same file and 14,968,720 in Historic "
+        "Table 2."
+    ),
+    ("limited_state_local_taxes_amount", "A18425"): (
+        "A18425 is state and local income taxes before the $10,000 limit "
+        "(Schedule A line 5a), not the limited deduction (line 5e, A18460) "
+        "salt_deduction measures. Chronicle's congressional_district_2022 "
+        "package reads it under this measure id: TY2022 US $250.4B against "
+        "$121.2B for A18460 in the same file, so on the full surface the "
+        "congressional-district rows summed to 1.98x the national Historic "
+        "Table 2 target."
+    ),
+    ("premium_tax_credit_returns", "N85530"): (
+        "N85530 counts returns with additional Medicare tax (Form 8959 line 24), "
+        "not returns with the premium tax credit (Form 8962 line 24, N85770) "
+        "assigned_aca_ptc measures. Chronicle's congressional_district_2022 "
+        "package reads it under this measure id: TY2022 US 6,848,330 returns "
+        "against 7,713,000 for N85770 in the same file."
+    ),
+    ("premium_tax_credit_amount", "A85530"): (
+        "A85530 is additional Medicare tax (Form 8959 line 24), not the premium "
+        "tax credit (Form 8962 line 24, A85770). Chronicle's "
+        "congressional_district_2022 package reads it under this measure id: "
+        "TY2022 US $14.2B against $53.1B for A85770 in the same file."
+    ),
+}
+
 
 @dataclass(frozen=True)
 class SimpleTaxExpenditureReform:
@@ -1033,7 +1102,7 @@ def compile_us_fiscal_target_registry(
             auditable waiver instead of raising; every waived target carries
             ``period_contract_waiver`` metadata in diagnostics.
     """
-    materialized_facts = tuple(facts)
+    materialized_facts = _without_source_column_concept_exclusions(tuple(facts))
     if congressional_district_vintage_crosswalk is not None:
         materialized_facts = translate_congressional_district_facts_to_current_vintage(
             materialized_facts,
@@ -1114,6 +1183,10 @@ def us_fiscal_target_exclusion_receipt(
       vintage included (dropped).
     - ``m_chip_state_chip_enrollment``: CMS CHIP enrollment facts for an
       ``_M_CHIP_STATE_FIPS`` state, at every period (dropped).
+    - ``source_column_concept_exclusion``: SOI facts read from an IRS column
+      that ``US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS`` lists for their
+      measure id, at every vintage, as the feed carries them, before the
+      congressional-district crosswalk (dropped).
     - ``allowlisted_vintage_bypass``: facts latest-vintage selection activates
       at ``target_period`` that are another vintage of an excluded cell and
       carry a reviewed ``US_FISCAL_TARGET_EXCLUSION_VINTAGE_BYPASSES`` entry
@@ -1124,7 +1197,13 @@ def us_fiscal_target_exclusion_receipt(
     it raises on a vintage bypass nobody reviewed.
     """
 
-    materialized_facts = tuple(facts)
+    raw_facts = tuple(facts)
+    source_column = {
+        _source_record_id(fact)
+        for fact in raw_facts
+        if _is_source_column_concept_exclusion(fact) and _source_record_id(fact)
+    }
+    materialized_facts = _without_source_column_concept_exclusions(raw_facts)
     if congressional_district_vintage_crosswalk is not None:
         materialized_facts = translate_congressional_district_facts_to_current_vintage(
             materialized_facts,
@@ -1168,6 +1247,22 @@ def us_fiscal_target_exclusion_receipt(
                 "entries": sorted(US_FISCAL_TARGET_ALL_VINTAGE_SUPPORT_EXCLUSIONS),
                 "source_record_ids": sorted(all_vintage),
             },
+            "source_column_concept_exclusion": {
+                "action": "dropped",
+                "scope": "SOI facts read from a listed column under a listed "
+                "measure id, every vintage",
+                "entries": [
+                    {
+                        "measure_id": measure_id,
+                        "source_column_id": source_column_id,
+                        "reason": reason,
+                    }
+                    for (measure_id, source_column_id), reason in sorted(
+                        US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS.items()
+                    )
+                ],
+                "source_record_ids": sorted(source_column),
+            },
             "m_chip_state_chip_enrollment": {
                 "action": "dropped",
                 "scope": "CMS CHIP enrollment rows for these states, every period",
@@ -2916,6 +3011,55 @@ def _period_free_source_record_id(source_record_id: str) -> str:
     return _normalized_record_set_id(source_record_id)
 
 
+class MissingSoiSourceColumnError(ValueError):
+    """A congressional-district fact of a listed measure names no IRS column."""
+
+
+def _is_source_column_concept_exclusion(fact: object) -> bool:
+    """Whether an SOI fact reads a column its measure id does not mean.
+
+    A congressional-district fact of a measure the register lists must name
+    its column: without one the exclusion could not tell a mislabeled column
+    from a corrected one, so the compile refuses instead of guessing.
+    """
+    if _source_name(fact) != "irs_soi":
+        return False
+    measure_id = _measure_id(fact)
+    if measure_id not in {
+        measure for measure, _ in US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS
+    }:
+        return False
+    column = _str_at(fact, "layout", "source_column_id")
+    if not column:
+        # Any congressional-district package, whatever its boundary vintage.
+        if ".congressional_district_" in _str_at(fact, "layout", "record_set_id"):
+            raise MissingSoiSourceColumnError(
+                f"{_source_record_id(fact)} is a congressional-district "
+                f"{measure_id} fact with no layout.source_column_id, so "
+                "US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS cannot tell which "
+                "IRS column it reads (microcosm#1038)."
+            )
+        return False
+    return (measure_id, column) in US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS
+
+
+def _without_source_column_concept_exclusions(
+    facts: tuple[object, ...],
+) -> tuple[object, ...]:
+    """Drop facts read from a listed column, before the district crosswalk.
+
+    The crosswalk sums several source districts into one current district and
+    keeps one contributor's layout, so the column check must see each source
+    fact: afterwards an excluded contribution could inherit a corrected
+    column, and a fact with no column could hide behind one that has it.
+    Raises ``MissingSoiSourceColumnError`` for a listed district fact with no
+    column.
+    """
+    return tuple(
+        fact for fact in facts if not _is_source_column_concept_exclusion(fact)
+    )
+
+
 def _is_all_vintage_support_exclusion(source_record_id: str) -> bool:
     return _period_free_source_record_id(source_record_id) in (
         _period_free_source_record_ids(US_FISCAL_TARGET_ALL_VINTAGE_SUPPORT_EXCLUSIONS)
@@ -2950,18 +3094,6 @@ def _is_soi_congressional_district_record_set(fact: object) -> bool:
     return ".congressional_district_2022." in record_set_id
 
 
-def _is_soi_cd_premium_tax_credit_amount_conflict(
-    fact: object, *, measure_id: str
-) -> bool:
-    # The SOI CD A85530 amount is not the same gross annual PTC control used by
-    # assigned_aca_ptc. Keep Historic Table 2 national/state PTC amount controls
-    # until Ledger has an explicit CD reconciliation for this concept.
-    return (
-        measure_id == "premium_tax_credit_amount"
-        and _is_soi_congressional_district_record_set(fact)
-    )
-
-
 def _is_period_token(value: str) -> bool:
     normalized = value.lower().replace("-", "_")
     if normalized.startswith("month"):
@@ -3028,6 +3160,8 @@ def _reference_from_ledger_fact(
         return None
     if _is_all_vintage_support_exclusion(source_record_id):
         return None
+    if _is_source_column_concept_exclusion(fact):
+        return None
     source_name = _source_name(fact)
     if source_name == "irs_soi":
         return _soi_reference_from_fact(
@@ -3085,8 +3219,6 @@ def _soi_reference_from_fact(
         and geography_level != "congressional_district"
     ):
         return None
-    if _is_soi_cd_premium_tax_credit_amount_conflict(fact, measure_id=measure_id):
-        return None
     variable = SOI_AMOUNT_MEASURE_VARIABLES.get(measure_id)
     is_count = False
     if variable is None: