Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/1335-registry-geography-label-check.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
`TargetRegistry` checks that one geography id has exactly one label across a registry (microcosm#1335). A US registry refuses a second label with `GeographyLabelConflictError`, naming the id, both labels and the first target carrying each; every other country's registry reports it as a `GeographyLabelConflictWarning`, because UK publishers name some regions, nations and local authorities differently. Builds assemble their registry before calibration, and the US ACS local materialize stage assembles its admin and ladder population targets into one registry before the engine pass, so a conflict stops a build at its start. A lean checkpoint written before this change with two labels for one US geography is refused when the calibrate stage reloads it, with a message to re-run `--stage materialize`.
4 changes: 4 additions & 0 deletions changelog.d/1335-us-geography-one-label.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
The US ACS local release's Census population targets label each state and congressional district the way the release's other targets label the same id (microcosm#1335). The release `populace-us-2024-buildo-acs-local-766b7c7-20261017T030317Z` labelled each of its 51 states with its Census name on the IRS SOI, SNAP and Medicaid targets ("Maine") and with its postal code on the state population target ("ME").

- **States.** A population target's state label is the state's Census name from the new shared table `microcosm.calibrate.geography_constants.US_STATE_FIPS_TO_NAME`. Its 51 names equal the release's ledger state labels and the names every Chronicle source package of the pinned US feed declares. The block ladder's `US_STATES` table now derives its P.L. 94-171 directory names from it (the derived table equals the previous literal).
- **Congressional districts.** A district's label is the Census Bureau's name ("Congressional District 1 (119th Congress), Alabama", "Congressional District (at Large) (119th Congress), Alaska", "Delegate District (at Large) (119th Congress), District of Columbia"), from `congressional_district_label` in `microcosm.build.us_runtime.congressional_district_vintage`. The IRS SOI district rows translated to the 119th Congress plan take their label from the same function. The two producers used to disagree on at-large districts ("AK at-large congressional district" against "AK congressional at-large"), so a surface binding both at one at-large id gave it two labels. A feed with Chronicle's ACS district facts, which carry the Census names, now agrees with both.
22 changes: 22 additions & 0 deletions docs/calibration-target-hierarchy.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,28 @@ contain `materialization`. If Chronicle does not yet publish the required fact,
add the source fact to Chronicle before declaring the Microcosm target. Do not
construct an observed target value from a Microcosm runtime input.

## One label per geography

One geography id has exactly one label across a target registry, so a
calibration diagnostics file names each geography once (microcosm#1335).
`TargetRegistry` checks this whenever a build assembles targets into a
registry, which is before calibration. A US registry refuses a second label
with `GeographyLabelConflictError`, naming the id, the labels and the first
target carrying each. Every other country's registry reports the conflict as a
`GeographyLabelConflictWarning`: UK publishers name some regions, nations and
local authorities differently ("Yorkshire and The Humber" and "Yorkshire and
the Humber", "St Helens" and "St. Helens"), and one label per UK code has not
been chosen yet.

A target Microcosm declares itself takes the label its other sources give the
same id. The US ACS local release's Census population targets label a state
with `US_STATE_FIPS_TO_NAME` ("Maine", as Chronicle's state facts name it) and
a congressional district with the Census Bureau's district name
("Congressional District 1 (119th Congress), Alabama"), from the same function
that labels the IRS SOI district rows translated to the 119th Congress plan.
That release's materialize stage assembles its admin and population targets
into one registry before the engine pass, so a conflict stops it at the start.

## Propagation stages

The expected data flow is:
Expand Down
15 changes: 12 additions & 3 deletions docs/shared-constants.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ and build packages. It currently defines:
- `UK_WIDE_GEOGRAPHY_IDS` (the K-prefixed UK-wide aggregates among
`UK_GEOGRAPHY_ID_TO_LABEL`, which the nation grain of the UK target-weight
rules splits on; microcosm#1124)
- `US_STATE_FIPS_TO_NAME` (each state's Census name, "District of Columbia"
included: the names Chronicle's state facts carry, so a target Microcosm
labels itself gives a state the same label as the ledger targets;
microcosm#1335)
- `US_STATE_FIPS_TO_POSTAL`
- `US_STATE_NUMERIC_FIPS_TO_POSTAL`
- `US_STATE_POSTAL_TO_NUMERIC_FIPS`
Expand All @@ -51,9 +55,14 @@ from microcosm.calibrate.geography_constants import (
)
```

Do not add another state FIPS/postal mapping to a country runtime or diagnostics
module. Extend the shared definition and its consistency test when supported
geographies change.
Do not add another state FIPS/postal or FIPS/name mapping to a country runtime
or diagnostics module. Extend the shared definition and its consistency test
when supported geographies change. The block ladder's `US_STATES` table derives
its P.L. 94-171 directory names from `US_STATE_FIPS_TO_NAME`. A congressional
district's label is the Census Bureau's district name, from
`congressional_district_label` in
`microcosm.build.us_runtime.congressional_district_vintage`; it spells the
state with `US_STATE_FIPS_TO_NAME`.

The engine's `local_authority` member names are not a constants table: the 361
April 2023 authority names are published ONS data, held in the sha-pinned
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,64 +45,22 @@

import numpy as np

from microcosm.calibrate.geography_constants import (
US_STATE_FIPS_TO_NAME,
US_STATE_FIPS_TO_POSTAL,
)

if TYPE_CHECKING:
from microcosm.build.us_runtime.geography_ladder import UsBlockLadder

#: The 50 states plus DC: (state FIPS, USPS abbreviation, P.L. 94-171
#: directory name). Puerto Rico and the island territories are outside the
#: US artifact's state spine.
US_STATES: tuple[tuple[str, str, str], ...] = (
("01", "AL", "Alabama"),
("02", "AK", "Alaska"),
("04", "AZ", "Arizona"),
("05", "AR", "Arkansas"),
("06", "CA", "California"),
("08", "CO", "Colorado"),
("09", "CT", "Connecticut"),
("10", "DE", "Delaware"),
("11", "DC", "District_of_Columbia"),
("12", "FL", "Florida"),
("13", "GA", "Georgia"),
("15", "HI", "Hawaii"),
("16", "ID", "Idaho"),
("17", "IL", "Illinois"),
("18", "IN", "Indiana"),
("19", "IA", "Iowa"),
("20", "KS", "Kansas"),
("21", "KY", "Kentucky"),
("22", "LA", "Louisiana"),
("23", "ME", "Maine"),
("24", "MD", "Maryland"),
("25", "MA", "Massachusetts"),
("26", "MI", "Michigan"),
("27", "MN", "Minnesota"),
("28", "MS", "Mississippi"),
("29", "MO", "Missouri"),
("30", "MT", "Montana"),
("31", "NE", "Nebraska"),
("32", "NV", "Nevada"),
("33", "NH", "New_Hampshire"),
("34", "NJ", "New_Jersey"),
("35", "NM", "New_Mexico"),
("36", "NY", "New_York"),
("37", "NC", "North_Carolina"),
("38", "ND", "North_Dakota"),
("39", "OH", "Ohio"),
("40", "OK", "Oklahoma"),
("41", "OR", "Oregon"),
("42", "PA", "Pennsylvania"),
("44", "RI", "Rhode_Island"),
("45", "SC", "South_Carolina"),
("46", "SD", "South_Dakota"),
("47", "TN", "Tennessee"),
("48", "TX", "Texas"),
("49", "UT", "Utah"),
("50", "VT", "Vermont"),
("51", "VA", "Virginia"),
("53", "WA", "Washington"),
("54", "WV", "West_Virginia"),
("55", "WI", "Wisconsin"),
("56", "WY", "Wyoming"),
#: US artifact's state spine. The directory name is the state's Census name
#: with underscores for spaces ("District_of_Columbia"); the codes and names
#: come from ``microcosm.calibrate.geography_constants``.
US_STATES: tuple[tuple[str, str, str], ...] = tuple(
(state_fips, postal, US_STATE_FIPS_TO_NAME[state_fips].replace(" ", "_"))
for state_fips, postal in US_STATE_FIPS_TO_POSTAL.items()
)

_PL_GEO_SUMMARY_LEVEL_FIELD = 2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,17 @@
import numpy as np
import pandas as pd

from microcosm.calibrate.geography_constants import US_STATE_FIPS_TO_POSTAL
from microcosm.calibrate.geography_constants import (
US_STATE_FIPS_TO_NAME,
US_STATE_FIPS_TO_POSTAL,
)

SOURCE_CONGRESSIONAL_DISTRICT_PREFIX = "5001700US"
CURRENT_CONGRESSIONAL_DISTRICT_PREFIX = "5001900US"
CURRENT_CONGRESSIONAL_DISTRICT_VINTAGE = "119th_congress"
#: The Congress whose plan ``CURRENT_CONGRESSIONAL_DISTRICT_PREFIX`` identifies,
#: as the Census Bureau's district names spell it.
CURRENT_CONGRESS_NAME = "119th Congress"
#: The packaged default crosswalk, built from Census sources by
#: ``tools/build_us_congressional_district_vintage_crosswalk.py`` and shipped
#: under ``microcosm.build.us_runtime.data``. See the sibling ``.provenance.json``
Expand Down Expand Up @@ -617,7 +623,7 @@ def _translated_fact_shell(
target_geoid = target_geography_id.removeprefix(
CURRENT_CONGRESSIONAL_DISTRICT_PREFIX
)
target_geography_label = _congressional_district_label(target_geography_id)
target_geography_label = congressional_district_label(target_geography_id)
geography = dict(_mapping_at(translated, "geography"))
geography["id"] = target_geography_id
geography["level"] = "congressional_district"
Expand Down Expand Up @@ -672,23 +678,45 @@ def _translated_fact_shell(
return translated


def _congressional_district_label(geography_id: str) -> str:
"""Return a reviewed display label for a Census congressional-district ID."""
def congressional_district_label(geography_id: str) -> str:
"""The Census Bureau's name for a current congressional district.

The Census Bureau names a 119th Congress district "Congressional District
1 (119th Congress), Alabama", an at-large seat "Congressional District (at
Large) (119th Congress), Alaska", and the District of Columbia's
non-voting seat (district ``00`` here, Census's ``98``) "Delegate District
(at Large) (119th Congress), District of Columbia". Chronicle's ACS
district facts carry these names for the same ids. The IRS SOI district
rows translated here and the ACS local release's population targets both
take their label from this function, so each district has one label
across target sources (microcosm#1335).
"""

geoid = geography_id.removeprefix(CURRENT_CONGRESSIONAL_DISTRICT_PREFIX)
if len(geoid) != 4 or not geoid.isdigit():
if (
not geography_id.startswith(CURRENT_CONGRESSIONAL_DISTRICT_PREFIX)
or len(geoid) != 4
or not geoid.isdigit()
):
raise ValueError(
f"Invalid current congressional-district geography id {geography_id!r}."
)
state_fips, district = geoid[:2], geoid[2:]
postal = US_STATE_FIPS_TO_POSTAL.get(state_fips)
if postal is None:
state_name = US_STATE_FIPS_TO_NAME.get(state_fips)
if state_name is None:
raise ValueError(
f"Unknown state FIPS code in congressional-district id {geography_id!r}."
)
if district == "00":
return f"{postal} at-large congressional district"
return f"{postal} congressional district {int(district)}"
seat = (
"Delegate District"
if US_STATE_FIPS_TO_POSTAL[state_fips] == "DC"
else "Congressional District"
)
district_name = f"{seat} (at Large)"
else:
district_name = f"Congressional District {int(district)}"
return f"{district_name} ({CURRENT_CONGRESS_NAME}), {state_name}"


def _append_lineage_contribution(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,10 @@ def _ledger_concept_budget_key(
#:
#: - ``ledger_fact_label`` and ``ledger_layout_groupby_value_label``: the
#: human-readable labels the ledger compiler (``ledger_targets.py``) stamps
#: on every row, which name the district ("WV congressional district 1").
#: on every row, which name the district. A translated IRS SOI row's group
#: value label is the current district's Census name ("Congressional
#: District 1 (119th Congress), West Virginia"); its fact label is the
#: label of a source-plan fact, which the translation does not change.
#: :data:`US_FISCAL_TARGET_CONCEPT_METADATA_EXCLUSIONS` does not list them,
#: so under the national mapping every compiled district row is its own
#: group; the national mapping is kept as it is here.
Expand Down
Loading
Loading