Repository navigation
SLD layer for the local artifact: per-district ACS calibration (targets, membership vintage, doctrine solve, validation) #625
Description
Why
State policy analysis needs district-credible breakdowns at three lenses — household, statewide, and by state legislative district. The statewide and household lenses exist today; the by-district lens does not. #449's CD experiment already taught the doctrine lesson: binding small-area targets into the national solve halved the within-10% rate. So the SLD product is a local product by construction: per-district reweighting on the local-area artifact, with the national/state calibration objective untouched (#492 embargo on new knobs applies in full — this lane changes national calibration nowhere).
Input artifact: the #512 rebuild (buildo-acs-local on the certified Build O lineage, ~1.3M households ≈ 270–670 per district across the ~4,800 SLDL+SLDU districts). The layer is built and tested against the artifact shape so it drops onto #512's release when that lane certifies; nothing here blocks or modifies #512.
Design
1. Target facts (ledger lane)
Census ACS 5-year 2020–2024 tables at SLD summary levels 610 (upper) / 620 (lower) — SLDs are published only in the 5-year dataset, so the 5-year reference window is a stated boundary of the product, not a footnote:
- B01001 → population by age band (collapsed, sexless bands — the honest small-area grain),
- B11001 → total households,
- B19001 → household income brackets (the calibratable income instrument),
- B19013 median household income → validation-only: a linear reweighting operator can honestly hit bracket counts; it cannot honestly target a median. The median is checked after the solve, never calibrated.
Witnessed source packages in the ledger (the geography levels state_legislative_district_upper/lower are already in the core vocabulary; the CD packages are the template, switched to the 5-year instrument). Verified live: all four tables serve at 610/620 with district labels carrying the "(2024)" boundary vintage; per-state pulls (national SLD wildcards are not served).
2. District membership at the ACS boundary vintage
The artifact's baked sldl/sldu columns are 2020-BAF vintage (2010-cycle districts) and donor-spine-only; the ACS 5-year tables tabulate on 2024 SLD boundaries. So the layer derives district membership itself, at the target vintage, from a new pinned source: the Census 2024 SLD BEFs (sldu24.zip/sldl24.zip, same RDO mapping-files family as the pinned cd119.zip).
One uniform seeded operator, conditioning on the finest geography each row carries:
- donor rows (block/tract certified): direct lookup where the tract lies wholly in one district, population-weighted block draw within tract where split;
- ACS-spine rows (no sub-PUMA geography): seeded block draw conditional on (PUMA × assigned CD × assigned county) via the block ladder, then read the block's SLDU and SLDL off the same draw — chambers stay mutually coherent and coherent with the row's existing seeded geography.
Reproducible from seed; recorded as a reviewed limitation in the same class as the artifact's existing seeded CD/county assignment for ACS rows.
3. Per-district solve (the doctrine port, #503 → US)
- Assigned-rows-per-district: each district's candidate pool is exactly its assigned households (~270–670). The problem is block-diagonal with no shared rows or targets, so it decomposes into independent per-district solves — one uniform operator applied ~4,800 times, not 4,800 knobs.
- Doctrine module (US analog of
uk_runtime/local_doctrine.py): declared loss cap, declared stretch bound, closed scale-rule and weighting-rule vocabularies, structurally knob-free release-path signature (per-target anything is aTypeErrorby construction). - Anchor declared (max_weight_ratio anchors differently per arm: 5x vs design (dense) but ~25x effective (sparse refit re-anchors) — declare one stretch contract #493's lesson): starting weights are the artifact's calibrated household weights restricted to the district; single-stage stretch envelope vs that anchor, realized max ratio recorded per district.
- Past-cap census per district (Past-cap census as a first-class calibration diagnostic #494 generalized): init/final/escaped/frozen/pushed-out per district plus a cross-district roll-up naming the worst districts.
4. Validation protocol
- Per-district achieved-vs-target table for every calibrated cell (the release diagnostic, not a side artifact).
- Sanity against published ACS district profiles — including the B19013 median check (achieved median within a declared within-bracket-interpolation tolerance).
- Statewide coherence report: sum of district estimates vs the statewide artifact estimate per metric — reported, never constrained (the districts' own ACS targets already sum ≈ state totals; residual incoherence is a declared property).
- Honest-boundaries statement generated INTO the artifact: demographics and household-income brackets are district-calibrated; tax and program detail is inherited from the state-calibrated solve; ACS 5-year 2020–2024 reference window; seeded-assignment class for ACS-spine rows; small-area tail caveats on the register columns.
Product shape
A sidecar in the release directory (per-chamber long weights (area_type, area_code, household_id, weight) + per-district diagnostics + doctrine/census JSON), registered as non-microdata artifacts in the release manifest with sha256sums coverage — riding the #398 non_default_local_area contract, which is require-based and validates sidecars that declare themselves.
Pilot: Utah, then generalization
Pilot on Utah: both chambers present (unlike NE), single-member districts throughout (no NH floterial or multi-member representation quirks), maps adopted 2021 and stable through the 2024 tabulation vintage (clean external-profile comparison), 29 SLDU + 75 SLDL = right pilot scale, and an active state-policy engagement to serve. Generalization is the same code over the state enumeration, with the special cases declared per state (NE unicameral, DC wards-as-SLDU, NH floterial tabulation, multi-member districts as Census tabulates them).
Deliverables
- Ledger: SLD source packages (pilot scope, then national) — witnessed, pinned, 5-year vintage declared.
- Populace: SLD membership operator (2024-BEF vintage) + per-district solver under doctrine + past-cap census; TDD against the artifact shape; no national-calibration diffs.
- Validation protocol + honest-boundaries statement wired into the sidecar outputs.
- PRs with cross-family review; merge is the owner session's call.
Sequenced behind #512 (its lane owns the rebuild; this lane consumes the artifact). Relates #449, #492, #493, #494, #398, #503.
Activity
MaxGhenis commentedon Aug 5, 2026
Utah pilot ran on the real #512 artifact — machinery proven end to end. The rebuild published while this lane was building (
populace-us-2024-buildo-acs-local-77e2061-20260724T110908Z), so the pilot ran on it directly rather than the buildl shape. Receipts committed underexperiments/sld_local_pilot/on branchsld-local-layer-625.Inputs: six ledger ACS 2020-2024 5-year SLD packages (Utah, 610/620; ledger branch
census-acs-sld-2024, 3,744 facts), the national 2024-SLD membership ladder (population conserved exactly at 331,449,281; 1,950 SLDU + 4,833 SLDL districts), and the artifact's 14,551 Utah households (861 donor-spine rows, the rest ACS-spine).Results: all 104 districts solved under the declared doctrine (cap 10.0, stretch 100.0 vs the artifact-weight anchor, uniform weights). Median within-10% share 1.00 in both chambers (worst district 0.94). Past-cap census: 26 rows past cap at init, all escaped, zero frozen, zero pushed out. Published-median sanity (B19013, validation-only): median abs gap 3.3%, p90 8.0%, 2/104 flagged past the 15% review threshold. Statewide coherence (sum of districts vs sum of targets): [0.948, 1.006] across all 70 state-metric sums. Membership method mix: 12,972 (PUMA×CD×county) draws, 718 fallback draws (4.9% — the measured independent-seeding incoherence class), 490 tract-exact + 371 tract-split for donor rows.
Adjudications recorded for review, not tuned away: (1) the 100× stretch bound binds (realized max ≈ 100.0 in both chambers) — the declared bound is load-bearing and its level is a review decision; (2) thin-district concentration, min district ESS fraction ≈ 0.11.
Cross-family sol reviews of both committed diffs (ledger + populace) are running; PRs follow with the review receipts. Merge remains the owner session's call.
MaxGhenis commentedon Aug 7, 2026
PRs up, both cross-family-reviewed, corrected pilot receipts committed.
missing_raw_r2_linkpending the owner'spublish-rawrun).Review disposition (GPT-5.6-sol, read-only, committed-diff-scoped, both repos): initial verdicts were block (layer) and fidelity-fail (packages). Every confirmed finding is fixed and the Utah pilot re-run. The material corrections: group-quarters rows no longer count as households in the B19001/B19013 universes (1,617 UT rows); the money-income recipe was corrected against policyengine-us definitions (tips double-count, SSTB omission, Schedule-J farm field, under-15 universe); cross-state membership draws are foreclosed; the ladder embeds and enforces its boundary vintage; zero-support districts appear in every report;
min_initial_weightis no longer a wrapper parameter.Corrected pilot (receipts + correction narrative in
experiments/sld_local_pilot/): all 104 districts solve; within-10% median 1.00 (SLDU) / 0.97 (SLDL); zero frozen/pushed-out; published-median sanity improved to a 2.5% median gap (2/104 flagged); coherence [0.865, 1.009] — the low tail is the SLDL low-income brackets, where thin districts hit the 100× bound before attaining bracket mass: a real artifact-support finding now on the record. Standing adjudications: the bound binds in 15/75 SLDL districts (0/29 SLDU); min district ESS ≈ 0.08–0.12.Merge remains the owner session's call; the chronicle PR's one pending owner step is
publish-raw(R2 credentials).Add the SLD per-district local layer: 2024-vintage membership, doctri…
Review fix batch: household universes, recipe corrections, membership…
MaxGhenis commentedon Aug 14, 2026
Both PRs verified green and mergeable — the lane's work is complete; merge is yours.
gh pr checksall four jobs pass;mergeable: MERGEABLE. The branch absorbed the microcosm namespace rename (Rename Populace to Microcosm: namespace, dists, docs #638) and the spine-blindness registry (the five SLD modules are classified as reviewed non-registry modules; the district compile's sort was restructured to satisfy the guard's static-selector rule).mergeable: MERGEABLE. Rebased across the Chronicle rename and the bundle-surface churn (pins now 96 packages / 16 year-skipped / 44,867 rows). Heads-up for the merge window: the bundle-count pins intests/test_chronicle_bundle.pyshift with every new package PR, so a quick re-resolve may be needed if other chronicle PRs land first.uv run ledger publish-raw --root db/data(ledger-raw R2), which writes thestorage.r2blocks back into the six manifests — the CI acceptance pin then tightens from "clean except missing_raw_r2_link" to fully clean.