Repository navigation
PSID income and wealth readers and field registry (D1a) - #575
Merged
Merged
Conversation
A registry of 1,463 PSID family-file fields across the requested income waves (1999-2023) and wealth waves (2015-2023): each field's variable, label and column positions from the wave's pinned setup file, and its value domain, definition notes and codebook page. It holds no codebook frequencies. The readers verify setup bytes, labels and columns before reading, refuse unregistered waves and concepts, and reconcile income and wealth identities. PSID use follows the project's data-access record. Built by a Subfleet build lane in a standalone clone from PSID metadata only; committed by the coordinating session. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unit tests build invented records and check refusals, conservation of families, missing and top codes, and the income and wealth identities with Hypothesis. The integration_psid tests verify the staged setup files and build invented records from their layouts; they skip when the metadata root is absent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The PSID income and wealth readers are opt-in source readers that nothing in the historical projection imports. List their two source files as post-review exclusions in the birth-evidence script and its expected tuple, and assert that neither module is reachable from the birth-evidence reducer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
D1a adds 94 unit and 21 integration_psid tests in its two PSID income and wealth modules; its seal edits add no tests. Counted by collecting those named modules per marker, added to master's counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The table in tests/README-tiers.md now matches tests/tier_counts.json at this head: unit 18,433, integration_psid 1,471, total 25,003. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
marked this pull request as ready for review
October 25, 2026 11:00
Contributor
Author
|
Merged by the Microcosm Dynamics coordinator at 11:00:20Z on 2026-10-25, as merge commit Gates at the merged head:
The description. The coordinator updated it at landing to match the head. The tier counts were described as unchanged, which was no longer true, and the PR was still labelled WIP. The coordinator also added an "At the slot" section. The lane's other text is unchanged. The slot now passes to #573. |
MaxGhenis
added a commit
that referenced
this pull request
Oct 25, 2026
Master gained D1a (#575). Both branches extended the birth-evidence check's post-review exclusions and its reachability test, so the merge keeps D1a's two PSID income and wealth readers and D4's five aggregate-table modules, each with its own reachability assertion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 25, 2026
MaxGhenis
added a commit
that referenced
this pull request
Oct 25, 2026
Master brings #575 (D1a: PSID income and wealth readers). It adds two new modules under src/populace_dynamics/data/ and changes no existing source file, so nothing B0.2 imports changes. The tier file and the README's tier table conflicted. Both are resolved to master's counts plus this branch's unit delta: 18,433 + 233 = 18,666 unit, with master's integration_psid 1,471, for a total of 25,236. The branch's README table had not been updated since a3b6236 (18,514); it now matches the tier file. This is derived; CI's collection checks it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5 of 8 tasks
MaxGhenis
added a commit
that referenced
this pull request
Oct 25, 2026
Master brings #587 (ledger delta), #575 (D1a) and #570 (SM1). None of them changes the registry module, its tests or its mutation specs, and the registry module imports only the standard library and yaml. The tier conflict resolves to master's counts plus this branch's nine unit tests: unit 18,549 + 9 = 18,558, total 25,128. The README tier table now matches the tier file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 25, 2026
Held-out data registry: round 7's follow-ups (unset event path, record ids, zero push boundary)
#580
Merged
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the first part of milestone D1 of the income-and-resources design (#559: the D1 row of the milestone table): readers for the PSID family-file waves already staged. The wealth supplements and pension-section items come in later parts. The tier counts were updated at the coordinator's slot, and the independent reviews approved (see below).
PSID use follows the project's data-access record. This build read PSID metadata only, namely setup files and codebooks. It opened no microdata and computed no statistic from PSID records.
What it adds
src/populace_dynamics/data/psid_income_wealth_registry.py: 1,463 verified fields. Waves covered:.spssetup (with its SHA-256);src/populace_dynamics/data/psid_income_wealth.py: income and wealth readers, labelled likefamily_income.py. Before reading, they verify the setup bytes, labels and columns, and they refuse unregistered waves, concepts, setups, labels and columns.Invariants
Open questions
family_income's private_identity_countsand_normalizedhelpers. The review should decide whether to make them public.Tests
-n 4). They build invented records.integration_psidtests:black -l 79andruffare clean.tests/tier_counts.json, unit goes from 18,339 to 18,433 and integration_psid from 1,450 to 1,471. The table intests/README-tiers.mdmatches (total 25,003).Since the build
0d704e36mergesorigin/masteratfe5ab5abcleanly.1ec99fb0keeps the birth-evidence seal. The two new modules are listed inPOST_REVIEW_SOURCE_EXCLUSIONSand in the test's expected tuple, and a new assertion checks that neither is reachable from the birth-evidence reducer. That fixestest_reducer_input_identity_matches_reviewed_branch.tests/test_tier_policy.pywas expected to fail until the tier-count slot.1ec99fb0with no high or medium findings. Its prescribed runs passed 111 and 21 tests. It judged the private helper imports safe as used.1ec99fb0passed lint and the seal. Two failures remained, both expected and outside this PR's code:test_tier_policy, which waits for the tier-count slot, and Track B's holdout-registry append-only check, which waits for Track B's fix on master.At the slot
4f2dc624merges master at9a344a1acleanly. Its tree equalsgit merge-tree --write-tree 1ec99fb0 9a344a1a.5dd8464brecounts the tiers: 94 more unit tests and 21 more integration_psid tests, which are the two new modules.7ca55400updates the README tier table to match.7ca55400(Tests run 40180470423): all four shards, both graph-integration jobs,pytest (3.14), lint and build. The tier-policy test is in the suite and no shard failed.7ca55400. It checked the merge tree, that D1a's own files are unchanged since1ec99fb0, the counts against a static count of the two modules, the README table and CI's head.Description updated by the Microcosm Dynamics coordinator at landing (10:59Z, 2026-10-25) to match head
7ca55400; the lane's text is otherwise unchanged.axiom: n/a: survey-file readers and metadata registry for the microsim; no PolicyEngine country-model or RuleSpec encoding changed
🤖 Generated with Claude Code