diff --git a/data/external/track_u2/u1_identity.json b/data/external/track_u2/u1_identity.json index e8b49a6b7..d09eed9b2 100644 --- a/data/external/track_u2/u1_identity.json +++ b/data/external/track_u2/u1_identity.json @@ -55,12 +55,12 @@ }, "u2_specification": { "path": "docs/design/boomers2004_1946_55_comparison.md", - "note": "Moved out of the unchanged-byte list by u2-m1b: Part C writes the proposed revision u2-draft-4 into the specification. The draft-3 hash is the milestone-1 pin; the current hash pins u2-draft-4 with Max's 2026-09-30 rulings recorded in §16b (d637) and his 2026-10-10 ruling on blockers B1 and B2 recorded in §16c (d1090); every line a registry cites keeps its draft-3 number and text.", + "note": "Moved out of the unchanged-byte list by u2-m1b: Part C writes the proposed revision u2-draft-4 into the specification. The draft-3 hash is the milestone-1 pin; the current hash pins u2-draft-4 with Max's 2026-09-30 rulings recorded in §16b (d637) and his 2026-10-10 ruling on blockers B1 and B2 recorded in §16c (d1090), with his 2026-10-19 ruling on the birth law's support recorded in §16d (d1724); every line a registry cites keeps its draft-3 number and text.", "draft_3_sha256": "7badf89e896ece28abcea1cb2db9bd55d468346b405ed06eb357aaf5c7b51df2", "draft_3_commit": "f7412e00cdcdba6ee1156b74fa93039f5329cb8f", "draft_3_git_blob": "e07ee0d589d20f3b6d100e5d4ce173f9a653f5e9", "version": "u2-draft-4", - "sha256": "c0c2685f8e23756895b66d09a6bbd618f930c488cc5ad5c52415f42b228d8f98", + "sha256": "4ad771e296791ce82f5b5cb0cc30c64460f209467fd60504a6cae3bd7581bcb0", "cited_lines_sha256": { "74": "7b299c9e4473fd51d04bdfec51af83268e517181ab734d3e76ccd1c744f6daa9", "75": "44454cc04e6a50863035d69ccb53edec175b4ad70a19d8877437901d2deb8428", diff --git a/docs/design/boomers2004_1946_55_comparison.md b/docs/design/boomers2004_1946_55_comparison.md index e0013ce8e..b3944cab4 100644 --- a/docs/design/boomers2004_1946_55_comparison.md +++ b/docs/design/boomers2004_1946_55_comparison.md @@ -1647,11 +1647,47 @@ Both rules are fixed by this ruling, before any count, and each is vacuous if no - Each registry effect is its own reviewed commit (§16a, "Registry effect of the rulings"). For B1 and B2 the effect replaces eight refusals, the seven role refusals and `pension:2015.route.respondent_slots`, with the rules above and releases their dependents. §§3–15 keep their draft-3 line numbers, and their draft-3 text except §15's version key (line 1047), because the registries cite them. - No count, statistic or outcome was computed to record this ruling, and no file on the restricted list was opened for reading. Reading and exposure are recorded in `docs/design/u2_m1b_exposure.md` (2026-10-10 section). +## 16d. Max's ruling on the birth law's support (2026-10-19, d1724) + +> **RULED on 2026-10-19.** Max ruled on 2026-10-19 (decision d1724), after the first §20 step-4 structural pass stopped before writing any output, and on 2026-10-20 approved the widening this section records. This section is the §3 note the ruling names. §3 keeps its draft-3 text and line numbers, because the registries cite them (§16c), so the note stands here and amends §3 as §§16b and 16c do; where it and §3 differ, it governs. §§16a–16c are unchanged. + +The decision put to Max, as recorded in the chief-of-staff decision log: + +> "dynamics U2 §20 step 4, second run: after the d1649 run stopped on a career-bound assertion with no output, fix U2's birth universe so it counts out people whose seed coordinate implies a birth year outside career's [1889, 2016] (as Track M does; career.py untouched), then rerun step 4 once under d1649's scope" + +The ruling, as recorded there: + +> "Yes: fix, then rerun once. A reviewed PR counts out the out-of-bounds people in U2's cohort code and discloses the counts, with a §3 note; career.py is untouched. Once it merges and master's Tests pass at the run head, step 4 (structure, then component diagnostics) runs once more under d1649's scope, with --authorized-by this decision. Max, dynamics coordinator session 2026-10-19 (guided AskUserQuestion)." + +**What stopped the first pass.** The pass Max authorized on 2026-10-16 (d1649) ran once, on 2026-10-19 at `640adccf`, and exited without writing any output; the component diagnostics were not run (evidence: `EV/track-u2-step4-pass-20261019/MANIFEST.md`, SHA-256 `1e91407b…a366fa`). It stopped in the birth law, `estimates.career.derive_birth_years`, called from `cohort.derive_u2_births`. For a person the law dates by the seed coordinate (age code 2–125), the law takes the birth year as the seed year minus the age and asserts that it lies between `career.DERIVED_BIRTH_MIN` and `career.DERIVED_BIRTH_MAX`, 1889 and 2016. §3's universe takes every positive-weight, in-family person of any support wave, at any age (draft-3 lines 114 and 126), so a child aged 2 or 3 in the 2021 wave or 2–5 in the 2023 wave, or a person aged 124 or 125 in the 2013 wave, reaches that assertion. The law also refuses, after its loop, every seed age code but 1–125 and 999. The individual file documents age as 0 (Inap.), 1 (newborn), 2–125 or 999 (DK, NA or refused), and the loader refuses any other code, so a code 0 on an in-family, positive-weight presence would reach that refusal; the assertion, raised inside the loop, would have masked it. The masked error record does not show which end was crossed, so the rule covers both ends and the refusal. This record states no count or value from the pass. + +**The note to §3 (amends draft-3 line 114).** Before the law runs, the rule below applies to each person of the universe whose birth year the law would take from the seed coordinate: neither the marriage history nor the observed earnings give one, as the law itself shows when run on the same frames without seed coordinates (it leaves the person unresolved). + +1. If the seed's age code is one the law refuses (0, or any code but 1–125 and 999), the seed becomes the person's earliest positive-weight, in-family presence with an age of 2–125 whose implied year lies in 1889–2016 (the person is re-seeded). A person with no such presence is counted out: under `seed_age_0` when the first code is 0, and under `no_valid_seed_age` otherwise. +2. If the seed's age is 2–125 and the year it implies is before 1889 or after 2016, the person is counted out on that side. + +A counted-out person leaves the universe and the frames the law reads. Every other person keeps the law's precedence and the record the law gives, whatever the seed implies or whatever code it carries: a marriage-history or earnings year still wins. Seed codes 1 and 999 stay unresolved. Apart from re-seeding, the seed is still the earliest positive-weight presence; a later presence never replaces a seed whose code the law takes. Code 0 is documented as Inap., not as an age, so a member of the cohort could carry it on one presence; it is therefore handled as any refused code, not counted out at once. Where code 0 marks an infant, the person's later ages imply a year after 2010, so re-seeding never dates one into 1946–55. A counted-out person had a positive-weight presence, so is not outside the universe, and the annuitant extension (line 188) does not seed it either; where such a person is an annuitant, its age takes the inherited fallback (the wave age, or missing), counted. The extension seeds only presences whose code the law takes and whose implied year it supports, as before, so it cannot reach the assertion or the refusal. + +**How the ruling's words are read.** The decision speaks of "people whose seed coordinate implies a birth year outside" the support. This record reads that as the people whose birth year the seed coordinate gives. The seed gives no birth year to a person with a marriage-history or earnings year. Counting such a person out would let the seed override the precedence §3 keeps (line 114: "with its existing history inputs and precedence") and could remove a member of the cohort. The two readings differ only for a person with both an out-of-support seed and a history or earnings year. The decision names out-of-bounds seeds only. The refusal the first pass masked would stop the one further pass the ruling authorizes, so the rule also covers seeds whose code the law refuses, toward the ruling's stated purpose: that step 4 runs once more and discloses the counts. Max approved that widening on 2026-10-20, as noted on d1724 in the decision log, including that code 0 (Inap.) is re-seeded from the earliest presence the law can date, like any refused code. The widening as described to Max also named code 999 (DK/NA) as refused. The law does not refuse 999; it leaves it unresolved, as it does a missing age, so this rule keeps 999 on that path, narrower than described, as the correction noted on d1724 the same day records. Re-seeding follows from code 0's documented meaning (above); it can keep a member of the cohort that counting out would lose, and its count is disclosed. (An earlier request from the coordinating session had counted code 0 out at once.) + +**Why no member of the cohort is lost.** A person counted out on a side has no birth-year source but a seed whose year lies outside 1889–2016. A person counted out under a refused code has no presence the law can date inside 1889–2016. That range contains 1946–55, so no counted-out person can belong to the cohort in any row, and a member of the cohort with one refused code is re-seeded and keeps its observations. + +**Precedent.** Track M counts out, before the same law, the in-family, positive-weight persons aged 0–5 in its one seed wave, 2023, whose seed (income year 2022 minus age) implies a year after 2016 (`src/populace_dynamics/min_benefit_track_m/structure.py`, the funnel count `age_0_5_counted_out_before_birth_law`, and `min_benefit_track_m/cohort.py`). U2 seeds from six waves, so its rule tests the implied year on both sides, not an age, and can re-seed a person past a refused code from a later presence; it keeps code 1 on the law's unresolved path; and it applies only to persons the seed would date, so the law's precedence is kept. + +**Disclosure.** Each reason's count is disclosed in every row's structural counts, as `n_seed_birth_year_counted_out` with the keys `before_1889`, `after_2016`, `seed_age_0` and `no_valid_seed_age`, and the re-seeded persons' count as `n_seed_birth_year_reseeded`. Each decides nothing. The rule is fixed before any count, and none was computed for this record. + +**What the ruling leaves unchanged.** + +- `src/populace_dynamics/estimates/career.py` is unchanged. +- No row, plan cell, observation weight, disposition rule, design frame, role rule or income concept changes. U2 stays unregistered and unauthorized (§20). +- The ruling authorizes one more step-4 pass, structure and then component diagnostics, under d1649's scope with `--authorized-by d1724`, once this change merges and master's Tests pass at the run head (§20 step 4). +- No count, statistic or outcome was computed to record this ruling, no PSID record was read, and no file on the restricted list was opened for reading. Reading and exposure are recorded in `docs/design/u2_m2_exposure.md` (d1724 section). + ## 17. Review record and outstanding work The adversarial review of `u2-draft-1` returned **RATIFIABLE AFTER EDITS**. Rev1 (`u2-draft-2`) applied those edits. Its round-diff check returned **REVISE**, identifying R1–R8. Round 2 (`u2-draft-3`) applied every exact edit and retained the earlier resolutions. -Milestone 1 then built the documentary source registries (`data/external/track_u2/`, master record `docs/design/u2_m1_source_adjudication.md`). The independent adjudication of that work (`EV/phase2-20260927/out/u2-adjudicate.md`) returned **ERRORS FOUND**: 18 wrong citations, now corrected; 16 items resolvable from documents, now resolved; 8 routes refused, now encoded as explicit refusals; and 24 items that need conventions, still open. Milestone 1b researched the three documentary blockers from official PSID documentation and wrote `u2-draft-4` (§16a). Max ruled on §16a on 2026-09-30, and §16b records the rulings. PSID answered the three documentary questions on 2026-10-07, Max ruled on blockers B1 and B2 on 2026-10-10, and §16c records that ruling. This document claims no new referee verdict on the specification; §20 steps 2 and 3 record the completed implementation and the reviews of every later change. +Milestone 1 then built the documentary source registries (`data/external/track_u2/`, master record `docs/design/u2_m1_source_adjudication.md`). The independent adjudication of that work (`EV/phase2-20260927/out/u2-adjudicate.md`) returned **ERRORS FOUND**: 18 wrong citations, now corrected; 16 items resolvable from documents, now resolved; 8 routes refused, now encoded as explicit refusals; and 24 items that need conventions, still open. Milestone 1b researched the three documentary blockers from official PSID documentation and wrote `u2-draft-4` (§16a). Max ruled on §16a on 2026-09-30, and §16b records the rulings. PSID answered the three documentary questions on 2026-10-07, Max ruled on blockers B1 and B2 on 2026-10-10, and §16c records that ruling. After the first structural pass stopped, Max ruled on the birth law's support on 2026-10-19, and §16d records that ruling. This document claims no new referee verdict on the specification; §20 steps 2 and 3 record the completed implementation and the reviews of every later change. What remains before ratification: @@ -1889,6 +1925,14 @@ Revision verification comprised the two touched test files and `tests/track_u2/t - Max confirmed §16c's proposed clarification (d1242): B2's disclosed count is the number of code-90 and code-92 persons in 2019, 2021 and 2023 family units that would otherwise supply any U2 observation. `sources.PERSON_COUNTED_EXCLUSIONS` now names B2 as well as B1, so the structural pass discloses both counts, and each decides nothing (§16c). Each is disclosed per row in two forms: per wave, whose sum counts a person who brings the exclusion in two waves twice, and as distinct persons across the waves, which is the number d1242 names. §16c's text, which records the proposal, is unchanged. - No count, statistic or outcome was computed, no model or estimator was run on real data, and no file on the restricted list was opened for reading. The revision's reading is recorded with the round-3 closeout lane in `docs/design/u2_m2_exposure.md`. +**`u2-draft-4`, Max's ruling on the birth law's support recorded and applied (2026-10-19, d1724):** + +- Added §16d, the §3 note d1724 names. The first step-4 pass (d1649) stopped on the birth law's support assertion without writing any output, and that assertion masked the law's refusal of seed age codes outside 1–125 and 999. Before the law runs, for a universe person whom only the seed coordinate would date, a seed whose code the law refuses is replaced by the person's earliest presence the law can date, and the person is counted out when there is none (`seed_age_0`, `no_valid_seed_age`) or when the seed implies a year before 1889 or after 2016, as Track M counts out young children. The law's precedence is kept, codes 1 and 999 keep their unresolved path, no member of the cohort can be counted out, and each count is disclosed and decides nothing. The decision names out-of-bounds seeds; on 2026-10-20 Max approved widening the rule to refused codes (noted on d1724), including that code 0, documented as Inap., is re-seeded like any refused code, so that the one further pass cannot stop on the masked refusal. The widening described to Max also named 999, which the law leaves unresolved, so the rule keeps 999 on that path (correction noted on d1724). §§16a–16c are unchanged. +- The cohort code applies it: `cohort.derive_u2_births` re-seeds (`U2Births.reseeded`) and counts the persons out (`U2Births.counted_out`), the annuitant extension never re-admits them, and every row's structural counts disclose `n_seed_birth_year_counted_out` and `n_seed_birth_year_reseeded`. `career.py` is unchanged. Two invented variants hold every case, members counted out for each reason, members at both ends of the support and two cohort members re-seeded past a refused code, and `tests/track_u2/test_u2_birth_bounds.py` holds each reason, the years 1889 and 2016, re-seeding, the counts, the unchanged cohort and other counts, the derivation completing on every case, and a property test against the law run on each person alone. +- §17's review paragraph and §20 step 4 point to §16d; step 4 records the stopped pass and the one further pass d1724 authorizes. +- Changed no line before §16a. The specification pin in `data/external/track_u2/u1_identity.json` moves to the new hash; its `version` stays `u2-draft-4`. The d1090 record test now cuts §16c's record at §16d, and a new test pins §16d line by line. +- No count, statistic or outcome was computed, no model or estimator was run on real data, and no file on the restricted list was opened for reading. Reading and exposure are recorded in `docs/design/u2_m2_exposure.md` (d1724 section). + ## 20. Ratification and execution record **Pending.** @@ -1896,7 +1940,7 @@ Revision verification comprised the two touched test files and `tests/track_u2/t 1. Resolve the proposed decisions and source-verification blockers. **Decisions: done (Max, d514, 2026-09-28, adopting every §16 default).** Source-verification blockers remain for steps 2–5. **`u2-draft-4` amendments 1–6 (§16a): ruled by Max on 2026-09-30 (d637; §16b).** B3 is resolved by option 1. B1 and B2 were held until PSID answered; **Max ruled on them on 2026-10-10 (d1090; §16c)**: B1 by option 2 and B2 by option 1, each an exclusion under a named disposition. Max confirmed the reading of amendment 1's parts 1a–1c in §16b on 2026-10-12 (d745), so this step is done. 2. Complete implementation, manifests and parameter capture. **Done (2026-10-16).** Implementation: milestone 2 (#487) and its three review rounds, the last landing as #525 (`f7a84d0b`); and #544, which records with the frame digest the pandas version and string inference setting it was taken under and refuses a frozen load or registered run under any other (`test_u2_loader.py::test_any_other_frame_environment_refuses_the_registered_check`, `::test_a_frozen_load_refuses_another_frame_environment`). Manifests: the documentary source records, 1,766 registry entries and 26 SSI and Census records, are all RESOLVED, none REFUSED or TO VERIFY (`test_adjudication_applied.py::test_status_counts_after_independent_adjudication`), and the loader's source preflight passes (`test_u2_mappings.py::test_source_preflight_passes_at_the_committed_registries`; #524, #540). A RESOLVED record is a documentary finding; reconciliation on real records is step 4's. Parameter capture: the U2 SSI capture, the Track M threshold capture and U1's inherited life tables, each pinned by hash in the code (§17 item 5; `test_u2_parameters.py::test_threshold_capture_is_the_pinned_track_m_file`; `parameters._LIFE_TABLE_PINS`). The final parameter block is step 5's. 3. Complete independent mapping review, invented tests, historical-isolation checks and exact U1 differential evidence. **Done (2026-10-16).** Independent mapping review: the independent adjudication of milestone 1's registries (ERRORS FOUND); milestone 1b applied its corrections in #486, whose independent review requested changes on 11 findings and whose blind verifier then returned U2-M1B: VERIFIED (`test_adjudication_applied.py::test_dispositions_match_the_adjudication`; §17). Every later registry change had an independent review before it merged: the d1090 rounds (§19), ending in APPROVE at round 4 (`e7a558f`) and at #524's merged head (`742590b`; #524's merge audit); the d637 and d745 commit reviews, the last a delta review, APPROVE (§19); and #540's gating review, an Opus 5.5 review with a shell, APPROVE (#540's merge audit). Invented tests and historical isolation: the milestone-2 report's §13 and §14 audit (#487, #525), closed by three review rounds and a fourth-pass mutation run in which a test kills every mutant; §13's optional structure-only U1 load needs Max's approval and is not run. Exact U1 differential evidence: §17 item 7. -4. Complete the authorized pre-registration structural, reconciliation and F17 component pass; resolve mapping failures. The pass reads real PSID records, so it needs Max's authorization, recorded as a chief-of-staff decision. `scripts/track_u2_structure.py` and `scripts/track_u2_component_diagnostics.py` refuse to run without `--authorized-by `, exiting non-zero before any PSID path is resolved or opened, and the loader refuses an unfrozen load without the same record. The flag records the decision id, the git head and the UTC time in the pass's output; it only records the authorization and does not grant it. +4. Complete the authorized pre-registration structural, reconciliation and F17 component pass; resolve mapping failures. The pass reads real PSID records, so it needs Max's authorization, recorded as a chief-of-staff decision. `scripts/track_u2_structure.py` and `scripts/track_u2_component_diagnostics.py` refuse to run without `--authorized-by `, exiting non-zero before any PSID path is resolved or opened, and the loader refuses an unfrozen load without the same record. The flag records the decision id, the git head and the UTC time in the pass's output; it only records the authorization and does not grant it. **First pass (d1649): stopped.** It ran once, on 2026-10-19 at `640adccf`, and stopped in the birth law without writing any output; the component diagnostics were not run (§16d). **Max's ruling of 2026-10-19 (d1724; §16d)** authorizes one more pass, structure and then component diagnostics, under d1649's scope with `--authorized-by d1724`, once the §16d change merges and master's Tests pass at the run head. 5. Materialize the final parameter block and U2 rulings record; require exact code/specification agreement. 6. Ratify the specification by merge. 7. Obtain and independently check the fresh forecast. diff --git a/docs/design/u2_m2_exposure.md b/docs/design/u2_m2_exposure.md index 7a2d63ee9..ce06615e2 100644 --- a/docs/design/u2_m2_exposure.md +++ b/docs/design/u2_m2_exposure.md @@ -353,3 +353,23 @@ The hook refused nothing at any of these runs. Its only refusals were the self-t **The d1242 change.** After #544 merged (`2ef10532`), the same lane applied Max's d1242 ruling: B2 joins `sources.PERSON_COUNTED_EXCLUSIONS`, with its tests and a §19 entry. It read only the decision ledger's d1242, d1597 and d1649 entries, the code it changed and the roles tests. It ran the U2 files under the same hook, which refused nothing. Answering #546's review, it added an invented variant (a second, sex-unknown code-92 member in one excluded 2019 unit) and a distinct-person form of each count. For this it read the review, the invented-data generator, the runner and the scripts test. It ran the reviewer's mutations and the U2 files under the same hook, which refused nothing. **Values:** this lane and its lanes stated and inferred no 1946–55 value or direction. Every number they printed is invented (offsets, invented cells, invented-run counts and dispositions), a count, a hash, a timing or a public parameter from the committed captures. + +## d1724 lane (2026-10-19) + +A Claude Code session (Claude Opus 5.5), started by the dynamics coordinator session (`local_6085d52a-…`), applied Max's ruling d1724 (specification §16d) in a standalone clone of `origin/master` at `640adccf`. It read no PSID record and no comparator, and it does not forecast U2. + +**Read:** + +- `EV/RESTRICTED-FILES.md` itself, in full, as its brief required, to know what not to open. It opened nothing the file lists. +- The coordinator's shared notes folder: the step-4 pass session's value-free note and the d637 registry session's note. +- The chief-of-staff decision ledger's d1649 and d1724 entries (decision, ruling and status lines), and, on 2026-10-20, d1724's two notes (Max's approval of the widening, and the coordinating session's correction on code 999). +- In `EV/track-u2-step4-pass-20261019/`: the directory listing (file names only) and the SHA-256 of `MANIFEST.md`, computed without printing it. It did not open `track-u2-structure.stderr.txt` (restricted), its masked copy, or any other file there. +- The repository: the U2 package, `estimates/career.py` (`derive_birth_years` and its constants, unchanged), Track M's `structure.py` and `cohort.py` (the count-out precedent), the U2 scripts and tests, `tests/conftest.py`, the specification (its header and §§3, 16c, 17, 19 and 20, and the tests' pins) and this file. + +**Ran:** the U2 test files, the new `tests/track_u2/test_u2_birth_bounds.py` among them, with `POPULACE_DYNAMICS_PSID_DIR` set to an empty directory, which stayed empty; so the checks that need PSID documentation skipped. It ran `career.derive_birth_years` and master's `cohort.derive_u2_births` (read from Git at `640adccf`) on invented inputs only, to show the fix equals master wherever master's law completes. It ran Black, Ruff, the tier recount by collection and the mutation checks listed in the PR. + +**Values:** it stated and inferred no 1946–55 value or direction. Every number it printed is invented (generator persons, ages and counts), a test count, a hash, or a constant of the code (`career.DERIVED_BIRTH_MIN` and `DERIVED_BIRTH_MAX`, 1889 and 2016). + +**2026-10-20, the widened rule.** The coordinating session asked that the rule also cover seeds whose age code the law refuses, and later reported that Max approved that widening, re-seeding of code 0 included. The lane read the coordinator's messages, d1724's two notes in the decision ledger (to cite the approval and its correction), the loader's documented age domain and its value check (`loader.py`), and the code and tests it changed. It read nothing else outside the repository, and its runs were as above: an empty PSID directory, invented inputs only, and the mutation check, all listed in the PR. + +**Review lane.** Subfleet `20261020-030251-d1724-review-552b` reviewed head `8a6a8192` and returned APPROVE. No Codex lane could admit the job, so Subfleet served it on Claude Opus 5.5 (lane `claude-7`), in its own standalone clone with its own virtual environment. The lane reports that its empty PSID directory stayed empty and that it opened nothing under the evidence directory, `REFS` or `PSID`. It loaded master's `cohort.py` from Git and ran invented inputs only. Its scratch scripts were in `/tmp/review552`. Its findings (two test gaps, one record-accuracy point, nits) were addressed in the next commit, which declined one (the header's version line, pinned as d1090's `header_version` clause). The delta review (Subfleet `20261020-045107-d1724-review-552-delta`, GPT-6.1 Sol) approved that commit. Records: `EV/rulings-records-20261019-d1724/review-552-out.md`. diff --git a/scripts/track_u2_structure.py b/scripts/track_u2_structure.py index 3a08ee55f..13c7ce2c9 100644 --- a/scripts/track_u2_structure.py +++ b/scripts/track_u2_structure.py @@ -7,6 +7,14 @@ committed-registry role context and writes ``track-u2-structure.json``: dispositions, observations, roles, pairings, design strata, the family income and WEALTH1 identity counts per wave and the registry status. +Each row's counts include ``n_seed_birth_year_counted_out``: the persons +counted out before the birth law, by reason -- a seed year before 1889 +(``before_1889``) or after 2016 (``after_2016``), or no presence with an +age the law can date (``seed_age_0``, ``no_valid_seed_age``) -- and +``n_seed_birth_year_reseeded``, the persons re-seeded past an age code +the law refuses (d1724, specification section 16d; +:func:`populace_dynamics.uniform_cut_track_u2.cohort.derive_u2_births`). +Each is disclosed and decides nothing. It also writes ``track-u2-preregistration-evidence.json``: the frozen input identities section 14 says execution rechecks -- the SHA-256 of diff --git a/src/populace_dynamics/uniform_cut_track_u2/cohort.py b/src/populace_dynamics/uniform_cut_track_u2/cohort.py index b70984f0f..bd98e5d6d 100644 --- a/src/populace_dynamics/uniform_cut_track_u2/cohort.py +++ b/src/populace_dynamics/uniform_cut_track_u2/cohort.py @@ -29,7 +29,16 @@ U1's per-row ``earliest_presence_wave``, which :class:`U2CohortSpec` refuses). Births are derived once (:func:`derive_u2_births`) and reused by every row. Support waves identify people; they add no observation -and no design unit to a row. +and no design unit to a row. Max's ruling of 2026-10-19 (d1724; section +16d) keeps every input the law would refuse away from it, for the +persons whose birth year only the seed coordinate could give: a seed +whose age code the law refuses (0, or any code but 1-125 and 999) is +replaced by the person's earliest presence with an age of 2-125 whose +implied year the law supports; a person with no such presence, or whose +seed implies a year outside the law's derived-age support +(``career.DERIVED_BIRTH_MIN``-``DERIVED_BIRTH_MAX``, 1889-2016), is +counted out before the law runs, as Track M counts out young children. +Each reason's count is disclosed and decides nothing. **Roles** (section 3 relationship-code amendment) come from a :class:`~populace_dynamics.uniform_cut_track_u2.sources.RoleContext`. @@ -126,6 +135,7 @@ "PSID_FILES", "SNAPSHOT_KEY", "ROWS", + "SEED_BIRTH_COUNTED_OUT", "SEED_WAVE_RULE", "SUPPORT_WAVES", "TARGET_AGE", @@ -162,6 +172,21 @@ #: changes on U0's population (:mod:`.rows`). ROWS: tuple[str, ...] = ("U0", "U1") SEED_WAVE_RULE = "earliest_presence_in_common_support_waves" +#: d1724 (section 16d): the reasons :func:`derive_u2_births` counts a +#: person out before the birth law, each disclosed. The law +#: (``career.derive_birth_years``, read-only under section 14) asserts +#: that a year it derives from a seed (age code 2-125) lies in +#: ``[DERIVED_BIRTH_MIN, DERIVED_BIRTH_MAX]``, and refuses every seed age +#: code but 1-125 and 999. A person the seed alone would date is counted +#: out when the seed implies a year before or after that support, or +#: when no presence carries an age the law can date (the first code 0, +#: or another refused code). +SEED_BIRTH_COUNTED_OUT: tuple[str, ...] = ( + f"before_{career.DERIVED_BIRTH_MIN}", + f"after_{career.DERIVED_BIRTH_MAX}", + "seed_age_0", + "no_valid_seed_age", +) U1_EVEN_BIRTH_YEAR_WEIGHT = 0.5 U1_SINGLE_OBSERVATION_WEIGHT = 1.0 U1_EVEN_BIRTH_AGES: tuple[int, int] = (66, 68) @@ -198,6 +223,12 @@ ("died", 81, 89), ) _MAX_AGE_CODE = 125 +#: The seed age codes career's derived-age clause turns into a birth year +#: (``seed year - age``); 1, 999 and missing are unresolved there, and any +#: other code is refused there. +_DERIVED_SEED_AGES = (2, 125) +#: The documented DK/NA/refused age code, which the law leaves unresolved. +_NA_AGE_CODE = 999 _FAMILY_UNIT_SCALE = 100_000 _UNRESOLVED_STATES = ("unknown", "no_marriage_history") _OMITTED_ROWS = ( @@ -955,12 +986,25 @@ def _check_role_context( # =========================================================================== @dataclass(frozen=True) class U2Births: - """Birth-year records for the universe and the annuitant extension.""" + """Birth-year records for the universe and the annuitant extension. + + ``counted_out`` maps each reason of :data:`SEED_BIRTH_COUNTED_OUT` to + the persons counted out of the universe before the law ran (d1724); + they have no record, so their birth year is ``None``. ``reseeded`` + holds the persons whose earliest presence carried an age code the law + refuses and whose seed is their earliest presence it can date. + """ records: Mapping[int, career.BirthYearRecord] universe: frozenset[int] seed: pd.DataFrame inputs_sha256: str + counted_out: Mapping[str, frozenset[int]] = field( + default_factory=lambda: MappingProxyType( + {reason: frozenset() for reason in SEED_BIRTH_COUNTED_OUT} + ) + ) + reseeded: frozenset[int] = frozenset() def birth_year(self, person_id: int) -> int | None: record = self.records.get(int(person_id)) @@ -1048,16 +1092,110 @@ def check_person_joins(inputs: U2Inputs) -> None: ) +def _seed_for_the_law( + presences: list[pd.DataFrame], + history: pd.DataFrame, + earnings: pd.DataFrame, +) -> tuple[pd.DataFrame, dict[str, frozenset[int]], frozenset[int]]: + """The seed rows the law can take, the persons counted out by reason, + and the persons re-seeded (d1724, section 16d). + + ``presences`` holds every positive-weight, in-family presence of every + support wave. The law reaches a seed only for a person whose birth + year neither the marriage history nor the observed earnings give, so + the law itself, run on the same frames without seed coordinates, + names those persons (its unresolved records); for everyone else the + earliest presence stays the seed and the law's precedence decides. + For each person the seed alone would date: + + * if the earliest presence's age code is one the law refuses (0, or + any code but 1-125 and 999), the seed is the person's earliest + presence with an age of 2-125 whose implied year the law supports + (re-seeded); with none, the person is counted out under + ``seed_age_0`` (earliest code 0) or ``no_valid_seed_age``; + * if the seed's age is 2-125 and its year (``seed year - age``) lies + before 1889 or after 2016, the person is counted out on that side. + + Codes 1 and 999 keep the law's unresolved path. + """ + + first = _seed_frame(presences) + people = set(int(pid) for pid in first["person_id"]) + clauses = career.derive_birth_years( + history[history["person_id"].isin(people)], + earnings[earnings["person_id"].isin(people)], + required_person_ids=people, + ) + seed_only = { + record.person_id + for record in clauses + if record.source is career.BirthSource.UNRESOLVED + } + ages = first["age"].astype("int64") + accepted = ages.eq(_NA_AGE_CODE) | ages.between(1, _MAX_AGE_CODE) + refused = first["person_id"].isin(seed_only) & ~accepted + seed = first + reseeded: frozenset[int] = frozenset() + if refused.any(): + every = pd.concat(presences, ignore_index=True) + every_ages = every["age"].astype("int64") + every_years = every["year"].astype("int64") - every_ages + valid = every[ + every_ages.between(*_DERIVED_SEED_AGES) + & every_years.between( + career.DERIVED_BIRTH_MIN, career.DERIVED_BIRTH_MAX + ) + & every["person_id"].isin(first.loc[refused, "person_id"]) + ] + replacement = _seed_frame([valid]) + reseeded = frozenset(int(pid) for pid in replacement["person_id"]) + seed = ( + pd.concat([first[~refused], replacement], ignore_index=True) + .astype(first.dtypes.to_dict()) + .sort_values("person_id", kind="stable") + .reset_index(drop=True) + ) + unseeded = refused & ~first["person_id"].isin(reseeded) + seed_ages = seed["age"].astype("int64") + implied = seed["year"].astype("int64") - seed_ages + derived = seed["person_id"].isin(seed_only) & seed_ages.between( + *_DERIVED_SEED_AGES + ) + before = derived & (implied < career.DERIVED_BIRTH_MIN) + after = derived & (implied > career.DERIVED_BIRTH_MAX) + rows = ( + seed.loc[before, "person_id"], + seed.loc[after, "person_id"], + first.loc[unseeded & ages.eq(0), "person_id"], + first.loc[unseeded & ~ages.eq(0), "person_id"], + ) + counted_out = { + reason: frozenset(int(pid) for pid in persons) + for reason, persons in zip(SEED_BIRTH_COUNTED_OUT, rows, strict=True) + } + kept = seed[~(before | after)].reset_index(drop=True) + return kept, counted_out, reseeded + + def derive_u2_births(inputs: U2Inputs) -> U2Births: """The birth-year law over the common support waves (section 3). Universe: every person with a positive-weight, in-family presence in any support wave; seed coordinate: the earliest such presence - (``(wave - 1) - age``). Annuitant extension: in-family persons of - the administrative birth-support codes (10, 20, 22, 88, 90; not 92) - and marriage-history spouses of target-cohort members outside the - universe, seeded by their earliest in-family presence at any weight; - the law runs without a required population for them, so a conflict + (``(wave - 1) - age``). Before the law runs, for a person whose + birth year only the seed could give, a seed age code the law refuses + is replaced by the earliest presence it can date (``reseeded``), and + a person with no such presence, or whose seed implies a year outside + the law's derived-age support (1889-2016), is counted out of the + universe under its reason (d1724, section 16d; + :func:`_seed_for_the_law`); ``counted_out`` holds them, and no other + person's record changes. Annuitant extension: in-family + persons of the administrative birth-support codes (10, 20, 22, 88, + 90; not 92) and marriage-history spouses of target-cohort members + outside the universe, seeded by their earliest in-family presence at + any weight whose implied year the law supports (code 999 kept); a + counted-out person is not outside the universe, so never joins it. + The law runs without a required population for them, so a conflict leaves the person unresolved (the wave-age fallback, counted). """ @@ -1080,10 +1218,13 @@ def derive_u2_births(inputs: U2Inputs) -> U2Births: } ) ) - seed = _seed_frame(seeds) - universe = frozenset(int(pid) for pid in seed["person_id"]) history = inputs.marriage_history earnings = inputs.observed_earnings + present_ids = frozenset( + int(pid) for frame in seeds for pid in frame["person_id"] + ) + seed, counted_out, reseeded = _seed_for_the_law(seeds, history, earnings) + universe = frozenset(int(pid) for pid in seed["person_id"]) records = career.derive_birth_years( history[history["person_id"].isin(universe)], earnings[earnings["person_id"].isin(universe)], @@ -1111,7 +1252,7 @@ def derive_u2_births(inputs: U2Inputs) -> U2Births: ) | anchor["person_id"].isin(spouses) ) - & ~anchor["person_id"].isin(universe) + & ~anchor["person_id"].isin(present_ids) ] ages = rows["age"].astype("int64") seed_birth = (wave - 1) - ages @@ -1150,6 +1291,8 @@ def derive_u2_births(inputs: U2Inputs) -> U2Births: universe=universe, seed=seed, inputs_sha256=input_frames_sha256(inputs), + counted_out=MappingProxyType(counted_out), + reseeded=reseeded, ) @@ -1535,6 +1678,17 @@ def build_u2_cohort( "n_unresolved_birth_year": sum( births.birth_year(pid) is None for pid in outside ), + # d1724 (section 16d): the persons counted out of the universe + # before the birth law, whom only the seed could date, by reason + # (a seed year before 1889 or after 2016; no presence with an age + # the law can date, the first code 0 or another), and the persons + # re-seeded past an age code the law refuses. Disclosed; each + # decides nothing. + "n_seed_birth_year_counted_out": { + reason: len(births.counted_out[reason]) + for reason in SEED_BIRTH_COUNTED_OUT + }, + "n_seed_birth_year_reseeded": len(births.reseeded), "plan_birth_years": plan_years, "role_context": role_context.kind, "births_inputs_sha256": births.inputs_sha256, diff --git a/src/populace_dynamics/uniform_cut_track_u2/invented.py b/src/populace_dynamics/uniform_cut_track_u2/invented.py index 201664d1b..a679ef825 100644 --- a/src/populace_dynamics/uniform_cut_track_u2/invented.py +++ b/src/populace_dynamics/uniform_cut_track_u2/invented.py @@ -1595,6 +1595,166 @@ def _code_88_target_with_ofum_income_2019( return dataclasses.replace(changed, marriage_history=history) +def _host(base: cohort.U2Inputs, waves: tuple[int, ...]) -> int: + """The first target with a U1 observation in the earliest of ``waves`` + who is in-family with a positive weight in every one of them: its + family supplies an observation in the member's first wave.""" + + built = cohort.build_u2_cohort( + base, + cohort.U2CohortSpec(row="U1"), + role_context=sources.RoleContext.declared(), + ) + obs = built.observations + first = obs.loc[obs["wave"].eq(min(waves)), "person_id"].astype(int) + for person in sorted(set(first)): + if all( + bool( + ( + base.anchors[wave]["person_id"].eq(person) + & base.anchors[wave]["sequence"].between(1, 20) + & base.anchors[wave]["weight"].gt(0) + ).any() + ) + for wave in waves + ): + return person + raise AssertionError(f"no U1 target can host a member in {waves}") + + +def _added_member( + base: cohort.U2Inputs, + person_id: int, + *, + host: int, + ages: Mapping[int, int], + relationship: int, + sex: str, +) -> cohort.U2Inputs: + """``base`` with one OFUM member in ``host``'s family in each wave of + ``ages``, at that age, with a positive weight, and absent elsewhere. + + The member takes the host's stratum and cluster, so no row's design + frame gains a unit, and the next free sequence number in each family. + """ + + anchors = {} + for wave, frame in base.anchors.items(): + row = { + "person_id": person_id, + "interview": 0, + "sequence": 0, + "relationship": 0, + "age": 0, + "reported_birth_year": pd.NA, + "weight": 0.0, + } + if wave in ages: + interview = int( + frame.loc[frame["person_id"].eq(host), "interview"].iloc[0] + ) + family = frame[ + frame["interview"].eq(interview) + & frame["sequence"].between(1, 20) + ] + row.update( + interview=interview, + sequence=int(family["sequence"].max()) + 1, + relationship=relationship, + age=int(ages[wave]), + weight=500.0, + ) + anchors[wave] = pd.concat( + [frame, pd.DataFrame([row])], ignore_index=True + ).astype(frame.dtypes.to_dict()) + design = base.design.set_index("person_id").loc[host] + design = pd.concat( + [ + base.design, + pd.DataFrame( + [ + { + "person_id": person_id, + "stratum": int(design["stratum"]), + "cluster": int(design["cluster"]), + } + ] + ), + ], + ignore_index=True, + ).astype("int64") + persons = pd.concat( + [base.persons, pd.DataFrame([{"person_id": person_id, "sex": sex}])], + ignore_index=True, + ).astype({"person_id": "int64", "sex": "string"}) + return dataclasses.replace( + base, anchors=anchors, design=design, persons=persons + ) + + +#: The members the two d1724 variants add (section 16d). None has a +#: marriage history or earnings, so the seed is its only birth-year +#: source: its earliest positive-weight in-family presence, or, where +#: that presence's age code is one the birth law refuses, its earliest +#: presence with an age of 2-125 whose implied year (``(wave - 1) - +#: age``) the law supports. Each entry: person id, ``{wave: age code}`` +#: (the waves it is present in, in the family of :func:`_host`), the +#: relationship (30 child, 40 sibling, 50 parent, 60 grandchild; each +#: OFUM), sex. Code 998 stands for an age code outside the loader's +#: documented domain (0, 1-125, 999), which the law refuses. +_SEED_BIRTH_COUNTED_OUT_MEMBERS = ( + # 2020 - 3 = 2017: after 2016 (also present in 2023). + (_PERSON_BASE - 11, {2021: 3, 2023: 5}, 60, "female"), + # 2022 - 2 = 2020: after 2016. + (_PERSON_BASE - 12, {2023: 2}, 30, "male"), + # 2012 - 124 = 1888: before 1889. + (_PERSON_BASE - 13, {2013: 124}, 50, "female"), + # First code 0; the later age implies 2020, outside the support: no + # presence the law can date (seed_age_0). + (_PERSON_BASE - 16, {2021: 0, 2023: 2}, 60, "male"), + # First code 998; the later code 1 dates nothing (no_valid_seed_age). + (_PERSON_BASE - 17, {2017: 998, 2019: 1}, 30, "female"), +) +_SEED_BIRTH_KEPT_MEMBERS = ( + # 2012 - 123 = 1889: the support's first year. + (_PERSON_BASE - 14, {2013: 123}, 50, "male"), + # 2018 - 2 = 2016: the support's last year (later waves agree). + (_PERSON_BASE - 15, {2019: 2, 2021: 4, 2023: 6}, 60, "female"), + # First code 0, then 66 in 2017: re-seeded, born 1950, a U1 target + # observed in 2017 and 2019. + (_PERSON_BASE - 18, {2015: 0, 2017: 66, 2019: 68}, 40, "female"), + # First code 998 in 2013, then 68 in 2015: re-seeded, born 1946, a U1 + # target observed in both waves, the one with the refused code too. + (_PERSON_BASE - 19, {2013: 998, 2015: 68}, 40, "male"), +) + + +def _with_members( + base: cohort.U2Inputs, members: tuple[tuple[Any, ...], ...] +) -> cohort.U2Inputs: + changed = base + for person_id, ages, relationship, sex in members: + changed = _added_member( + changed, + person_id, + host=_host(base, tuple(sorted(ages))), + ages=ages, + relationship=relationship, + sex=sex, + ) + return changed + + +def _seed_births_counted_out(base: cohort.U2Inputs) -> cohort.U2Inputs: + return _with_members(base, _SEED_BIRTH_COUNTED_OUT_MEMBERS) + + +def _seed_births_every_case(base: cohort.U2Inputs) -> cohort.U2Inputs: + return _with_members( + base, _SEED_BIRTH_COUNTED_OUT_MEMBERS + _SEED_BIRTH_KEPT_MEMBERS + ) + + # Appended in order, so every earlier variant is unchanged. _VARIANT_BUILDERS = { "code_88_first_year_cohabitor": _code_88_first_year_cohabitor, @@ -1621,6 +1781,8 @@ def _code_88_target_with_ofum_income_2019( "second_trigger_in_excluded_unit_2019": ( _second_trigger_in_excluded_unit_2019 ), + "seed_births_counted_out": _seed_births_counted_out, + "seed_births_every_case": _seed_births_every_case, } _VARIANT_NOTES: dict[str, str] = { "code_88_first_year_cohabitor": ( @@ -1706,6 +1868,27 @@ def _code_88_target_with_ofum_income_2019( "and disposition is unchanged, but B2's person count (d1242) rises " "by one, since it counts persons, not family units, of any sex" ), + "seed_births_counted_out": ( + "five OFUM members with no marriage history or earnings join, in " + "their first wave, families that supply a U1 observation in that " + "wave; under d1724 (section 16d) the birth law cannot date any of " + "them, so each is counted out before it runs: a grandchild aged 3 " + "in 2021 (implied 2017) and a child aged 2 in 2023 (2020), after " + "2016; a parent aged 124 in 2013 (1888), before 1889; a grandchild " + "first coded 0 whose later age implies 2020 (seed_age_0); and a " + "child first coded 998, then 1 (no_valid_seed_age). Every " + "observation, disposition and other count is unchanged" + ), + "seed_births_every_case": ( + "the five counted-out members of seed_births_counted_out, plus four " + "kept under d1724 (section 16d): a parent aged 123 in 2013 (born " + "1889) and a grandchild aged 2 in 2019 (born 2016), at the ends of " + "the birth law's derived-age support; and two siblings whose " + "earliest presence carries an age code the law refuses (0 in 2015; " + "998 in 2013) and whose later age dates them, re-seeded: U1 " + "targets born 1950 and 1946, each with its own observations. The " + "law completes on every case" + ), } #: The generator's named variants (see :data:`_VARIANT_NOTES`). INVENTED_VARIANTS: tuple[str, ...] = tuple(_VARIANT_BUILDERS) diff --git a/tests/README-tiers.md b/tests/README-tiers.md index 62c0f6f75..8b52da848 100644 --- a/tests/README-tiers.md +++ b/tests/README-tiers.md @@ -39,9 +39,9 @@ pytest --collect-only -q -m oracle_policyengine | tail -1 | Tier | Tests at HEAD | |---|---:| -| `unit` | 9,916 | +| `unit` | 9,958 | | `artifact` | 4,253 | -| `integration_psid` | 1,423 | +| `integration_psid` | 1,450 | | `reproduction_legacy` | 520 | | `oracle_policyengine` | 308 | -| **Total** | **16,420** | +| **Total** | **16,489** | diff --git a/tests/tier_counts.json b/tests/tier_counts.json index 88eaa58e6..3841240d4 100644 --- a/tests/tier_counts.json +++ b/tests/tier_counts.json @@ -1,9 +1,9 @@ { "schema_version": 1, "counts": { - "unit": 9916, + "unit": 9958, "artifact": 4253, - "integration_psid": 1423, + "integration_psid": 1450, "reproduction_legacy": 520, "oracle_policyengine": 308 } diff --git a/tests/track_u2/test_adjudication_applied.py b/tests/track_u2/test_adjudication_applied.py index bd70e3574..e3df6e62a 100644 --- a/tests/track_u2/test_adjudication_applied.py +++ b/tests/track_u2/test_adjudication_applied.py @@ -1604,7 +1604,8 @@ def _heading_line(lines, number): def d1090_locations(spec): """The text of each place the d1090 record stands in ``spec``, cut at - the section heading lines.""" + the section heading lines. Section 16c's record ends where section + 16d (d1724, a later ruling) begins.""" lines = spec.split("\n") def section(first, last=None): @@ -1615,7 +1616,7 @@ def section(first, last=None): return { "header_version": lines[2], "header_status": lines[3], - "record": section("16c", "17"), + "record": section("16c", "16d"), "outstanding": section("17", "18"), "execution": section("20"), } @@ -1623,8 +1624,9 @@ def section(first, last=None): #: The sections the record's places are cut at, and section 20. Each must #: open with exactly one ATX heading, in any Markdown spacing, outside code -#: (reviews of 0c4eab5 and 91695a5). -D1090_SECTIONS = ("16a", "16b", "16c", "17", "18", "20") +#: (reviews of 0c4eab5 and 91695a5). Section 16d (d1724) ends 16c's +#: record, so it is held to the same rule. +D1090_SECTIONS = ("16a", "16b", "16c", "16d", "17", "18", "20") #: Pinned lines that stand more than once in the whole specification, #: whitespace ignored, with their counts; every other pinned line stands #: once. "What the ruling does not change." also heads part of §16b. @@ -2109,6 +2111,362 @@ def test_the_d1090_checks_catch_each_mutation(place, old, new): check_d1090_record(mutated) +#: Max's ruling d1724 (2026-10-19): every non-blank line of section 16d, +#: the section 3 note the ruling names, in order. +D1724_RECORD = [ + "## 16d. Max's ruling on the birth law's support (2026-10-19, " "d1724)", + "> **RULED on 2026-10-19.** Max ruled on 2026-10-19 (decision " + "d1724), after the first \N{SECTION SIGN}20 step-4 structural pass " + "stopped before writing any output, and on 2026-10-20 approved the " + "widening this section records. This section is the " + "\N{SECTION SIGN}3 note the ruling names. \N{SECTION SIGN}3 keeps " + "its draft-3 text and line numbers, because the registries cite " + "them (\N{SECTION SIGN}16c), so the note stands here and amends " + "\N{SECTION SIGN}3 as \N{SECTION SIGN}\N{SECTION SIGN}16b and 16c " + "do; where it and \N{SECTION SIGN}3 differ, it governs. " + "\N{SECTION SIGN}\N{SECTION SIGN}16a\N{EN DASH}16c are unchanged.", + "The decision put to Max, as recorded in the chief-of-staff " + "decision log:", + '> "dynamics U2 \N{SECTION SIGN}20 step 4, second run: after the ' + "d1649 run stopped on a career-bound assertion with no output, fix " + "U2's birth universe so it counts out people whose seed coordinate " + "implies a birth year outside career's [1889, 2016] (as Track M " + "does; career.py untouched), then rerun step 4 once under d1649's " + 'scope"', + "The ruling, as recorded there:", + '> "Yes: fix, then rerun once. A reviewed PR counts out the ' + "out-of-bounds people in U2's cohort code and discloses the counts, " + "with a \N{SECTION SIGN}3 note; career.py is untouched. Once it " + "merges and master's Tests pass at the run head, step 4 (structure, " + "then component diagnostics) runs once more under d1649's scope, " + "with --authorized-by this decision. Max, dynamics coordinator " + 'session 2026-10-19 (guided AskUserQuestion)."', + "**What stopped the first pass.** The pass Max authorized on " + "2026-10-16 (d1649) ran once, on 2026-10-19 at `640adccf`, and " + "exited without writing any output; the component diagnostics were " + "not run (evidence: `EV/track-u2-step4-pass-20261019/MANIFEST.md`, " + "SHA-256 `1e91407b\N{HORIZONTAL ELLIPSIS}a366fa`). It stopped in " + "the birth law, `estimates.career.derive_birth_years`, called from " + "`cohort.derive_u2_births`. For a person the law dates by the seed " + "coordinate (age code 2\N{EN DASH}125), the law takes the birth " + "year as the seed year minus the age and asserts that it lies " + "between `career.DERIVED_BIRTH_MIN` and `career.DERIVED_BIRTH_MAX`, " + "1889 and 2016. \N{SECTION SIGN}3's universe takes every " + "positive-weight, in-family person of any support wave, at any age " + "(draft-3 lines 114 and 126), so a child aged 2 or 3 in the 2021 " + "wave or 2\N{EN DASH}5 in the 2023 wave, or a person aged 124 or " + "125 in the 2013 wave, reaches that assertion. The law also " + "refuses, after its loop, every seed age code but 1\N{EN DASH}125 " + "and 999. The individual file documents age as 0 (Inap.), 1 " + "(newborn), 2\N{EN DASH}125 or 999 (DK, NA or refused), and the " + "loader refuses any other code, so a code 0 on an in-family, " + "positive-weight presence would reach that refusal; the assertion, " + "raised inside the loop, would have masked it. The masked error " + "record does not show which end was crossed, so the rule covers " + "both ends and the refusal. This record states no count or value " + "from the pass.", + "**The note to \N{SECTION SIGN}3 (amends draft-3 line 114).** " + "Before the law runs, the rule below applies to each person of the " + "universe whose birth year the law would take from the seed " + "coordinate: neither the marriage history nor the observed earnings " + "give one, as the law itself shows when run on the same frames " + "without seed coordinates (it leaves the person unresolved).", + "1. If the seed's age code is one the law refuses (0, or any code " + "but 1\N{EN DASH}125 and 999), the seed becomes the person's " + "earliest positive-weight, in-family presence with an age of " + "2\N{EN DASH}125 whose implied year lies in 1889\N{EN DASH}2016 " + "(the person is re-seeded). A person with no such presence is " + "counted out: under `seed_age_0` when the first code is 0, and " + "under `no_valid_seed_age` otherwise.", + "2. If the seed's age is 2\N{EN DASH}125 and the year it implies is " + "before 1889 or after 2016, the person is counted out on that side.", + "A counted-out person leaves the universe and the frames the law " + "reads. Every other person keeps the law's precedence and the " + "record the law gives, whatever the seed implies or whatever code " + "it carries: a marriage-history or earnings year still wins. Seed " + "codes 1 and 999 stay unresolved. Apart from re-seeding, the seed " + "is still the earliest positive-weight presence; a later presence " + "never replaces a seed whose code the law takes. Code 0 is " + "documented as Inap., not as an age, so a member of the cohort " + "could carry it on one presence; it is therefore handled as any " + "refused code, not counted out at once. Where code 0 marks an " + "infant, the person's later ages imply a year after 2010, so " + "re-seeding never dates one into 1946\N{EN DASH}55. A counted-out " + "person had a positive-weight presence, so is not outside the " + "universe, and the annuitant extension (line 188) does not seed it " + "either; where such a person is an annuitant, its age takes the " + "inherited fallback (the wave age, or missing), counted. The " + "extension seeds only presences whose code the law takes and whose " + "implied year it supports, as before, so it cannot reach the " + "assertion or the refusal.", + "**How the ruling's words are read.** The decision speaks of " + '"people whose seed coordinate implies a birth year outside" the ' + "support. This record reads that as the people whose birth year the " + "seed coordinate gives. The seed gives no birth year to a person " + "with a marriage-history or earnings year. Counting such a person " + "out would let the seed override the precedence \N{SECTION SIGN}3 " + 'keeps (line 114: "with its existing history inputs and ' + 'precedence") and could remove a member of the cohort. The two ' + "readings differ only for a person with both an out-of-support seed " + "and a history or earnings year. The decision names out-of-bounds " + "seeds only. The refusal the first pass masked would stop the one " + "further pass the ruling authorizes, so the rule also covers seeds " + "whose code the law refuses, toward the ruling's stated purpose: " + "that step 4 runs once more and discloses the counts. Max approved " + "that widening on 2026-10-20, as noted on d1724 in the decision " + "log, including that code 0 (Inap.) is re-seeded from the earliest " + "presence the law can date, like any refused code. The widening as " + "described to Max also named code 999 (DK/NA) as refused. The law " + "does not refuse 999; it leaves it unresolved, as it does a missing " + "age, so this rule keeps 999 on that path, narrower than described, " + "as the correction noted on d1724 the same day records. Re-seeding " + "follows from code 0's documented meaning (above); it can keep a " + "member of the cohort that counting out would lose, and its count " + "is disclosed. (An earlier request from the coordinating session " + "had counted code 0 out at once.)", + "**Why no member of the cohort is lost.** A person counted out on a " + "side has no birth-year source but a seed whose year lies outside " + "1889\N{EN DASH}2016. A person counted out under a refused code has " + "no presence the law can date inside 1889\N{EN DASH}2016. That " + "range contains 1946\N{EN DASH}55, so no counted-out person can " + "belong to the cohort in any row, and a member of the cohort with " + "one refused code is re-seeded and keeps its observations.", + "**Precedent.** Track M counts out, before the same law, the " + "in-family, positive-weight persons aged 0\N{EN DASH}5 in its one " + "seed wave, 2023, whose seed (income year 2022 minus age) implies a " + "year after 2016 " + "(`src/populace_dynamics/min_benefit_track_m/structure.py`, the " + "funnel count `age_0_5_counted_out_before_birth_law`, and " + "`min_benefit_track_m/cohort.py`). U2 seeds from six waves, so its " + "rule tests the implied year on both sides, not an age, and can " + "re-seed a person past a refused code from a later presence; it " + "keeps code 1 on the law's unresolved path; and it applies only to " + "persons the seed would date, so the law's precedence is kept.", + "**Disclosure.** Each reason's count is disclosed in every row's " + "structural counts, as `n_seed_birth_year_counted_out` with the " + "keys `before_1889`, `after_2016`, `seed_age_0` and " + "`no_valid_seed_age`, and the re-seeded persons' count as " + "`n_seed_birth_year_reseeded`. Each decides nothing. The rule is " + "fixed before any count, and none was computed for this record.", + "**What the ruling leaves unchanged.**", + "- `src/populace_dynamics/estimates/career.py` is unchanged.", + "- No row, plan cell, observation weight, disposition rule, design " + "frame, role rule or income concept changes. U2 stays unregistered " + "and unauthorized (\N{SECTION SIGN}20).", + "- The ruling authorizes one more step-4 pass, structure and then " + "component diagnostics, under d1649's scope with `--authorized-by " + "d1724`, once this change merges and master's Tests pass at the run " + "head (\N{SECTION SIGN}20 step 4).", + "- No count, statistic or outcome was computed to record this " + "ruling, no PSID record was read, and no file on the restricted " + "list was opened for reading. Reading and exposure are recorded in " + "`docs/design/u2_m2_exposure.md` (d1724 section).", +] + + +#: Where section 17's review paragraph and section 20 step 4 point to the +#: d1724 record: each sentence stands in its place exactly once. +D1724_POINTERS = { + "outstanding": "After the first structural pass stopped, Max ruled on " + "the birth law's support on 2026-10-19, and \N{SECTION SIGN}16d records " + "that ruling.", + "execution": "**First pass (d1649): stopped.** It ran once, on " + "2026-10-19 at `640adccf`, and stopped in the birth law without writing " + "any output; the component diagnostics were not run " + "(\N{SECTION SIGN}16d). **Max's ruling of 2026-10-19 (d1724; " + "\N{SECTION SIGN}16d)** authorizes one more pass, structure and then " + "component diagnostics, under d1649's scope with " + "`--authorized-by d1724`, once the \N{SECTION SIGN}16d change merges " + "and master's Tests pass at the run head.", +} + + +def check_d1724_record(spec): + """Section 16d records d1724: it stands once, between sections 16c and + 17, outside code, and holds exactly :data:`D1724_RECORD`, in order; + sections 17 and 20 point to it; and the d1090 checks (which refuse + hiding by comment, raw HTML or fence anywhere) still hold.""" + check_d1090_record(spec) + lines = spec.split("\n") + start = _heading_line(lines, "16d") + end = _heading_line(lines, "17") + assert _heading_line(lines, "16c") < start < end + assert not any(_fenced(lines)[start:end]) + assert [x for x in lines[start:end] if x.strip()] == D1724_RECORD + places = d1090_locations(spec) + for place, sentence in D1724_POINTERS.items(): + assert places[place].count(sentence) == 1, place + + +def test_section_16d_records_the_d1724_ruling(): + """§16d is the §3 note d1724 names: every line pinned, and §17 and §20 + step 4 point to it. The rule it states is the one the cohort code + applies (tests/track_u2/test_u2_birth_bounds.py).""" + spec = (ROOT / SPEC).read_text(encoding="utf-8") + check_d1724_record(spec) + section = spec[spec.index("## 16d. ") : spec.index("## 17. ")] + for clause in ( + "before 1889 or after 2016", + "`n_seed_birth_year_counted_out`", + "`before_1889`, `after_2016`, `seed_age_0` and `no_valid_seed_age`", + "`n_seed_birth_year_reseeded`", + "Each decides nothing.", + "(amends draft-3 line 114)", + ): + assert section.count(clause) == 1, clause + + +#: Mutations of the d1724 record, each of which must fail +#: ``check_d1724_record``: a changed rule, side, code range, precedence, +#: disclosure, scope or authorization, a moved heading, an added line, and +#: hiding by comment. +D1724_MUTATIONS = [ + ( + "16d", + "before 1889 or after 2016, the person", + "before 1889, the person", + ), + ( + "16d", + "If the seed's age is 2\N{EN DASH}125", + "If the seed's age is 1\N{EN DASH}125", + ), + ( + "16d", + "(0, or any code but 1\N{EN DASH}125 and 999)", + "(any code above 125)", + ), + ( + "16d", + "under `seed_age_0` when the first code is 0", + "under `seed_age_0` when any code is 0", + ), + ( + "16d", + "it is therefore handled as any refused code, not counted out at once", + "it is counted out at once", + ), + ( + "16d", + "Max approved that widening on 2026-10-20", + "The coordinating session chose that widening on 2026-10-20", + ), + ( + "16d", + "including that code 0 (Inap.) is re-seeded", + "except that code 0 (Inap.) is counted out", + ), + ("16d", "keeps 999 on that path", "re-seeds 999 too"), + ( + "16d", + "it can keep a member of the cohort", + "it keeps members of the cohort", + ), + ( + "16d", + "and on 2026-10-20 approved the widening this section records", + "and later asked about a widening", + ), + ( + "16d", + " and the re-seeded persons' count as `n_seed_birth_year_reseeded`", + "", + ), + ( + "16d", + "Each decides nothing.", + "Each decides whether the pass proceeds.", + ), + ( + "16d", + "a marriage-history or earnings year still wins", + "the seed's year wins", + ), + ( + "16d", + "Seed codes 1 and 999 stay unresolved", + "Seed codes 1 and 999 are counted out", + ), + ("16d", "does not seed it either", "seeds it instead"), + ( + "16d", + "so no counted-out person can belong", + "so a counted-out person can", + ), + ("16d", "(amends draft-3 line 114)", "(amends draft-3 line 126)"), + ( + "16d", + "estimates/career.py` is unchanged.", + "estimates/career.py` changes.", + ), + ( + "16d", + "U2 stays unregistered and unauthorized (\N{SECTION SIGN}20).", + ".", + ), + ( + "16d", + "authorizes one more step-4 pass", + "authorizes two more step-4 passes", + ), + ( + "execution", + "authorizes one more pass, structure", + "authorizes passes, structure", + ), + ( + "outstanding", + "and \N{SECTION SIGN}16d records that ruling.", + "and that is all.", + ), + ("16d", "## 16d. Max's ruling", "### 16d. Max's ruling"), + ( + "16d", + "**What the ruling leaves unchanged.**", + "**What the ruling leaves unchanged.**\n\n- Exploratory counts may " + "be computed.", + ), + ( + "16d", + "**Disclosure.** Each reason's count", + "