Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions data/external/track_u2/u1_identity.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@
},
"u2_specification": {
"path": "docs/design/boomers2004_1946_55_comparison.md",
"note": "Moved out of the unchanged-byte list by u2-m1b: Part C writes the proposed revision u2-draft-4 into the specification. The draft-3 hash is the milestone-1 pin; the current hash pins u2-draft-4 with Max's 2026-09-30 rulings recorded in §16b (d637) and his 2026-10-10 ruling on blockers B1 and B2 recorded in §16c (d1090); every line a registry cites keeps its draft-3 number and text.",
"note": "Moved out of the unchanged-byte list by u2-m1b: Part C writes the proposed revision u2-draft-4 into the specification. The draft-3 hash is the milestone-1 pin; the current hash pins u2-draft-4 with Max's 2026-09-30 rulings recorded in §16b (d637) and his 2026-10-10 ruling on blockers B1 and B2 recorded in §16c (d1090), with his 2026-10-19 ruling on the birth law's support recorded in §16d (d1724); every line a registry cites keeps its draft-3 number and text.",
"draft_3_sha256": "7badf89e896ece28abcea1cb2db9bd55d468346b405ed06eb357aaf5c7b51df2",
"draft_3_commit": "f7412e00cdcdba6ee1156b74fa93039f5329cb8f",
"draft_3_git_blob": "e07ee0d589d20f3b6d100e5d4ce173f9a653f5e9",
"version": "u2-draft-4",
"sha256": "c0c2685f8e23756895b66d09a6bbd618f930c488cc5ad5c52415f42b228d8f98",
"sha256": "4ad771e296791ce82f5b5cb0cc30c64460f209467fd60504a6cae3bd7581bcb0",
"cited_lines_sha256": {
"74": "7b299c9e4473fd51d04bdfec51af83268e517181ab734d3e76ccd1c744f6daa9",
"75": "44454cc04e6a50863035d69ccb53edec175b4ad70a19d8877437901d2deb8428",
Expand Down
48 changes: 46 additions & 2 deletions docs/design/boomers2004_1946_55_comparison.md

Large diffs are not rendered by default.

20 changes: 20 additions & 0 deletions docs/design/u2_m2_exposure.md
Original file line number Diff line number Diff line change
Expand Up @@ -353,3 +353,23 @@ The hook refused nothing at any of these runs. Its only refusals were the self-t
**The d1242 change.** After #544 merged (`2ef10532`), the same lane applied Max's d1242 ruling: B2 joins `sources.PERSON_COUNTED_EXCLUSIONS`, with its tests and a §19 entry. It read only the decision ledger's d1242, d1597 and d1649 entries, the code it changed and the roles tests. It ran the U2 files under the same hook, which refused nothing. Answering #546's review, it added an invented variant (a second, sex-unknown code-92 member in one excluded 2019 unit) and a distinct-person form of each count. For this it read the review, the invented-data generator, the runner and the scripts test. It ran the reviewer's mutations and the U2 files under the same hook, which refused nothing.

**Values:** this lane and its lanes stated and inferred no 1946–55 value or direction. Every number they printed is invented (offsets, invented cells, invented-run counts and dispositions), a count, a hash, a timing or a public parameter from the committed captures.

## d1724 lane (2026-10-19)

A Claude Code session (Claude Opus 5.5), started by the dynamics coordinator session (`local_6085d52a-…`), applied Max's ruling d1724 (specification §16d) in a standalone clone of `origin/master` at `640adccf`. It read no PSID record and no comparator, and it does not forecast U2.

**Read:**

- `EV/RESTRICTED-FILES.md` itself, in full, as its brief required, to know what not to open. It opened nothing the file lists.
- The coordinator's shared notes folder: the step-4 pass session's value-free note and the d637 registry session's note.
- The chief-of-staff decision ledger's d1649 and d1724 entries (decision, ruling and status lines), and, on 2026-10-20, d1724's two notes (Max's approval of the widening, and the coordinating session's correction on code 999).
- In `EV/track-u2-step4-pass-20261019/`: the directory listing (file names only) and the SHA-256 of `MANIFEST.md`, computed without printing it. It did not open `track-u2-structure.stderr.txt` (restricted), its masked copy, or any other file there.
- The repository: the U2 package, `estimates/career.py` (`derive_birth_years` and its constants, unchanged), Track M's `structure.py` and `cohort.py` (the count-out precedent), the U2 scripts and tests, `tests/conftest.py`, the specification (its header and §§3, 16c, 17, 19 and 20, and the tests' pins) and this file.

**Ran:** the U2 test files, the new `tests/track_u2/test_u2_birth_bounds.py` among them, with `POPULACE_DYNAMICS_PSID_DIR` set to an empty directory, which stayed empty; so the checks that need PSID documentation skipped. It ran `career.derive_birth_years` and master's `cohort.derive_u2_births` (read from Git at `640adccf`) on invented inputs only, to show the fix equals master wherever master's law completes. It ran Black, Ruff, the tier recount by collection and the mutation checks listed in the PR.

**Values:** it stated and inferred no 1946–55 value or direction. Every number it printed is invented (generator persons, ages and counts), a test count, a hash, or a constant of the code (`career.DERIVED_BIRTH_MIN` and `DERIVED_BIRTH_MAX`, 1889 and 2016).

**2026-10-20, the widened rule.** The coordinating session asked that the rule also cover seeds whose age code the law refuses, and later reported that Max approved that widening, re-seeding of code 0 included. The lane read the coordinator's messages, d1724's two notes in the decision ledger (to cite the approval and its correction), the loader's documented age domain and its value check (`loader.py`), and the code and tests it changed. It read nothing else outside the repository, and its runs were as above: an empty PSID directory, invented inputs only, and the mutation check, all listed in the PR.

**Review lane.** Subfleet `20261020-030251-d1724-review-552b` reviewed head `8a6a8192` and returned APPROVE. No Codex lane could admit the job, so Subfleet served it on Claude Opus 5.5 (lane `claude-7`), in its own standalone clone with its own virtual environment. The lane reports that its empty PSID directory stayed empty and that it opened nothing under the evidence directory, `REFS` or `PSID`. It loaded master's `cohort.py` from Git and ran invented inputs only. Its scratch scripts were in `/tmp/review552`. Its findings (two test gaps, one record-accuracy point, nits) were addressed in the next commit, which declined one (the header's version line, pinned as d1090's `header_version` clause). The delta review (Subfleet `20261020-045107-d1724-review-552-delta`, GPT-6.1 Sol) approved that commit. Records: `EV/rulings-records-20261019-d1724/review-552-out.md`.
8 changes: 8 additions & 0 deletions scripts/track_u2_structure.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@
committed-registry role context and writes ``track-u2-structure.json``:
dispositions, observations, roles, pairings, design strata, the family
income and WEALTH1 identity counts per wave and the registry status.
Each row's counts include ``n_seed_birth_year_counted_out``: the persons
counted out before the birth law, by reason -- a seed year before 1889
(``before_1889``) or after 2016 (``after_2016``), or no presence with an
age the law can date (``seed_age_0``, ``no_valid_seed_age``) -- and
``n_seed_birth_year_reseeded``, the persons re-seeded past an age code
the law refuses (d1724, specification section 16d;
:func:`populace_dynamics.uniform_cut_track_u2.cohort.derive_u2_births`).
Each is disclosed and decides nothing.

It also writes ``track-u2-preregistration-evidence.json``: the frozen
input identities section 14 says execution rechecks -- the SHA-256 of
Expand Down
174 changes: 164 additions & 10 deletions src/populace_dynamics/uniform_cut_track_u2/cohort.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,16 @@
U1's per-row ``earliest_presence_wave``, which :class:`U2CohortSpec`
refuses). Births are derived once (:func:`derive_u2_births`) and reused
by every row. Support waves identify people; they add no observation
and no design unit to a row.
and no design unit to a row. Max's ruling of 2026-10-19 (d1724; section
16d) keeps every input the law would refuse away from it, for the
persons whose birth year only the seed coordinate could give: a seed
whose age code the law refuses (0, or any code but 1-125 and 999) is
replaced by the person's earliest presence with an age of 2-125 whose
implied year the law supports; a person with no such presence, or whose
seed implies a year outside the law's derived-age support
(``career.DERIVED_BIRTH_MIN``-``DERIVED_BIRTH_MAX``, 1889-2016), is
counted out before the law runs, as Track M counts out young children.
Each reason's count is disclosed and decides nothing.

**Roles** (section 3 relationship-code amendment) come from a
:class:`~populace_dynamics.uniform_cut_track_u2.sources.RoleContext`.
Expand Down Expand Up @@ -126,6 +135,7 @@
"PSID_FILES",
"SNAPSHOT_KEY",
"ROWS",
"SEED_BIRTH_COUNTED_OUT",
"SEED_WAVE_RULE",
"SUPPORT_WAVES",
"TARGET_AGE",
Expand Down Expand Up @@ -162,6 +172,21 @@
#: changes on U0's population (:mod:`.rows`).
ROWS: tuple[str, ...] = ("U0", "U1")
SEED_WAVE_RULE = "earliest_presence_in_common_support_waves"
#: d1724 (section 16d): the reasons :func:`derive_u2_births` counts a
#: person out before the birth law, each disclosed. The law
#: (``career.derive_birth_years``, read-only under section 14) asserts
#: that a year it derives from a seed (age code 2-125) lies in
#: ``[DERIVED_BIRTH_MIN, DERIVED_BIRTH_MAX]``, and refuses every seed age
#: code but 1-125 and 999. A person the seed alone would date is counted
#: out when the seed implies a year before or after that support, or
#: when no presence carries an age the law can date (the first code 0,
#: or another refused code).
SEED_BIRTH_COUNTED_OUT: tuple[str, ...] = (
f"before_{career.DERIVED_BIRTH_MIN}",
f"after_{career.DERIVED_BIRTH_MAX}",
"seed_age_0",
"no_valid_seed_age",
)
U1_EVEN_BIRTH_YEAR_WEIGHT = 0.5
U1_SINGLE_OBSERVATION_WEIGHT = 1.0
U1_EVEN_BIRTH_AGES: tuple[int, int] = (66, 68)
Expand Down Expand Up @@ -198,6 +223,12 @@
("died", 81, 89),
)
_MAX_AGE_CODE = 125
#: The seed age codes career's derived-age clause turns into a birth year
#: (``seed year - age``); 1, 999 and missing are unresolved there, and any
#: other code is refused there.
_DERIVED_SEED_AGES = (2, 125)
#: The documented DK/NA/refused age code, which the law leaves unresolved.
_NA_AGE_CODE = 999
_FAMILY_UNIT_SCALE = 100_000
_UNRESOLVED_STATES = ("unknown", "no_marriage_history")
_OMITTED_ROWS = (
Expand Down Expand Up @@ -955,12 +986,25 @@ def _check_role_context(
# ===========================================================================
@dataclass(frozen=True)
class U2Births:
"""Birth-year records for the universe and the annuitant extension."""
"""Birth-year records for the universe and the annuitant extension.

``counted_out`` maps each reason of :data:`SEED_BIRTH_COUNTED_OUT` to
the persons counted out of the universe before the law ran (d1724);
they have no record, so their birth year is ``None``. ``reseeded``
holds the persons whose earliest presence carried an age code the law
refuses and whose seed is their earliest presence it can date.
"""

records: Mapping[int, career.BirthYearRecord]
universe: frozenset[int]
seed: pd.DataFrame
inputs_sha256: str
counted_out: Mapping[str, frozenset[int]] = field(
default_factory=lambda: MappingProxyType(
{reason: frozenset() for reason in SEED_BIRTH_COUNTED_OUT}
)
)
reseeded: frozenset[int] = frozenset()

def birth_year(self, person_id: int) -> int | None:
record = self.records.get(int(person_id))
Expand Down Expand Up @@ -1048,16 +1092,110 @@ def check_person_joins(inputs: U2Inputs) -> None:
)


def _seed_for_the_law(
presences: list[pd.DataFrame],
history: pd.DataFrame,
earnings: pd.DataFrame,
) -> tuple[pd.DataFrame, dict[str, frozenset[int]], frozenset[int]]:
"""The seed rows the law can take, the persons counted out by reason,
and the persons re-seeded (d1724, section 16d).

``presences`` holds every positive-weight, in-family presence of every
support wave. The law reaches a seed only for a person whose birth
year neither the marriage history nor the observed earnings give, so
the law itself, run on the same frames without seed coordinates,
names those persons (its unresolved records); for everyone else the
earliest presence stays the seed and the law's precedence decides.
For each person the seed alone would date:

* if the earliest presence's age code is one the law refuses (0, or
any code but 1-125 and 999), the seed is the person's earliest
presence with an age of 2-125 whose implied year the law supports
(re-seeded); with none, the person is counted out under
``seed_age_0`` (earliest code 0) or ``no_valid_seed_age``;
* if the seed's age is 2-125 and its year (``seed year - age``) lies
before 1889 or after 2016, the person is counted out on that side.

Codes 1 and 999 keep the law's unresolved path.
"""

first = _seed_frame(presences)
people = set(int(pid) for pid in first["person_id"])
clauses = career.derive_birth_years(
history[history["person_id"].isin(people)],
earnings[earnings["person_id"].isin(people)],
required_person_ids=people,
)
seed_only = {
record.person_id
for record in clauses
if record.source is career.BirthSource.UNRESOLVED
}
ages = first["age"].astype("int64")
accepted = ages.eq(_NA_AGE_CODE) | ages.between(1, _MAX_AGE_CODE)
refused = first["person_id"].isin(seed_only) & ~accepted
seed = first
reseeded: frozenset[int] = frozenset()
if refused.any():
every = pd.concat(presences, ignore_index=True)
every_ages = every["age"].astype("int64")
every_years = every["year"].astype("int64") - every_ages
valid = every[
every_ages.between(*_DERIVED_SEED_AGES)
& every_years.between(
career.DERIVED_BIRTH_MIN, career.DERIVED_BIRTH_MAX
)
& every["person_id"].isin(first.loc[refused, "person_id"])
]
replacement = _seed_frame([valid])
reseeded = frozenset(int(pid) for pid in replacement["person_id"])
seed = (
pd.concat([first[~refused], replacement], ignore_index=True)
.astype(first.dtypes.to_dict())
.sort_values("person_id", kind="stable")
.reset_index(drop=True)
)
unseeded = refused & ~first["person_id"].isin(reseeded)
seed_ages = seed["age"].astype("int64")
implied = seed["year"].astype("int64") - seed_ages
derived = seed["person_id"].isin(seed_only) & seed_ages.between(
*_DERIVED_SEED_AGES
)
before = derived & (implied < career.DERIVED_BIRTH_MIN)
after = derived & (implied > career.DERIVED_BIRTH_MAX)
rows = (
seed.loc[before, "person_id"],
seed.loc[after, "person_id"],
first.loc[unseeded & ages.eq(0), "person_id"],
first.loc[unseeded & ~ages.eq(0), "person_id"],
)
counted_out = {
reason: frozenset(int(pid) for pid in persons)
for reason, persons in zip(SEED_BIRTH_COUNTED_OUT, rows, strict=True)
}
kept = seed[~(before | after)].reset_index(drop=True)
return kept, counted_out, reseeded


def derive_u2_births(inputs: U2Inputs) -> U2Births:
"""The birth-year law over the common support waves (section 3).

Universe: every person with a positive-weight, in-family presence in
any support wave; seed coordinate: the earliest such presence
(``(wave - 1) - age``). Annuitant extension: in-family persons of
the administrative birth-support codes (10, 20, 22, 88, 90; not 92)
and marriage-history spouses of target-cohort members outside the
universe, seeded by their earliest in-family presence at any weight;
the law runs without a required population for them, so a conflict
(``(wave - 1) - age``). Before the law runs, for a person whose
birth year only the seed could give, a seed age code the law refuses
is replaced by the earliest presence it can date (``reseeded``), and
a person with no such presence, or whose seed implies a year outside
the law's derived-age support (1889-2016), is counted out of the
universe under its reason (d1724, section 16d;
:func:`_seed_for_the_law`); ``counted_out`` holds them, and no other
person's record changes. Annuitant extension: in-family
persons of the administrative birth-support codes (10, 20, 22, 88,
90; not 92) and marriage-history spouses of target-cohort members
outside the universe, seeded by their earliest in-family presence at
any weight whose implied year the law supports (code 999 kept); a
counted-out person is not outside the universe, so never joins it.
The law runs without a required population for them, so a conflict
leaves the person unresolved (the wave-age fallback, counted).
"""

Expand All @@ -1080,10 +1218,13 @@ def derive_u2_births(inputs: U2Inputs) -> U2Births:
}
)
)
seed = _seed_frame(seeds)
universe = frozenset(int(pid) for pid in seed["person_id"])
history = inputs.marriage_history
earnings = inputs.observed_earnings
present_ids = frozenset(
int(pid) for frame in seeds for pid in frame["person_id"]
)
seed, counted_out, reseeded = _seed_for_the_law(seeds, history, earnings)
universe = frozenset(int(pid) for pid in seed["person_id"])
records = career.derive_birth_years(
history[history["person_id"].isin(universe)],
earnings[earnings["person_id"].isin(universe)],
Expand Down Expand Up @@ -1111,7 +1252,7 @@ def derive_u2_births(inputs: U2Inputs) -> U2Births:
)
| anchor["person_id"].isin(spouses)
)
& ~anchor["person_id"].isin(universe)
& ~anchor["person_id"].isin(present_ids)
]
ages = rows["age"].astype("int64")
seed_birth = (wave - 1) - ages
Expand Down Expand Up @@ -1150,6 +1291,8 @@ def derive_u2_births(inputs: U2Inputs) -> U2Births:
universe=universe,
seed=seed,
inputs_sha256=input_frames_sha256(inputs),
counted_out=MappingProxyType(counted_out),
reseeded=reseeded,
)


Expand Down Expand Up @@ -1535,6 +1678,17 @@ def build_u2_cohort(
"n_unresolved_birth_year": sum(
births.birth_year(pid) is None for pid in outside
),
# d1724 (section 16d): the persons counted out of the universe
# before the birth law, whom only the seed could date, by reason
# (a seed year before 1889 or after 2016; no presence with an age
# the law can date, the first code 0 or another), and the persons
# re-seeded past an age code the law refuses. Disclosed; each
# decides nothing.
"n_seed_birth_year_counted_out": {
reason: len(births.counted_out[reason])
for reason in SEED_BIRTH_COUNTED_OUT
},
"n_seed_birth_year_reseeded": len(births.reseeded),
"plan_birth_years": plan_years,
"role_context": role_context.kind,
"births_inputs_sha256": births.inputs_sha256,
Expand Down
Loading
Loading