Repository navigation
S0.1: source, exposure, access and power audit for SM2–SM4 and ED1–ED2 - #588
Merged
Merged
Conversation
docs/design/s0_1_source_audit.md is the frozen record that S0.1 owes the lane's design (#558, approved under d1922). It covers the gates SM3, SM2, SM4, ED1 and ED2, in that order. For each gate it records: - the sources, with their manifest hashes; - the population and estimands; - the exposure history; - the holdout partition and prediction origin; - the critical cells and tolerances; - the uncertainty treatment; - the pass conjunction and power procedure; - the suppression threshold; - the named conditions. Each item is marked frozen, proposed or open. The record ends with the input for Q7, what SM3's real-data step would need, and makes no recommendation on authorization. The group gates (GR1 to GR4) are out of scope; their audit is briefed separately. Provenance: - built by Subfleet job 20261024-110212-mgf-s0-1-build; - revised in fix rounds 20261024-211800-mgf-s01-fix-r1 and 20261025-015100-mgf-s01-fix-r2b; - values-scanned by 20261025-052455-mgf-s01-values-scan-r4, which returned CLEAR; - reviewed by 20261025-105651-mgf-s01-review-r3, which returned APPROVE with five low findings. A follow-up round fixes them. The build lanes' sandboxes could not write .git, so the lane session commits the tree they left. It matches the reviewed SHA-256 hashes. PSID use follows the project's data-access record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/s0_1_power.py simulates SM3's and ED2's registered tests on invented populations. It reports power at the 0.90 criterion over a declared grid of assumed counts. docs/design/s0_1_power_results.json holds its committed results. tests/s0_1/ holds 99 example and Hypothesis property tests on invented data. They cover type-I error under the null, power that is monotone in sample and effect size, seed determinism, Bonferroni coverage and refusals. Checks on the host, with an empty PSID root: - tests/s0_1: 99 passed; - Black (79 columns) and Ruff: clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/s0_1 adds 99 unit tests. On the base 2195258, unit goes from 18,088 to 18,187 and the total from 24,619 to 24,718. Full collection with -k tiers_match_policy_manifest passes on the host: 1 passed, 2 skipped. The counts are recounted on fresh master at the coordinator's tier-count slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Two fix rounds change scripts/s0_1_power.py, its tests and the audit note. The results file is unchanged: every non-anchor field reproduces exactly. Round 3 (Subfleet job 20261025-114320-mgf-s01-fix-r3) fixed review r3's five low findings: - the threshold helpers refuse a threshold the floating-point model cannot represent, verify each returned count through family_power_lower_bound, and find the minimal count by an integer search; - the SM3 boundary test reports pass counts and treats inconclusive runs explicitly, instead of a vacuous assertion; - the note gains a code-input metadata table and two locators for the DI composition in mortality_ses/model.py; - property tests check that an earlier FAIL survives any later inestimable cell, and that inestimability never yields a PASS. Round 4 (job 20261026-162925-mgf-s01-fix-r4) fixed review r4's two medium findings: - a reachable finite target no longer returns None. Reachability is decided by the same predicate the search uses, at the largest supported count. A target no count can reach raises UnreachableThreshold; one the model cannot represent raises UnrepresentableThreshold. Both, with ThresholdInputError, are ThresholdRefusal subclasses of ValueError; - every argument is validated on entry, one observation is tried first, a critical value must be finite and positive, very large family sizes are refused by name, and a closed-form union bound replaces the per-cell list for equal cells; - the note and docstrings state that sufficiency is defined by the helper's own floating-point predicate. Checks: - values scans 20261026-100223-mgf-s01-values-scan-r5 and 20261026-184550-mgf-s01-values-scan-r6: CLEAR; - review 20261026-190834-mgf-s01-review-r5: APPROVE, with two low documentation findings that a follow-up round fixes; - tests/s0_1 on the host, with an empty PSID root: 160 passed. The fix jobs left their changes uncommitted, so the lane session commits the tree. It matches the reviewed SHA-256 hashes. PSID use follows the project's data-access record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/s0_1 now holds 160 unit tests, up from 99. On the base 2195258, unit goes from 18,187 to 18,248 and the total from 24,718 to 24,779. The counts are recounted on fresh master at the coordinator's tier-count slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r5 approved the round-4 tree with two low findings in the audit note's wording. This round changes only docs/design/s0_1_source_audit.md: - It defines the supported counts as the code does: the integers from one through MAX_SUPPORTED_COUNT. - It replaces the round-4 record's unqualified preservation claim. The recorded planning and validation counts are unchanged, but round 4 refuses some inputs round 3 returned a result for. The note lists those kinds, with the exception each raises, and records that a non-double criterion can change a count. - It adds the round-5 record. The script, results, tests and tier files are unchanged. Made by Subfleet job 20261026-203129-mgf-s01-fix-r5b. Values scan 20261026-213217-mgf-s01-values-scan-r8: CLEAR. Review 20261026-215157-mgf-s01-review-r6: APPROVE, with one low finding on wording that predates this round. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review r6 approved the round-5b tree with one low finding. The audit note said each reported threshold passes and one fewer fails, but a reported threshold can be one, and zero is not a supported count. This round changes only docs/design/s0_1_source_audit.md: - Both statements of that property now except a threshold of one. - It adds the round-6 record. The script, results, tests and tier files are unchanged. Made by Subfleet job 20261026-220958-mgf-s01-fix-r6. Values scan 20261026-222719-mgf-s01-values-scan-r9: CLEAR. Review 20261026-223940-mgf-s01-review-r7: APPROVE, no findings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts: tests/README-tiers.md and tests/tier_counts.json only. Both take master's blobs here; the next commit recounts them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
unit 19,504 -> 19,664, total 26,074 -> 26,234. The 160 new unit tests are tests/s0_1, this PR's own modules; the other tiers keep master's counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
marked this pull request as ready for review
November 1, 2026 18:16
Contributor
Author
|
The Microcosm Dynamics coordinator merged this PR at
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
S0.1 of the mortality, groups and financing lane: the source, exposure, access and power audit for the gates SM2, SM3, SM4, ED1 and ED2. It is validation-only. It freezes what each gate's later registered test will use, and computes power on invented data. It estimates nothing from real data.
The coordinator lands this PR at its tier-count slot.
Sources:
docs/design/mortality_groups_financing.md, approved under d1922 (Design: mortality, groups and financing lane (priced milestones, approved under d1922) #558, at2c19f5b6). S0.1 is in its now-set, after SM1 (SM1: differential mortality by lifetime earnings, education and race (normalized within age-sex-year cells) #570) on lane M.s01-brief-20261024.md, in the lane's evidence folder.What it adds
docs/design/s0_1_source_audit.md: the frozen record, one section per gate, in the order SM3, SM2, SM4, ED1, ED2. For each gate it records:It marks each item frozen, proposed or open. It also has a code-input metadata table (each repository input's path, commit, SHA-256, verification date and reliance). It ends with the input for Q7, what SM3's real-data step would need, and makes no recommendation on authorization.
scripts/s0_1_power.pyand its committed results,docs/design/s0_1_power_results.json. The script simulates SM3's and ED2's registered tests on invented populations, over a declared grid of assumed counts, and reports power at the 0.90 criterion. Its threshold helpers return the minimal sufficient count or raise a named refusal.tests/s0_1/: 160 example and Hypothesis property tests on invented data.Housekeeping: the tier counts.
Invariants
Tested as properties (Hypothesis) and examples, on invented data:
MAX_SUPPORTED_COUNT. Every other input ends in a named refusal (ThresholdInputError,UnreachableThresholdorUnrepresentableThreshold, allThresholdRefusalsubclasses ofValueError), neverNoneor another exception;Review r3 found two places where earlier code broke an invariant it stated. Round 3 fixed both, and round 4 fixed the two numerical gaps review r4 then found in the threshold helpers. Round 4 deliberately refuses some inputs round 3 returned a result for; the note lists those kinds. The recorded planning and validation counts are unchanged.
Checks
tests/s0_1: 160 passed, under an audit hook that counted opens of the files the project bars from these runs. It counted none, and nopopulace_dynamicsmodule was loaded. The full-collection tier check runs at the coordinator's slot.20261026-190834-mgf-s01-review-r5(GPT-6.1 Sol): APPROVE, with no high or medium finding.20261026-215157-mgf-s01-review-r6(GPT-6.1 Sol), on round 5: APPROVE, with one low wording finding, fixed in round 6.20261026-223940-mgf-s01-review-r7(GPT-6.1 Sol), on round 6: APPROVE, with no findings.tests/s0_1(all passed), and found no earlier fix narrowed or regressed.Tiers (on master
6413d842):The other tiers keep master's counts. The 160 new unit tests are
tests/s0_1: a named collection on the merged tree gives 160 unit tests and none in any other tier, and CI's full collection checks the total. The master merge took master's two tier files, and one later commit recounts them. This PR's seven other files keep the blobs review r7 covered at781ba61f.Provenance
20261024-110212-mgf-s0-1-build.20261024-211800-mgf-s01-fix-r1,20261025-015100-mgf-s01-fix-r2b,20261025-114320-mgf-s01-fix-r3,20261026-162925-mgf-s01-fix-r4,20261026-203129-mgf-s01-fix-r5band20261026-220958-mgf-s01-fix-r6. The set-aside attempt was20261026-194239-mgf-s01-fix-r5.20261024-171232-mgf-s01-values-scan, then delta scans r2 to r9 (the last20261026-222719-mgf-s01-values-scan-r9).Review
20261025-105651-mgf-s01-review-r3) returned APPROVE with five low findings, fixed in round 3.20261026-131614-mgf-s01-review-r4) returned REVISE with two medium findings in the threshold helpers' numerical handling: a reachable finite target could returnNone, and some numerical endpoints raised raw exceptions. Round 4 fixed both.20261026-190834-mgf-s01-review-r5) returned APPROVE with two low documentation findings: the support definition omitted the cap, and the preservation claim was unqualified. Round 5 fixed both.20261026-215157-mgf-s01-review-r6) returned APPROVE with one low finding: the note's statement that each reported threshold passes and one fewer fails needed the lower-end exception, since the smallest supported count is one and zero is unsupported. Round 6 fixed it.20261026-223940-mgf-s01-review-r7) returned APPROVE with no findings.axiom: n/a: source audit and invented-data power calculations
🤖 Generated with Claude Code