Skip to content

Paper: report B2's failure and gate_epuf_fill's scores - #551

Merged
MaxGhenis merged 3 commits into
masterfrom
paper-b2-epuf-fill-20261018
Oct 20, 2026
Merged

MaxGhenis merged 3 commits into
masterfrom
paper-b2-epuf-fill-20261018

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Brings paper/paper.qmd in line with three results it still described as planned or absent. Facts are the same as #550's ledger update, each checked against code or committed artifacts.

B2 failed (#42 comment 6252752628; runs/track_b_b2_score.json, outcome: FAIL).

  • The result goes in the summary (one sentence), the forward-earnings test sentence, the coverage table and the "Forward earnings in the projection" paragraph, which moves from future to past tense.
    • The deciding cell is the prime-age (25–44) zero-earnings rate: about 35% above the PSID's on a log-ratio tolerance of 0.202, about 22%. It misses on all five refit seeds, and only prime-age p90 passes every check.
  • The consequences, from the design and the attempt ledger:
    • no rerun, new seeds or tolerance relief;
    • a successor whose changes must rest on evidence other than B2's result, registered before it runs;
    • two failed successors pause the mechanism.

gate_epuf_fill scored (#537; runs/epuf_fill_gate_test_v1.json). The Earnings Public-Use File paragraph no longer says we haven't scored anything against the file. It now describes the test from harness/epuf_fill_gate.py and states its results:

  • How the test works: it holds back 20% of the file's people (878,481). It masks the odd years 1997–2005 and the years before max(1968, birth year + 22), fills them, and compares the fills with the true records.
  • What it found: the blind tests' fill rules fail it, 100/183 and 131/136. The learned pre-career fill passes, 0/136, and the learned odd-year fill is adopted uncertified, 8/183. Neither is used by a blind test.
  • Table rows: two table rows note that both current rules fail it.

Payment rules.

tests/test_paper_text.py, test_anchor_provenance_erratum.py and test_track_b_b0_1_audit_record.py pass (38). The microcosm.institute paper bundle needs a rebuild to publish this. That rebuild must also include #536, which is still owed.

🤖 Generated with Claude Code

B2, the registered test that carried the forward-earnings law from 2010
to 2014, failed on 18 October (#42 comment 6252752628). The paper still
described it as planned. It now reports the result in the summary, the
coverage table and the forward-earnings section: about 35% too many
people aged 25 to 44 with no earnings, against a tolerance of about 22%,
missing in all five refit seeds. It also states the consequences: no
rerun or tolerance relief, and a successor law designed on independent
evidence and registered first.

It also replaces "we have not yet scored ... against the file" with what
gate_epuf_fill found on 878,481 held-out EPUF people: the blind tests'
fill rules fail it (100 of 183, 131 of 136), the learned pre-career fill
passes, and the learned odd-year fill is adopted uncertified. The
earnings-test paragraph now describes the built R2 ledger and R3
modules. The oracle paragraph notes the statutory delayed-credit window
and the erratum diagnostic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
social-security-model Ready Ready Preview Oct 19, 2026 4:46pm UTC

Request Review

…90, the family-maximum refusal

- B2's five seeds are household splits of one projection, not refits.
- Its 2012 and 2014 earnings come from the PSID's 2013 and 2015
  interviews, and its ages are ages at those interviews.
- The passing statistic is the 90th percentile of positive earnings.
- The summary calls B2 a further failed test of forward earnings, so it
  doesn't imply nothing else is unresolved (the odd-year fill is
  uncertified).
- The limits section now says gate_epuf_fill has scored the career-fill
  rules against the EPUF.
- The earnings-test ledger refuses an auxiliary deduction that frees
  binding family-maximum room, not deductions before age reduction, and
  takes its grace-year decision from a separate helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 18, 2026
- B2's seeds are household splits, and its 2012 and 2014 earnings come
  from the PSID's 2013 and 2015 interviews (ages at interview).
- The passing statistic is the 90th percentile of positive earnings.
- B2 is described as a further failed test of forward earnings.
- R2 part (b)'s refusal is stated as the code states it: an auxiliary
  deduction that frees binding family-maximum room. Grace years come
  from a separate helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Applies #551 round-2's optional findings: the abstract and the
forward-law paragraph now say the law fails B2, and two sentences that
called income years 'waves' (2016/2018 and 2006-2022) now say earnings
years.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit e0a731a into master Oct 20, 2026
12 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged at reviewed head 38cf362 (merge commit e0a731a) by the dynamics coordinator. Gates: gh pr checks exit 0, MERGEABLE, not a draft, no change requests. Paper text only, so tier counts are unaffected. tests/test_paper_text.py passes.

Independent review ran three rounds, each on a standalone clone with a shell:

  • Round 1 (GPT-6.1 Sol, 20261018-145018-md551-review): requested changes on eight points, each fixed in 8189111:
    • B2's seeds are household splits, not refits;
    • its 2012/2014 earnings come from the 2013/2015 interviews;
    • the passing statistic is the p90 of positive earnings;
    • the earnings-test refusal concerns the family maximum;
    • the limits section's EPUF sentences;
    • the summary's framing of the failure.
  • Round 2 (Opus 5.5, 20261018-150125-md551-review-r2): APPROVE, with three optional points, applied in 38cf362: B2 in the abstract, and income years in place of "waves".
  • Round 3 (Opus 5.5, 20261019-124532-md551-review-r3): APPROVE at 38cf362. The abstract's YAML parses and renders.

Publishing on microcosm.institute needs a paper-bundle rebuild, which must also carry #536.

This branch was successfully deployed

1 active deployment
Preview — 38cf3627 Deployed Oct 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant