Skip to content

Fix review r10's report findings for OP1 - #589

Merged
MaxGhenis merged 3 commits into
masterfrom
openpop-op1-report-r10-20261026
Oct 26, 2026
Merged

MaxGhenis merged 3 commits into
masterfrom
openpop-op1-report-r10-20261026

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Docs-only follow-up to #566. It fixes the one minor finding and four nits that delta check r10 left in docs/design/open_population_evidence/op1_report.md when it approved #566 at 88b5df28. No code, test or tier file changes.

Head 961f520a: 7640fe1c makes the fixes. 5c307826 fixes six phrases that independent review docs-r1 found wrong or loose in 7640fe1c's new text, and 961f520a takes its delta check's two optional nits (see below).

What changes in the report

  • r10's minor finding. The r9 mutation table's r8-harness row gave one run's property-test gains as a fixed result, though they depend on Hypothesis's draws. The row now separates them:
    • ten mutants gain the property test in each of r10's 40 seeded runs: K1, K3 to K7, S9, S12, S13 and X9;
    • five gain it only on some draws (K2, K8, S14, S15 and S16: 29, 22, 14, 25 and 20 of 40);
    • the author's run is stated as one run.
  • Nit: draw-dependent cells. The r5 and r6 tables give "7 or 8" and "54 or 55" for the length bound removed (A2) and r3's A12 (A4), where r10's run gave 8 and 55. The r9 table's r6 row says eight counts have varied, each a range in the r6 table, and names the four in the row of five.
  • Nit: the git diff --quiet check. §3 now says the command prints nothing either way, so its exit status is the check. Before, the sentence could be read as exiting 0 when a file differs.
  • Nit: wording. §3 says the tier-slot recount, not the merge, names d8302233, and says test modules rather than test files.
  • Nit: timing. Both of §3's timing statements give 8 to 20 s over the author's and reviews r8 to r10's runs, varying with host load. r10's OP1 command took 20.06 s at a one-minute load of 98.
  • New review r10 section. It gives the verdict and the findings table. It also gives the 11 new streams.py mutants r10 tried: five were killed, and the six survivors fall in r9's known classes, which the exact comparison at streams.py:108 rules out.
  • §5. It states that OP1's exit and harness tests import baselines/asfr.py through the baselines package, as other test modules on master do, so runs load it without displaying it.

Checks

  • git diff --check is clean, and the only file changed is op1_report.md.
  • I checked each new figure against r10's review. Cited lines test_exits.py:11, test_harness.py:20–21 and streams.py:108 were checked at 444f970d.
  • No test names the report. Tests that scan docs/ Markdown run in CI.

Review docs-r1 (Opus 5.5, at 7640fe1c): REQUEST_CHANGES, fixed in 5c307826

It found scope clean, all five r10 items fixed as asked, and the new figures exact against r10's seeded runs and the author's run. It asked for six phrase-level fixes:

  • K30 "never" killed: it was killed in none of r10's 20 seeded runs. docs-r1 found a two-step walk inside the strategy that kills it.
  • "Four counts vary": eight counts have varied, and the r6 table gives each as a range.
  • K25 and S23: S23 differs only for seeds of 2^1203 or more; K25 only for two seeds a multiple of 2^1203 apart, which within the strategy's range means exactly 2^1203.
  • The weakening sentence: the flips and steps past bit 64 carry the test, and the swap result comes from the exact-int test's variants.
  • "Every count" means every count r10's brief asked for.
  • The findings row for r10's fourth nit described the problem the wrong way round.

docs-r1's delta check (at 5c307826): APPROVE WITH NITS. All six fixes hold. Its two optional nits are taken in 961f520a: K25's general case ("a multiple of 2^1203"; this body had said "exactly"), and naming the deterministic test behind the swap result.

🤖 Generated with Claude Code

Review r10 approved #566 at 88b5df2 with one minor finding and four
nits, all in op1_report.md:

- The r9 table's r8-harness row now separates the ten mutants that gain
  the property test in every seeded run from the five that gain it on
  some draws.
- The r5 and r6 tables give "7 or 8" and "54 or 55" for the two
  Hypothesis-dependent cells, and the r9 table names the four counts
  that vary.
- Section 3 words the `git diff --quiet` check unambiguously, names the
  recount commit and says test modules.
- Both timing statements give 8 to 20 s over the author's and reviews
  r8 to r10's runs.

A review r10 section records the verdict and its new mutants. Section 5
states that OP1's exit and harness tests load baselines/asfr.py through
the baselines package import, without displaying it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
social-security-model Ready Ready Preview Oct 26, 2026 12:51am UTC

Request Review

Review docs-r1 asked for six phrase-level fixes to 7640fe1:

- K30: the property test killed it in none of r10's 20 seeded runs,
  not "never"; K24's 9 of 20 is likewise r10's count.
- The r9 table's r6 row says eight counts have varied with
  Hypothesis's draws, as the r6 table's ranges show, and names the four
  in the row of five, A1 included.
- K25 and S23 are told apart: S23 differs only for seeds of 2^1203 or
  more, K25 only for two seeds exactly 2^1203 apart.
- The weakening sentence credits the flips and steps past bit 64, and
  the swap result to the exact-int test's variants.
- r10 reproduced every count its brief asked for.
- The findings-table row for r10's fourth nit describes the problem
  correctly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- K25 conflates any two seeds a multiple of 2^1203 apart; within the
  strategy's range that means exactly 2^1203.
- The swap result names the deterministic test it comes from.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit 0f69bbb into master Oct 26, 2026
12 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Landing audit (Microcosm Dynamics coordinator, 2026-10-26 03:59Z).

Merged at the reviewed head 961f520a02f0d7102763e0555ae4e070bf3f9df6 with --match-head-commit, as merge commit 0f69bbb2. This is #566's docs-only follow-up. It changes only docs/design/open_population_evidence/op1_report.md, and no tier or test file, so it did not need the tier-count slot.

Gates checked at 03:57Z:

  • gh pr checks 589 exited 0. Tests run 40251263402 succeeded at 961f520a in every job.
  • MERGEABLE, merge state CLEAN, not a draft, no changes-requested review.
  • Three commits on master 444f970d, which was still master at merge time: 34 lines added and 13 removed, in the one file.

Review: one independent Claude Opus 5.5 check (docs-r1), run in-session because no Subfleet lane was open.

  • It asked for six phrase-level fixes at 7640fe1c, nothing blocking.
  • It approved 5c307826 with two optional nits.
  • It approved 961f520a after both were taken.

It checked each changed claim against review r10's evidence. This PR closes r10's one minor finding and four nits on #566.

MaxGhenis added a commit that referenced this pull request Oct 26, 2026
Master brings OP1 (#566) and its report fix (#589). Git auto-merged
scripts/first_estimates_birth_evidence.py and
tests/estimates/test_birth_evidence_artifact.py, which both branches
changed: the merge keeps master's OP1 lines and SSI0's exclusions and
reachability block, and removes no line from either side.

tests/tier_counts.json and tests/README-tiers.md conflicted. They now
read master's counts plus the SSI0 test modules' 562 tests, counted by
collection at this merge:
- unit: 547 (contract 20, runner 41, inventory 486)
- oracle_policyengine: 15 (engine 11, dataset engine 3, and the static
  offset test, one unparametrized function, counted from its source)

So they read unit 19,327, oracle_policyengine 323 and 25,912 in all; the
other tiers are master's. SSI0 adds no test to the birth-evidence module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 26, 2026
Master brings #566 (OP1: the open_population package and its evidence),
#589 (OP1's report) and changes to the birth-evidence script and its
artifact test. None of it changes the registry module, its two test
modules or its mutation specs, and the registry module imports only the
standard library and yaml.

The tier conflict resolves to master's counts plus this branch's nine
unit tests: unit 18,780 + 9 = 18,789, total 25,359. The README tier
table matches the tier file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 26, 2026
Master brings #589, #590 and #580: an OP1 report section, a docs
ledger, and the held-out registry's module, tests and mutation record.
None is a B0.2 file or anything B0.2's script imports.

The tier files conflicted. Resolved as master's counts plus this
branch's unit delta: 18,789 + 236 = 19,025 unit, total 25,595. The
other tiers are unchanged, and the README table matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 26, 2026
Master brings #566, #589, #590 and #580. Git auto-merged
scripts/first_estimates_birth_evidence.py and
tests/estimates/test_birth_evidence_artifact.py, which both branches
changed: against each parent the merged files only add lines, so the
merge keeps master's OP1 exclusions and CE1's exclusions and removes no
line from either side. No CE1 path changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis deleted the openpop-op1-report-r10-20261026 branch November 1, 2026 17:00

This branch was successfully deployed

1 active deployment
Preview — 961f520a Deployed Oct 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant