Skip to content

Modenov's computations as a corpus, with the book's errata as rows the library must not confirm - #2472

Open
Rafael-SOWNet wants to merge 1 commit into
masterfrom
feature/modenov-corpus
Open

Rafael-SOWNet wants to merge 1 commit into
masterfrom
feature/modenov-corpus

Conversation

@Rafael-SOWNet

Copy link
Copy Markdown
Member

A new harness, Sources/Tests/Harnesses/ModenovCorpus, in CasBench's style: the computations of P. S. Modenov, Problems in Geometry (tr. Yankovsky, Mir, 1981), as 642 rows, with the book's errata as rows the library must not confirm. This is the plan posted on the issue on 2026-11-15, carried to all five chapters, since every part of the workspace readings had the same shape.

  • modenov.tsv: id, source (chapter, section, problem), op, variable, input, expected, sample points, note. Four ops:
    • zero: the input is an identity; Simplify should reach 0.
    • equal: Simplify should reach the book's form, judged by size as CasBench does.
    • solve: each root found is substituted back, and each root the book gives must be found; a missing root is wrong. Integer families (2*pi*n_1) are handled.
    • erratum: the book is wrong here; the library must not reduce the difference to 0 or decide the equality true.
  • Each row's own claim is checked first, at 6 exact rational points in its domain (complex, real, positive, or on the unit circle from Pythagorean triples). A row whose claim fails is BadRow and fails the run, so a typo in the data cannot pass for a library failure. The library's answer is then checked against the input by value, vector and matrix rows included.
  • modenov-baseline.tsv holds a verdict per row. The run fails on a wrong answer, an erratum the library confirms, a bad row, or any change of the list in either direction, through the shared Harness.Baseline, as CanonCheck does. A wrong row is never baselined; a timeout counts as unmeasured.
chapter rows solved refused unsolved
I 55 39 6 10
II 62 41 6 15
III 347 261 23 63
IV 54 45 2 7
V 124 94 19 11
all 642 480 56 106

On master: 0 wrong, 0 error, 0 timeout, 0 bad rows. Of the 28 unsolved solve rows, 27 have complete, verified roots not reduced to the book's form. Six computations are left out with a note in the data file, because their values cannot be compared (a set defined by a condition, an infinite limit, sums to a natural n and one convergent only for |q| < 1, a congruence of angles); probes abbreviated with … and systems (no system op) are not included.

Every row was also checked numerically in SymPy or mpmath when extracted; some rows are assembled from formulas the readings state rather than verbatim probes, and one (III.P38.a) corrects a typo in the reading's own probe.

Wired into Harnesses.yml after Confluence; the harness README, CLAUDE.md and Docs/References.md updated. A run takes 59–94 s locally; the slowest rows (III.Ex17, III.P25.5, III.P25.4.inv) take 15–17 s against a 20 s budget each, and a timeout is unmeasured rather than a failure.

Two runs on the branch gave identical totals, and a third after the rebase onto c530e40: "The findings match modenov-baseline.tsv: 642." Full check before the rebase: 20,119 unit tests, 0 failed; MCP 444, Terminal 69, F# 134, Interactive 19; AOT 78/78; BoundCheck 0 disagreements, RootCheck 596/596, CasBench 120/121 with 0 wrong, SimpSweep 10463/10463, RuleCheck clean, CanonCheck and Confluence at their baselines.

Closes #2153.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.97%. Comparing base (1a80be1) to head (bc6dc69).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #2472   +/-   ##
=======================================
  Coverage   86.97%   86.97%           
=======================================
  Files         324      324           
  Lines       71840    71840           
  Branches    20952    20952           
=======================================
  Hits        62483    62483           
  Misses       5024     5024           
  Partials     4333     4333           
Flag Coverage Δ
cpp 37.77% <ø> (ø)
interactive 39.81% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Rafael-SOWNet

Copy link
Copy Markdown
Member Author

Rebased onto 73b77bf after #2470 merged (one .editorconfig conflict, both entries kept). The corpus on the rebased tree: 642 rows in 70 s, 480 solved, 56 refused, 106 unsolved, 0 wrong, 0 error, 0 timeout, 0 bad rows; the findings match the baseline.

…e library must not confirm

ModenovCorpus is a harness in the style of CasBench, over 642 computations from Modenov's
Problems in Geometry (Mir, 1981) that the #2131 readings found and checked by value. There are
55 rows from Chapter I, 62 from II, 347 from III, 54 from IV and 124 from V. By kind, there are
341 identities, 200 reductions to a form the book gives, 45 equations with the book's root sets,
and 56 errata.

Every row's own claim is checked first, at six exact sample points from the row's domain (complex,
real, positive, or the unit circle). The run fails on a row whose claim does not hold, so a
mistyped row cannot pass as the library's failure. Each answer is then checked by value:
- a simplification against its input;
- each root substituted back, and every root the book gives found;
- an erratum passes if the library neither reduces the difference to 0 nor decides the equality
  true.

modenov-baseline.tsv records each row's verdict. The run fails when one changes either way, as
CanonCheck's does, and on any wrong answer, which is never recorded. On 3abd4d9 the counts are
480 solved, 56 refused, 106 unsolved, 0 wrong, 0 error, 0 timeout, in about 85 s. Harnesses.yml
runs it.

Part of #2153

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Rafael-SOWNet

Copy link
Copy Markdown
Member Author

Rebased onto 1a80be1 (#2471, #2475). Its .editorconfig entry now sits near the top of the file instead of at its end, where every PR appends. The corpus on the rebased tree: 642 rows, 0 wrong, the findings match the baseline.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

Modenov's computations as a corpus, with the book's errors as rows that must be refused

1 participant