[antonwolfy]

Summary

The sign-flip interval search starts each end at about twice the largest delta and sums sign patterns of the shifted deltas, so for deltas near the float maximum it leaves the float range (#725). On main:

deltas	sign_flip_row / _describe_deltas on main
[1e308, 0.0]	RuntimeWarning: invalid value encountered in matmul at eval/sign_flip.py:141 (an error under -W error)
[1.7e308, 0.0], [max, 0.0]	the same warning
[max, -max]	the same warning, twice (sign_flip_row; _describe_deltas refuses it, the magnitudes sum past the range)
[max, 1, 1, 1, 1, 1] through sign_flip_summary	the same warning

Without -W error a nan flows through the comparisons, so the ends are silently wrong. repo compare (sign_flip_summary) and the arms track (_describe_deltas) both go through sign_flip_row -> sign_flip_interval, so one fix covers both.

Of the two options the issue lists, this takes scaling, not refusing: check_sum_range already lets these deltas through (the sum of their magnitudes is finite), and an open PR on the interaction contrasts asserts that a result comes back for [1.7e308, 0.0] and [max, 0.0], which a refusal in sign_flip_row would break. When the largest |delta| is above max float / (8 * n), sign_flip_interval divides every delta by a power of two (exact), runs the same search with EPS and TOL scaled the same way, and multiplies each end back. An end the float range cannot hold is None (open), never inf. sign_flip_p runs on the unscaled deltas: it neither warns nor gives a wrong value at 1e308 (plain float sums go to inf, which compares as it should), so it is not scaled.

The scaled search gives the right interval for every row tried: [1e308, 0.0], [1.7e308, 0.0], [max, 0.0] and [max, -max] have open ends (2 deltas); a 6-delta exact row and a 20-delta Monte Carlo row at 1e307 agree with the same rows scaled down by powers of two; [max, 1, 1, 1, 1, 1] has an open upper end because the true end is just past the maximum.

Closes #725

Changes

• dokimi/dokimi/eval/sign_flip.py: sign_flip_interval scales as above (_scale_exponent, _scaled_back, _ends); ShiftTest takes an eps and _end a tol, defaulting to EPS and TOL, so the unscaled path is the same call as before. The docstring states that an end beyond the range is open.
• dokimi/tests/test_sign_flip_float_range.py: rows near the maximum return a dict and warn nothing (sign_flip_row and _describe_deltas, under warnings.simplefilter("error")); the ends of a row 2**25 times larger equal 2**25 times the ends of the base row (exact and Monte Carlo rows); an end beyond the range is None.
• dokimi/tests/test_repo_compare_overflow.py: twelve deltas of plus and minus 1e307 were refused by sign_flip_summary ("too large to test") only because the interval search overflowed. The search now runs, so the test expects the summary (finite interval around the mean 0). The deltas alternate in sign: sum() compensates only from Python 3.12, and with six +1e307 before six -1e307 the 3.11 partial sum rounds, so the mean was -2.079e290 on the 3.11 legs. Alternating deltas cancel exactly in any order. The mean of the summary (checked_mean, a plain sum()) still differs by Python version for deltas that cancel only in a compensated sum: tracked in checked_mean returns a rounding residue for deltas that cancel on Python < 3.12 because it uses plain sum() #772.
• dokimi/docs/benchmarking-and-comparison.md: an end the float range cannot hold is open, never inf.
• Changelog fragment (Fixed).

The mde of a spread this large still warns in repo compare (_round of an overflowing product): #769.

Risk

Rows whose largest |delta| is at or below max float / (8 * n) run the same search as before; scaling by a power of two is exact, so every existing test of the interval passes unchanged. Rows above it used to warn and give nan-driven ends or OverflowError; they now give an interval. repo compare already drops task values past 1e150, so its CLI output does not change; sign_flip_summary called directly on deltas of about 1e307 returns a summary where it raised CompareError.

Testing

On a Linux review host, at 23497de89c0c462a596848668a85e567471188bc (on main 7137604):

• make ci-local: 0 failed, 33 passed, 14 skipped (docker-needing steps, jobs that do not run on a PR, matrix legs, micromamba, semgrep, fuzz). 20456 tests passed, 19 skipped.
• Repro: python -W error on [1e308, 0.0] fails at main with the matmul RuntimeWarning and returns p 1.0 with open ends here without a warning; the new tests fail on main's code (12 of 15 failed) and pass here.
• Mutants, one break per copy: no scaling (_scale_exponent always 0) fails 12 of the 15 tests of the new file, and the updated test_repo_compare_overflow.py test; an end that overflows on the way back kept as inf instead of None fails test_an_end_beyond_the_float_range_is_open (both rows). Not killable: EPS and TOL left unscaled, and scaling rows below the threshold: scaling is exact, so the ends are the same.
• Python 3.11 failure: the CI value -2.0790003224319997e+290 reproduces with a left-to-right sum (what sum() does before 3.12) of the old twelve deltas; the alternating deltas give 0.0 under that sum, and the test passes with finite_delta.sum replaced by it. Mutant: _scale_exponent always 0 fails the changed test on a copy.
• Not run: the other Python versions (3.11, 3.12, 3.14, the sum behaviour emulated above) and CI's merge commit.

Revert

A plain revert restores the old behaviour: the warning and nan-driven ends for deltas near the float maximum. Nothing is stored.

Checklist

• Tests added/updated and pytest passes
• ruff, mypy, and schema checks pass
• Docs updated (or N/A) — see the docs-drift checklist in CONTRIBUTING.md
• Self-reviewed per "Before opening a PR" in AGENTS.md
• No ad-hoc run artifacts committed
• New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)

[exai-bot]

✅ OpenCodeReview: Review complete: 0 finding(s) across 2 selected item(s).

CI at 20a7f92: 25 passing, 3 skipped.

Review details

• Commit reviewed: 20a7f92 against merge base 23497de
• Coverage: 2 files in 1 groups
• Engine: OpenCodeReview v1.13.5-exai.4.8f73eada, us.openai.gpt-6.1-sol
• Usage: 81k tokens (53k from cache), 17s
• Tool calls: 10 (7 file_read, 2 code_search, 1 local_check_results)
• small change set: 2 files

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.72%. Comparing base (7137604) to head (20a7f92).
⚠️ Report is 1 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #770   +/-   ##
=======================================
  Coverage   95.72%   95.72%           
=======================================
  Files         415      415           
  Lines       55363    55386   +23     
  Branches     9516     9520    +4     
=======================================
+ Hits        52994    53019   +25     
+ Misses       1466     1465    -1     
+ Partials      903      902    -1     

Files with missing lines	Coverage Δ
dokimi/dokimi/eval/sign_flip.py	100.00% <100.00%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[antonwolfy]

Review points answered at 23497de; CI is green.

Follow-up issues opened:

• checked_mean returns a rounding residue for deltas that cancel on Python < 3.12 because it uses plain sum() #772: checked_mean returns a rounding residue for deltas that cancel on Python < 3.12 because it uses plain sum()

codecov has not reported on this head yet.

[vchamarthi]

Pushed 1 commit to fix/sign-flip-interval-range (23497de..20a7f92).

• sign_flip_p divides deltas above max float / (8 * n) by a power of two before summing.
  Run: the full suite with main merged in, 20504 passed.

The PR description needs these corrections:

• "sign_flip_p runs on the unscaled deltas: it neither warns nor gives a wrong value at 1e308 (plain float sums go to inf, which compares as it should), so it is not scaled." should read: sign_flip_p divides the deltas by the same power of two, so its p and the interval agree near the float maximum.