Repository navigation
test: every registered suite without a pytest twin is declared, in both directions (#1341) - #1509
Merged
Merged
Conversation
…th directions (#1341) #1341's twin-parity entry: "recording shell suites have no pytest twin, and no list declares them". test_every_pair_in_the_tree_is_declared grades the pairs that exist, so a registered suite with no twin was in no list at all: 168 of the 298 were twin-less, nothing named them, and a new shell-only suite could land unnoticed. test/pytest/unported_suites.txt declares them, one stem per line so two ports never conflict there. A new guard, test_every_registered_suite_is_ported_or_declared_unported, reads the runner's SUITES array the way run_all_versions.sh does (its sed range and an eval, through bash) and holds the file in both directions: - with "audit" dropped from the list: red, "got 'audit'"; - with test_audit.py created while listed: red, "got 'audit'"; - with a stem that is not a suite: red, "got 'nosuch_suite'". Before the file existed the guard was red on its premise. Clean: green. guard_tests 438 -> 439; TESTS.md section 36 gains the row. The no-cluster guard set: 439/439, with PGC_REQUIRE=pyarrow as CI runs it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OffgridwithJD
approved these changes
Oct 16, 2026
OffgridwithJD
left a comment
Collaborator
There was a problem hiding this comment.
I re-measured the guard at the PR head. It's based on current main (#1508 included), and it passes as committed. Then each break went red under the arm's own name:
| tree | result |
|---|---|
audit deleted from the list |
red, got 'audit' (undeclared) |
test_audit.py touched while audit is listed |
red, got 'audit' (ported but still listed) |
nosuch_suite appended |
red, got 'nosuch_suite' (phantom) |
audit listed twice |
red, got 169, want 168 (duplicate premise) |
The pairing rule, test_{stem}.py, matches what test_every_pair_in_the_tree_is_declared and the other pair arms in this file use, so the two guards can't disagree about what "ported" means. Every name the SUITES eval prints is a bare stem, with no paths or arguments that test_{stem}.py would mis-pair.
I'll remove each port's line from unported_suites.txt in my next twin PRs.
🤖 Generated with Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1341's entry "225 recording shell suites have no pytest twin, and no list declares them". It's 168 today.
What was missing
test_every_pair_in_the_tree_is_declaredgrades the pairs that exist. A registered suite with no twin was in no list at all:The fix
test/pytest/unported_suites.txt: the 168 stems, one per line. A port PR removes its own line, so two ports never conflict there.test_every_registered_suite_is_ported_or_declared_unported: it reads the runner'sSUITESthe wayrun_all_versions.shdoes (the same sed range andeval, through bash), so the guard and the runner can't disagree. It holds the file in both directions.Measured, red first
premise: the unported declaration exists: got 'missing'auditdropped from the list… is declared unported: got 'audit'test_audit.pycreated whileauditis listed… no longer declared unported: got 'audit'… is a registered suite: got 'nosuch_suite'guard_tests438 → 439, and TESTS.md §36 gains the row. The no-cluster guard set passes 439/439 withPGC_REQUIRE=pyarrow, as CI runs it. There are no ledger or suite changes.For whoever ports next: remove the stem from
unported_suites.txtand declare it COMPLETE or INCOMPLETE. The guard says which, by name.🤖 Generated with Claude Code