Skip to content

test: every registered suite without a pytest twin is declared, in both directions (#1341) - #1509

Merged
jdatcmd merged 1 commit into
mainfrom
test/1341-declare-unported-suites
Oct 16, 2026
Merged

jdatcmd merged 1 commit into
mainfrom
test/1341-declare-unported-suites

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

Fixes #1341's entry "225 recording shell suites have no pytest twin, and no list declares them". It's 168 today.

What was missing

test_every_pair_in_the_tree_is_declared grades the pairs that exist. A registered suite with no twin was in no list at all:

The fix

  • test/pytest/unported_suites.txt: the 168 stems, one per line. A port PR removes its own line, so two ports never conflict there.
  • A new guard, test_every_registered_suite_is_ported_or_declared_unported: it reads the runner's SUITES the way run_all_versions.sh does (the same sed range and eval, through bash), so the guard and the runner can't disagree. It holds the file in both directions.

Measured, red first

tree result
before the file existed red: premise: the unported declaration exists: got 'missing'
audit dropped from the list red: … is declared unported: got 'audit'
test_audit.py created while audit is listed red: … no longer declared unported: got 'audit'
a stem that isn't a suite red: … is a registered suite: got 'nosuch_suite'
clean green

guard_tests 438 → 439, and TESTS.md §36 gains the row. The no-cluster guard set passes 439/439 with PGC_REQUIRE=pyarrow, as CI runs it. There are no ledger or suite changes.

For whoever ports next: remove the stem from unported_suites.txt and declare it COMPLETE or INCOMPLETE. The guard says which, by name.

🤖 Generated with Claude Code

…th directions (#1341)

#1341's twin-parity entry: "recording shell suites have no pytest twin,
and no list declares them". test_every_pair_in_the_tree_is_declared
grades the pairs that exist, so a registered suite with no twin was in no
list at all: 168 of the 298 were twin-less, nothing named them, and a new
shell-only suite could land unnoticed.

test/pytest/unported_suites.txt declares them, one stem per line so two
ports never conflict there. A new guard,
test_every_registered_suite_is_ported_or_declared_unported, reads the
runner's SUITES array the way run_all_versions.sh does (its sed range and
an eval, through bash) and holds the file in both directions:
- with "audit" dropped from the list: red, "got 'audit'";
- with test_audit.py created while listed: red, "got 'audit'";
- with a stem that is not a suite: red, "got 'nosuch_suite'".
Before the file existed the guard was red on its premise. Clean: green.

guard_tests 438 -> 439; TESTS.md section 36 gains the row. The no-cluster
guard set: 439/439, with PGC_REQUIRE=pyarrow as CI runs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Approved at b69ae70.

I re-measured the guard at the PR head. It's based on current main (#1508 included), and it passes as committed. Then each break went red under the arm's own name:

tree result
audit deleted from the list red, got 'audit' (undeclared)
test_audit.py touched while audit is listed red, got 'audit' (ported but still listed)
nosuch_suite appended red, got 'nosuch_suite' (phantom)
audit listed twice red, got 169, want 168 (duplicate premise)

The pairing rule, test_{stem}.py, matches what test_every_pair_in_the_tree_is_declared and the other pair arms in this file use, so the two guards can't disagree about what "ported" means. Every name the SUITES eval prints is a bare stem, with no paths or arguments that test_{stem}.py would mis-pair.

I'll remove each port's line from unported_suites.txt in my next twin PRs.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missing tests: 87 confirmed behaviours no shell or pytest arm exercises (#1339 audit)

2 participants