Skip to content

test: range_pruning's twin is a port by name, and its unbounded arms can fail (#1341) - #1529

Merged
jdatcmd merged 1 commit into
mainfrom
test/1341-range-pruning-twin
Oct 17, 2026
Merged

jdatcmd merged 1 commit into
mainfrom
test/1341-range-pruning-twin

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator

For #1341's box "Shell arms added after 2026-09-09 with no pytest counterpart" (range_pruning, one of its INCOMPLETE suites), and #1339's test_range_pruning.py:1 [twin_mismatch].

What changes

  • The twin is now a port by name. It had its own fixture and names, and 28 of the shell's 31 arms had no counterpart. It now uses the shell's fixture (50,000 rows, one row group of five vectors) and the shell's checks by name, one check per test. It goes from 2 tests to 34. compare_to_bash.py gives 30 literal matches, 1 by template, 0 missing. range_pruning moves from INCOMPLETE to COMPLETE.

  • Every reading goes through query_error(). A failed query fails under its check's name, and the test: twins turn a failed query into query_error(), never their own error text (#1339) #1527 guard passes.

  • The shell's two unbounded-range oracle arms can now fail. They are "a unit holding an unbounded range is never pruned from above" and "the same for a containment probe far to its right". They couldn't fail before, for three reasons, each measured on PG17:

    1. fingerprint sums payload, which u_col and u_heap didn't have. Both sides errored, q returned "", and check compared "" with "": u_col=[] u_heap=[].
    2. With that fixed, an 'empty' row in the same unit kept them green. An empty range sorts first, so it becomes the unit's minimum, and native_range_excludes never prunes a unit whose minimum is empty. e_col already holds the empty case on its own.
    3. With that fixed, the unbounded row's separate INSERT put it in a row group of its own. That group had no finite bound to prune on, so the shell stayed green while the twin (one statement) went red.

    The tables now have payload, no empty row, and one INSERT. Both arms use check_text, which refuses an empty side.

Evidence

Mutant Shell Pytest
writer ignores an unbounded range (columnar_write_state.c, the rupper.infinite branch does nothing) the 2 unbounded arms + "present but empty", on each of 15-19 the same 3 (PG17)
the same mutant against main "present but empty" only 0 red
reader never uses max_upper (haveUpper = false, the ledger's existing mutation) "skips the four vectors…" + "reads no chunk group at all" the same 2

Clean runs:

  • Shell: 33/33 on each of 15-19.
  • Pytest: 34 passed on each.
  • harness_selftest: 1247.

Rebased onto #1527:

  • guards 442;
  • test_harness_deps 20;
  • collection 932, which is cluster_tests 900 + 32.

Ledger:

  • Three rows move from never to dated, by the writer mutant on 15-19.
  • No row is added, removed or renamed.
  • Census 3080 → 3077 never, 1378 dated, 4455 rows.
  • The gate passes against origin/main; orphan scan 0.

The other INCOMPLETE suites in this box are native_format, large_values, iceberg_data_files and native_upgrade_converge. Separate PRs.

🤖 Generated with Claude Code

…can fail (#1341)

The twin was an independent port: its own fixture and names, and 28 of
the shell's 31 arms had no counterpart (#1339 test_range_pruning.py:1).
It is now the shell's fixture and checks by name, one check per test,
with every reading through query_error() (2 -> 34 tests).

Porting it showed that the shell's two unbounded-range oracle arms could
not fail, for three reasons, each measured:
- the fingerprint sums payload, which u_col and u_heap lacked, so both
  sides errored and check compared "" with "" (PG17: u_col=[] u_heap=[]);
- an empty row in the same unit made the unit's minimum empty, and such a
  unit is never pruned;
- the unbounded row went in by its own INSERT, into a row group of its
  own.
With all three fixed, and check_text refusing an empty side, a writer
that ignores unbounded ranges reddens both arms and 'present but empty'
on each of 15-19, in both twins. Three ledger rows dated, census 3080 ->
3077.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Approving 9cf1e78d. A reviewer agent reproduced every claim on PG17 and PG18. On each mutant, the shell's FAIL lines and the twin's red set as shipped name the same checks.

mutant shell reds twin reds
none 0 (33/33) 0 (34 passed)
yours: writer ignores an unbounded range the two unbounded arms + "present but empty" same 3
yours: reader never uses max_upper "skips the four vectors only their upper bounds rule out", "reads no chunk group at all" same 2
ours: containment never checks the least lower bound "so does a containment probe" same 1
ours: overlap's lower-side test compares probeLower instead of probeUpper week-long window, overlap ["A","C"], overlap ["B","M"], "the rows themselves match" same 4
ours: writer treats an empty range as unbounded "an empty range contributes no bound, so its units record NULL" same 1

The writer mutant against main's copies gives the shell "present but empty" only and the twin 0 reds, so the shell fix is what lets the two arms fail. A probe twin without payload now fails with "the left side is a failed query" instead of passing.

Bookkeeping checked:

  • compare_to_bash: 30 literal, 1 by template, 0 missing.
  • Gate: rc=0 (3077 / 1378 / 4455; ceiling 185 → 185).
  • Orphan scan: 0.
  • sort -c: sorted.
  • Guards: 442.
  • Collection: 932, the same as cluster_tests.

Notes, none blocking:

  1. The collation choice can differ from the shell's (test_range_pruning.py:312-320). The twin takes candidates in pg_collation order, with no ORDER BY array_position(...), so on an en_US.UTF-8 box the shell picks ucs_basic and the twin picks C. Both discriminate. It also drops ICU (collprovider <> 'i'), so on a host where only unicode discriminates, the twin would decline while the shell runs. _q already turns a failing ICU query into query_error(), so the filter isn't needed.
  2. test_a_discriminating_collation_was_found (:381-388) can't fail once one is found. It re-asks the question the fixture used to choose. The shell records the name only as a SKIP.
  3. _ids_match (:363-367) treats None vs None as the same ids, as pgc_set_hash does in the shell. The "both tables hold the same rows" premise rules it out in practice.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd merged commit 169c265 into main Oct 17, 2026
15 checks passed
@jdatcmd

jdatcmd commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD Merged at 169c265d. Thanks for the three extra mutants. I'll take notes 1 and 2 in a small follow-up: candidates in the shell's order (ORDER BY array_position), with no ICU filter since _q already turns a failing collation into query_error(). For the found-collation test, I'll make it assert that the chosen collation is the shell's first discriminating candidate, so a twin that picked differently goes red, rather than re-asking its own question.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants