Repository navigation
test: native_bloom gains its pytest twin (#1341) - #1519
Conversation
jdatcmd
left a comment
There was a problem hiding this comment.
@OffgridwithJD, requesting changes at a7e42a8d for the same thing as #1518. In CI's harness, the first red in a test stops it, so the twin reports less than the shell.
My mutant: the bloom setting inverted
In columnar_reader.c:2053, pgcolumnar_enable_bloom_filter becomes !pgcolumnar_enable_bloom_filter, so the bloom is consulted when off and ignored when on. On PG17:
| red | |
|---|---|
| shell | 2: "bloom on skips groups" (got no), "bloom off skips nothing" (got 9) |
| twin | 1: "bloom on skips groups" (got 0). 5 pass, 1 fail; the off arm never ran |
Both arms sit in test_the_bloom_and_not_min_max_is_what_prunes, and the first one raises.
About "the soft twin" in your evidence tables. The tree has no soft mode: grep -i soft over pgc_vacuity.py and conftest.py finds nothing. So I read those tables as measured with expect patched locally not to raise. That shows the names can go red. It doesn't show what CI's twin reports, and that's the claim "redden the same names" makes. Could you take the red-set tables from the twin as shipped? Then the next defect of this kind shows up in your own table.
Suggested fix: one check per test, by parametrizing the cases as in #1518. Or, if you'd rather fix it once for every twin, a harness change that defers expect failures to the end of the test. That's a bigger question, and the owner should probably weigh in before the harness changes.
Everything else checks out: the -1 sentinel for a missing counter, the at_least on the skip count, and the never-reddened "absent value parity" explained by min/max.
a7e42a8 to
e75d7fd
Compare
…the index (#1341) The shell's "indexed lookup after gap" counted with the defaults, which plan an Index Only Scan. That fetches each row only while the visibility map is unset (so the old suite was already red under an index fetch that finds no row) and answers from the index alone once a VACUUM marks the pages. It is now forced to fetch every row through n_id (custom scan, seqscan and bitmap scans off, count(v)), and a premise asserts the plan, as native_truncate's lookup is. A guard, not a fix. The twin covers all 9 checks by the shell's name, one check per test: a module fixture runs every step in the shell's order and records each reading, so one red cannot stop the checks after it. The file-growth arm is an at_least on the measured growth rather than yes/no. Four runs on PG18 (none, gap fill removed as before 6a16582, the test hook advancing nothing, index fetch finding no row) redden the same names in the shell and in the twin as shipped: 0, 6, 1 and 1. Ledger: native_gap seeded, 4415 -> 4424 rows, never 3066 -> 3069, dated 1349 -> 1355, suites_not_covered 189 -> 188. cluster_tests 754 -> 763, TESTS.md §177, and the stem leaves unported_suites.txt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…osses its row groups (#1341) The suite's header says the read crosses several row groups, and nothing checked it. A new check, "the native table holds five row groups", counts them: stripe_row_limit 1000 over 5000 rows gives five. The twin covers all 9 checks by the shell's name, one check per test: the reads and the answers are parametrized by the shell's names, so one red cannot stop the next. Five runs on PG18 (none; a null by-value column read back as a present 0; a null by-reference one read back as a present empty value, jdatcmd's; the last row group never loaded; zone-map pruning inverted) redden the same names in the shell and in the twin as shipped: 0, 2, 4, 6 and 1. The old suite is red on the same names under each (the by-reference one as jdatcmd measured it). Ledger: native_roundtrip seeded, 4424 -> 4433 rows, never 3069 -> 3071, dated 1355 -> 1362, suites_not_covered 188 -> 187. cluster_tests 763 -> 772, TESTS.md §178, and the stem leaves unported_suites.txt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The twin covers all 7 checks by the shell's name, one check per test, so one red cannot stop the next. Its skip counter reads -1 when the plan has none, so "bloom off skips nothing" cannot pass on a plan without the columnar scan, and "bloom on skips groups" is an at_least on the count rather than yes/no. The shell is unchanged: an absent counter there is an empty string, which neither of its skip arms accepts. Six runs on PG18 (none; the bloom consulted whatever the GUC says; the GUC inverted, jdatcmd's; never consulted; answering absent for every value; never written) redden the same names in the shell and in the twin as shipped: 0, 1, 2, 1, 1 and 2. Ledger: native_bloom seeded, 4433 -> 4440 rows, never 3071 -> 3074, dated 1362 -> 1366, suites_not_covered 187 -> 186. cluster_tests 772 -> 779, TESTS.md §179, and the stem leaves unported_suites.txt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e75d7fd to
e53ba5d
Compare
|
@jdatcmd agreed on all of it, and thank you for both mutants. I took (a), one check per test, in all three PRs:
The red-set tables now come from the twin as shipped, not from my local soft runner. You were right that the soft runner showed only that a name could go red. My runner now runs the real twin file and reads the
Also done:
On (b), a deferred-failure mode for 🤖 Generated with Claude Code |
jdatcmd
left a comment
There was a problem hiding this comment.
Approving at e53ba5d6. I took (a) in this PR and reran my mutant on the twin as shipped, comparing exact red-name sets from E AssertionError: <name>: got lines against the shell's FAIL lines. PG17:
| mutant | shell red | twin red | sets |
|---|---|---|---|
| the bloom setting inverted (`columnar_reader.c:2053`) | 2 | 2 | identical: bloom on skips groups, bloom off skips nothing |
Counts. One cluster_tests line, 779, and collection at this head is 779.
At the stack's tip (e53ba5d6), test_harness_deps, test_compare_to_bash and test_mutation_ledger pass 105.
I'll merge in stack order as CI goes green: #1517, then #1518, then #1519.
|
@OffgridwithJD merging #1519 now at |
…1339) In CI's harness the first red in a test raises, so a twin holding many checks in one test reports fewer reds than its shell. On #1517-#1519 I asked OffgridwithJD to split those twins; this applies the same rule to two of mine. test_native_agg.py went from 3 tests holding 40 checks to 40 tests. Three module fixtures, one per data set, take every reading in order (the deletes, the zone-map counter deltas and the parallel plans happen in the fixture), and each test asserts one. With #133 restored (the metadata path priced at the scan's cost), on PG17: shell 4 red: both "beats a parallel plan", both "no Gather" old twin 1 red: "count(*) beats a parallel plan" new twin the same 4 as the shell test_latest_reading.py went from 1 test holding 9 checks to 9 tests, the same way. With the document's index statement replaced by SELECT 1: shell 2 red: "the document's index exists", "under a tenth" old twin 1 red: the index premise new twin the same 2 as the shell Also from #1514's review: the zone-map read arm takes the measured delta (at_least(delta, 1)) rather than a yes/no, and _plan's RESETs run in a finally. The twin grader still matches every name: 40 of 40, and 9 of 9. GATES and PARITY are literal tables so the grader reads them, and native_agg leaves the loop-reader list. cluster_tests 779 -> 824 by collection. test_native_skip.py, which has the same shape, waits for Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1339) In CI's harness the first red in a test raises, so a twin holding many checks in one test reports fewer reds than its shell. On #1517-#1519 I asked OffgridwithJD to split those twins; this applies the same rule to two of mine. test_native_agg.py went from 3 tests holding 40 checks to 40 tests. Three module fixtures, one per data set, take every reading in order (the deletes, the zone-map counter deltas and the parallel plans happen in the fixture), and each test asserts one. With #133 restored (the metadata path priced at the scan's cost), on PG17: shell 4 red: both "beats a parallel plan", both "no Gather" old twin 1 red: "count(*) beats a parallel plan" new twin the same 4 as the shell test_latest_reading.py went from 1 test holding 9 checks to 9 tests, the same way. With the document's index statement replaced by SELECT 1: shell 2 red: "the document's index exists", "under a tenth" old twin 1 red: the index premise new twin the same 2 as the shell Also from #1514's review: the zone-map read arm takes the measured delta (at_least(delta, 1)) rather than a yes/no, and _plan's RESETs run in a finally. The twin grader still matches every name: 40 of 40, and 9 of 9. GATES and PARITY are literal tables so the grader reads them, and native_agg leaves the loop-reader list. cluster_tests 792 -> 837 by collection. test_native_skip.py, which has the same shape, is next, now that #1510 has merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd Stacked on #1518, which is on #1517. Review only the top commit, e53ba5d.
The twin
test/pytest/test_native_bloom.pycovers all 7 checks by the shell's name, one check per test._skippedreturns -1 when the plan has no "Chunk Groups Removed" counter, so "bloom off skips nothing" can't pass on a plan without the columnar scan.at_least(..., 1)on the count.The shell is unchanged: there an absent counter is an empty string, which neither skip arm accepts.
Evidence: the shell against the twin as shipped (raising
expect), PG18The head run is 7/7 on each of PG15–19.
Bookkeeping
native_bloomseeded with 7 rows: 4433 → 4440, never 3071 → 3074, dated 1362 → 1366,suites_not_covered187 → 186.cluster_tests772 → 779, seven tests.unported_suites.txt.🤖 Generated with Claude Code