Repository navigation
test: native_parquet_flba gets its pytest twin, and two arms see the faults they name (#1341) - #1497
Merged
Merged
Conversation
…through the index it was written for (#1341) The suite forced an index scan with enable_seqscan=off, which leaves the columnar custom scan on. Measured: the single-row fetch, the near and far UPDATEs and the mirror join all planned as Custom Scan (PgColumnarScan), so none of them fetched by row number. The rank off by one left the single-row arm green, and the rank walking every earlier row left both timing arms green. FORCE now also sets pgcolumnar.enable_custom_scan = off, and four premises assert that each measured query's plan is an Index Scan using its index. The timing arms then still missed the walk, because they timed an UPDATE. A columnar UPDATE writes each row again into a new row group, so only the first of three passes fetched from the far end, and the median was always a later pass. They now time a read of the same range, sum(fixed) and sum(length(txt)), so every pass fetches the same positions. Measured at head on 15-19: near and far within 1.03x, small and big group within 1.1x. Measured with the rank walking: 50 ms against 779 ms, and 390 against 779. test_native_fetch_position.py carries all 12 checks by the shell's name in four tests. Under each of four mutations (rank counting rows, rank off by one, rank walking, offset walking), the twin reddens exactly the shell's set of checks. Like the shell, it reads PGC_FETCHPOS_ROWS. COMPLETE gains native_fetch_position. TESTS.md section 167, cluster_tests 722 -> 726. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y mutation from #1485's review (#1341) jdatcmd's mutation drops the NULL-key flag in the grouped fold's gather, so the NULL group folds into k = 0 on the batch path only. Nine checks redden, and the twin reddens the same nine by name. Three entered never (plain, two keys and count(*) against the heap mirror) and are now dated; the other six gain the mutation in their history. Census 4189 rows / 2969 never / 1220 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…our rank and offset mutations (#1341) The suite had no rows. suites_not_covered 199 -> 198; census 4201 rows / 2975 never / 1226 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t says why its UPDATE fetches by row number (#1341) test_native_fetch_cache.py carries all 26 checks by the shell's name in four tests, rebuilding each fixture before every reading as the shell does (#801). Under each of nine mutations, the twin reddens exactly the shell's set of checks: the cache never hitting, the key without the storage id, no earlier-command reject, each of three recheck fields dropped, the discard dropped from executor end and from transaction end, and the cap dropping every column rather than the one that crossed it. Section 1 sets only enable_seqscan/enable_bitmapscan off, and its UPDATE plans as Custom Scan (PgColumnarScan). It measures the cache anyway: the UPDATE fetches its target rows by number whatever the scan. Measured with the cache missing on every lookup, the unchanged suite is red at 844 ms against 93 ms. Its comment said the opposite reason ("index-driven so every row goes through the fetch path rather than a scan") and now says this. COMPLETE gains native_fetch_cache. TESTS.md section 168, cluster_tests 726 -> 730. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… cache mutations (#1341) The suite had no rows. suites_not_covered 198 -> 197; census 4227 rows / 2991 never / 1236 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_native_parquet_pushdown.py carries all 36 checks by the shell's name in four tests, each writing its file with pyarrow and declining under the shell's own pgc_skip name when pyarrow is absent. Under each of seven mutations of the FDW's skip decision, the twin reddens exactly the shell's set of checks: no group ever skipped, the inverted-interval guard dropped, float > and >= pushed, a NaN equality pushed, INT64 statistics not decoded, col < const skipping the groups that match, and an equality skipping every group. The last two are what date the oracle arms: a skip that only stops saving work cannot change an answer. Each test creates its own foreign server, because a server is database-wide and pgc_conn isolates tests by schema only. COMPLETE gains native_parquet_pushdown, and its literal tables join the loop-reader guard's list. TESTS.md section 169, cluster_tests 730 -> 734. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y seven skip mutations (#1341) The suite had no rows. suites_not_covered 197 -> 196; census 4263 rows / 3002 never / 1261 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…faults they name (#1341) test_native_parquet_flba.py carries all 15 checks by the shell's name in three tests. Under each of eight mutations of the FLBA and DECIMAL read path, the twin reddens exactly the shell's set of checks. Two arms change, in both twins, each measured red first: - "pyarrow uuid reads as uuid" read only the smallest value, the all-zero uuid, which every byte order leaves unchanged. With the reader reversing a uuid's bytes it stayed green. It now reads both values in order. - "crafted out-of-range scale is rejected, not decoded" took any error as REJECTED. With the bind guard (pq_decimal_bounds_ok) removed, the decoder's own scale check still refused the file, as XX001 "could not decode Parquet column 0" after decoding began, and the arm stayed green. It now requires the bind's refusal, 42804. COMPLETE gains native_parquet_flba. TESTS.md section 170, cluster_tests 734 -> 737. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…x read-path mutations (#1341) The suite had no rows. suites_not_covered 196 -> 195; census 4278 rows / 3006 never / 1272 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jdatcmd
approved these changes
Oct 16, 2026
jdatcmd
left a comment
Collaborator
There was a problem hiding this comment.
Approve, at 9502a8a (its own commits 6608a71 and 9502a8a, on #1496).
My own mutant, different from your seven: the uuid decode zeroes the last byte after the copy (u->data[UUID_LEN - 1] = 0), not a reversal. PG17 shell, 5 red:
- the three round trips and the FDW scan (md5 mismatch);
- "pyarrow uuid reads as uuid":
got [00000000-…-000000000000,11111111-2222-3333-4444-555555555500] want [...,…555555555555].
The old form of that arm read only the all-zero uuid, which this mutant also leaves unchanged. So your strengthened arm is what catches it, not the round trips alone.
What I could not verify: as on #1496, I can't run the pytest twin here: /root/pyenv has no pyarrow, so it declines. Twin parity rests on your measurement and CI.
This was referenced Oct 16, 2026
Collaborator
|
@OffgridwithJD, merging #1497 now: approved at 9502a8a, 15/15 green, CLEAN. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@jdatcmd The next twin-less suite, stacked on #1496 (#1489 → #1493 → #1496 → this). This PR's own commits are the top two.
Two arms couldn't see the faults they name
ORDER BY u LIMIT 1, the all-zero uuid, which every byte order leaves unchanged. With the reader reversing a uuid's bytes, the old arm stayed green. It now reads both values in order, and that mutant is red in both twins.pq_decimal_bounds_okinpq_want_phys_for) left it green: the decoder's ownscale > 38check refused the file instead, asXX001 could not decode Parquet column 0, after decoding had begun. The bind refuses with42804, and the arm now requiresREJECTED 42804(the shell'spgc_try_flbareturns the SQLSTATE). With the guard removed it's red in both twins.The twin, name for name
All 15 checks by the shell's name, in three tests. Under each mutation, the twin reddens exactly the shell's set:
columnar_parquet_reader.c)parquet_schemaadvises integer for INT32 DECIMALFive-major runs at head: 15/15 for the shell and 15/15 for the twin, on each of 15–19.
Ledger
Census 4278 / 3006 never / 1272 dated (on #1496's 4263 / 3002 / 1261), with suites_not_covered 196 → 195. 11 of the 15 checks are dated. The gate gives rc 0, and orphan-scan finds 0. 4 arrive never:
Collection gives
cluster_tests 734 -> 737, and TESTS.md gets §170. The grader is 15/15 by name, and the guards pass 438/438.🤖 Generated with Claude Code