Repository navigation
test: native_index gets its pytest twin, and every lookup fetches through the index (#1341, #1339) - #1499
Conversation
…faults they name (#1341) test_native_parquet_flba.py carries all 15 checks by the shell's name in three tests. Under each of eight mutations of the FLBA and DECIMAL read path, the twin reddens exactly the shell's set of checks. Two arms change, in both twins, each measured red first: - "pyarrow uuid reads as uuid" read only the smallest value, the all-zero uuid, which every byte order leaves unchanged. With the reader reversing a uuid's bytes it stayed green. It now reads both values in order. - "crafted out-of-range scale is rejected, not decoded" took any error as REJECTED. With the bind guard (pq_decimal_bounds_ok) removed, the decoder's own scale check still refused the file, as XX001 "could not decode Parquet column 0" after decoding began, and the arm stayed green. It now requires the bind's refusal, 42804. COMPLETE gains native_parquet_flba. TESTS.md section 170, cluster_tests 734 -> 737. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…x read-path mutations (#1341) The suite had no rows. suites_not_covered 196 -> 195; census 4278 rows / 3006 never / 1272 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_native_parquet_multifile.py carries all 32 checks by the shell's name in three tests. Under each of nine mutations of the directory walk and glob expansion, the twin reddens exactly the shell's set of checks: an empty directory and an unmatched glob reading as nothing, an unstat-able path dropped, a FIFO opened, _ and . names walked, a symlinked directory descended, the files unsorted, no depth bound, and every extension read. A read that could block runs on its own connection in a thread with a deadline, as the shell runs psql under timeout. A cancel is no substitute: a backend blocked in open(2) on a FIFO resumes the open after a signal. On a timeout the twin opens the FIFO for writing, which releases the reader. The FIFO arm is its own test, so it can decline under the shell's "the FIFO case" when mkfifo is unavailable. COMPLETE gains native_parquet_multifile. TESTS.md section 171, cluster_tests 737 -> 740. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by nine walk and glob mutations (#1341) The suite had no rows. suites_not_covered 195 -> 194; census 4310 rows / 3010 never / 1300 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ough the index (#1341, #1339) native_index.sh forced its index path with enable_seqscan=off alone, which leaves the columnar custom scan on. Measured: the range aggregate planned as Custom Scan (PgColumnarScan), so it fetched nothing by row number. With the index fetch reading the neighbouring row, or finding nothing, the old suite's "index range aggregate" stayed green. iscan now also turns off the custom scan, bitmap and index-only scans, and each lookup first asserts that its plan is an Index Scan through n_pkey (pgc_uses_row_fetch). Both mutations now redden it. test_native_index.py carries all 13 checks by the shell's name in two tests, and under both mutations it reddens exactly the shell's set. COMPLETE gains native_index. TESTS.md section 172, cluster_tests 740 -> 742. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fetch mutations (#1341) The suite had no rows. suites_not_covered 194 -> 193; census 4323 rows / 3017 never / 1306 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jdatcmd
left a comment
There was a problem hiding this comment.
Approve, at 0c4edf8 (its own commits 7cd05c4 and 0c4edf8, on #1498).
My own mutant, different from your two: the deferred decode (pgcolumnar_slot_getsomeattrs, the #157 path an index fetch hands its slot to) returns k as NULL on every 7th row number. PG17:
- shell:
index range aggregate: got [301|644140|row-100|row-400] want [301|752500|row-100|row-400]; - pytest: the same.
That's the arm your custom-scan fix moved onto the fetch path, so the fix is what makes it see this.
My own miss, for the record: my first attempt put the mutant at the end of pgcolumnar_index_fetch_tuple and stayed green. The slot is deferred there, so the later decode overwrote what I set: an inert mutant, not a gap in the suite. Moving it into getsomeattrs is what reached the runtime.
|
@OffgridwithJD, merging #1499 now: approved at 0c4edf8, 15/15 green, CLEAN. |
@jdatcmd The first of the #1339 custom-scan audit entries I posted, done as a twin PR because
native_index.shhad none. Stacked on #1498 (#1496 → #1497 → #1498 → this). This PR's own commits are the top two.native_index.sh's range aggregate never fetched by row number
iscan()forced the index path withenable_seqscan=offalone, which leaves the columnar custom scan on. Measured on PG18: "index range aggregate" (id BETWEEN 100 AND 400) planned asCustom Scan (PgColumnarScan). Red first, with two mutations ofpgcolumnar_index_fetch_tuple:rowNumber + 1)Fix:
iscannow also setsenable_bitmapscan,enable_indexonlyscanandpgcolumnar.enable_custom_scanoff, so every lookup is an Index Scan that fetches. Four premises assert that plan throughn_pkey, using lib.sh'spgc_uses_row_fetch.The twin
All 13 checks by the shell's name, in two tests. Under both mutations it reddens exactly the shell's set. Five-major runs at head: 13/13 for the shell and 13/13 for the twin, on each of 15–19.
Ledger
Census 4323 / 3017 never / 1306 dated (on #1498's 4310 / 3010 / 1300), with suites_not_covered 194 → 193. 6 of the 13 checks are dated. The gate gives rc 0, and orphan-scan finds 0. 7 arrive never:
Collection gives
cluster_tests 740 -> 742, and TESTS.md gets §172. The grader shows 9 literal + 1 template match with 0 missing, and the guards pass 438/438.🤖 Generated with Claude Code