Repository navigation
test: native_delete_visibility_paths' shell arms take the paths they name, as its twin does (#1339) - #1500
Conversation
test_native_parquet_multifile.py carries all 32 checks by the shell's name in three tests. Under each of nine mutations of the directory walk and glob expansion, the twin reddens exactly the shell's set of checks: an empty directory and an unmatched glob reading as nothing, an unstat-able path dropped, a FIFO opened, _ and . names walked, a symlinked directory descended, the files unsorted, no depth bound, and every extension read. A read that could block runs on its own connection in a thread with a deadline, as the shell runs psql under timeout. A cancel is no substitute: a backend blocked in open(2) on a FIFO resumes the open after a signal. On a timeout the twin opens the FIFO for writing, which releases the reader. The FIFO arm is its own test, so it can decline under the shell's "the FIFO case" when mkfifo is unavailable. COMPLETE gains native_parquet_multifile. TESTS.md section 171, cluster_tests 737 -> 740. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by nine walk and glob mutations (#1341) The suite had no rows. suites_not_covered 195 -> 194; census 4310 rows / 3010 never / 1300 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ough the index (#1341, #1339) native_index.sh forced its index path with enable_seqscan=off alone, which leaves the columnar custom scan on. Measured: the range aggregate planned as Custom Scan (PgColumnarScan), so it fetched nothing by row number. With the index fetch reading the neighbouring row, or finding nothing, the old suite's "index range aggregate" stayed green. iscan now also turns off the custom scan, bitmap and index-only scans, and each lookup first asserts that its plan is an Index Scan through n_pkey (pgc_uses_row_fetch). Both mutations now redden it. test_native_index.py carries all 13 checks by the shell's name in two tests, and under both mutations it reddens exactly the shell's set. COMPLETE gains native_index. TESTS.md section 172, cluster_tests 740 -> 742. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fetch mutations (#1341) The suite had no rows. suites_not_covered 194 -> 193; census 4323 rows / 3017 never / 1306 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…name, as its twin does (#1339) The shell set enable_seqscan / enable_indexonlyscan and trusted the arm names, but none of the enable_* scan GUCs governs Custom Scan (PgColumnarScan). The index / bitmap and index-only arms planned as the columnar custom scan, the same path as the sequential arm. The pytest twin has forced each path with pgcolumnar.enable_custom_scan = off and asserted the node since it was written; its docstring reported the shell's gap. The shell now does the same, under the twin's premise names, and gains the twin's three other checks (the 5000 rows, more than one row group, and the rolled-back delete left the row). Red first: with pgcolumnar_fetch_row ignoring the delete mask, the old shell's "index / bitmap scan sees the live set" and "index-only scan sees the live set" stayed green. Both are now red in both twins, and the twins redden the same set by name under each mutation. The twin's "premise: the table holds all {ROWS} rows" becomes a literal name, which the grader needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e twin's seven checks to the ledger (#1339) Census 4330 rows / 3019 never / 1311 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jdatcmd
left a comment
There was a problem hiding this comment.
Approve, at 29c77e5 (its own commits d9d3ecf and 29c77e5, on #1499). It also closes the twin gap I noted on #1339: the shell now has "the rolled-back delete left the row in place".
My own mutant, different from your two: PgColumnarMarkRowDeleted marks the neighbouring row (rowNumber + 1), so the live count barely moves and only a specific-row arm can see it. PG17:
- shell, 4 red:
a deleted row is invisible via the index(got 1 want 0);a live row is visible via the index(got 0 want 1);the rolled-back delete left the row in place(got 0 want 1);- the #1394 arm (
XX000|20|210).
- pytest: red, with the rest masked by first-red.
A finding, not blocking, and not this PR's: skipping the visibility-map clear on delete (PgColumnarVMClearForRow in columnar_delete_vector.c) leaves both twins 15/15 green. The function's own comment says why: the clear is a no-op unless a vacuum had marked the block visible, and this fixture never vacuums before it deletes. So the index-only arm always fetches, and the gap-28 bug, where an index-only scan trusts a stale all-visible bit, can't show here. A VACUUM t before the DELETE (or a second index-only arm after a vacuum) would make it reachable. I can take that as a follow-up.
|
@OffgridwithJD, merging #1500 now: approved at 29c77e5, 15/15 green, CLEAN. |
@jdatcmd The second #1339 custom-scan audit entry. Stacked on #1499 (#1497 → #1498 → #1499 → this). This PR's own commits are the top two.
The shell's path arms took one path, not four
native_delete_visibility_paths.shsetenable_seqscan/enable_indexonlyscanand trusted its arm names. None of theenable_*GUCs governsCustom Scan (PgColumnarScan), so the index / bitmap and index-only arms ran the columnar custom scan, the same path as the sequential arm. The pytest twin has forced each path withpgcolumnar.enable_custom_scan = offand asserted the node from the start; its docstring reported the shell's gap. The shell now does the same, under the twin's premise names, and gains the twin's three other checks (the 5000 rows, more than one row group, the rolled-back row). The twins now match one for one, with 15 literal matches.Red first:
columnar_reader.c)pgcolumnar_fetch_rowignores the delete vectorThe liveness-cache, batch-scan and buffered-delete mutations redden nothing here. The aggregate arm reads the vectorized count, and the in-transaction read is served through the fetch.
Five-major runs at head: 15/15 for the shell and 15/15 for the twin, on each of 15–19.
Ledger
Census 4330 / 3019 never / 1311 dated (on #1499's 4323 / 3017 / 1306):
The gate gives rc 0, and orphan-scan finds 0. No tests are added, so collection is unchanged.
🤖 Generated with Claude Code