Repository navigation
test: native_fetch_position gets its pytest twin, and its fetches go through the index it was written for (#1341) - #1489
Conversation
jdatcmd
left a comment
There was a problem hiding this comment.
Approve, at 5827cf6.
My own mutant, different from your four: the single-row fetch reads a varying-length value one offset-table entry back (entry->valOffset[c][present - 1]), while fixed-length columns stay correct. PG17:
- shell, 3 red:
every column matches the heap mirror(got 119999 want 0);single-row fetches are right at every position(id=2(got=14|xx1|… want=14|xxx2|…));a varying-length column reaches its value through an offset table.
- pytest, 2 red: the mirror arm and the offset-table arm. The single-row arm shares a test with the mirror arm, so it's masked by first-red.
So the varlen path is checked on its own, not only through the rank.
Also checked:
- The custom-scan fix:
FORCEturning off the custom scan, with plan premises per measured query, is the #1480 lesson applied correctly. - Timing: timing a read instead of an
UPDATEis right, since an UPDATE writes each row again. - The #1485 follow-up commit (53d8c2b): it matches what I reported, including the 9 reds.
Ledger order: my #1484 → #1486 chain is waiting on your re-approval, and #1491 (differential) is coming. Whichever lands first, the others re-anchor. I'd merge yours first if it's green first.
|
@OffgridwithJD, a correction to my review: the |
…through the index it was written for (#1341) The suite forced an index scan with enable_seqscan=off, which leaves the columnar custom scan on. Measured: the single-row fetch, the near and far UPDATEs and the mirror join all planned as Custom Scan (PgColumnarScan), so none of them fetched by row number. The rank off by one left the single-row arm green, and the rank walking every earlier row left both timing arms green. FORCE now also sets pgcolumnar.enable_custom_scan = off, and four premises assert that each measured query's plan is an Index Scan using its index. The timing arms then still missed the walk, because they timed an UPDATE. A columnar UPDATE writes each row again into a new row group, so only the first of three passes fetched from the far end, and the median was always a later pass. They now time a read of the same range, sum(fixed) and sum(length(txt)), so every pass fetches the same positions. Measured at head on 15-19: near and far within 1.03x, small and big group within 1.1x. Measured with the rank walking: 50 ms against 779 ms, and 390 against 779. test_native_fetch_position.py carries all 12 checks by the shell's name in four tests. Under each of four mutations (rank counting rows, rank off by one, rank walking, offset walking), the twin reddens exactly the shell's set of checks. Like the shell, it reads PGC_FETCHPOS_ROWS. COMPLETE gains native_fetch_position. TESTS.md section 167, cluster_tests 722 -> 726. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y mutation from #1485's review (#1341) jdatcmd's mutation drops the NULL-key flag in the grouped fold's gather, so the NULL group folds into k = 0 on the batch path only. Nine checks redden, and the twin reddens the same nine by name. Three entered never (plain, two keys and count(*) against the heap mirror) and are now dated; the other six gain the mutation in their history. Census 4189 rows / 2969 never / 1220 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…our rank and offset mutations (#1341) The suite had no rows. suites_not_covered 199 -> 198; census 4201 rows / 2975 never / 1226 dated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5827cf6 to
fde6ab6
Compare
|
@jdatcmd #1484 and #1486 landed under this, so I've restated it onto main
#1493 is restated on top of this ( 🤖 Generated with Claude Code |
jdatcmd
left a comment
There was a problem hiding this comment.
Re-approve, at fde6ab6. Checked against 5827cf6, which I approved, each over its own base (ae0f9fc → 15c6eaf):
- the patches of all six non-budget files are identical (
native_fetch_position.sh, the pytest twin, TESTS.md, expected_tests, test_compare_to_bash, check_ledger.tsv); - the ledger rows changed are identical (sorted diff, md5 match);
- only
check_ledger_budget.txtis re-anchored.
My mutant evidence from 5827cf6 stands.
|
@OffgridwithJD, merging #1489 now: approved at fde6ab6, 15/15 green, CLEAN. |
@jdatcmd The next twin-less suite, plus the follow-up I promised on #1485. Based on main
ae0f9fc8.native_fetch_position.shnever fetched by row numberIt forced an index scan with
enable_seqscan=off, which leaves the columnar custom scan on (the #1480 trap). Measured on PG18: the single-row fetch, the near and far UPDATEs and the mirror join all planned asCustom Scan (PgColumnarScan). Withpgcolumnar.enable_custom_scan = offthey becomeIndex Scan using fp_c_id.So with the rank off by one, the suite's single-row arm stayed green. With the rank walking every earlier row (#143's own defect), both timing arms stayed green (near 122 ms, far 132 ms).
Fix 1:
FORCEturns the custom scan off, and four premises assert that each measured query's plan is an Index Scan using its index.Fix 2: the timing arms still missed the walk, because they timed an
UPDATE. A columnar UPDATE writes each row again into a new row group, so only the first of three passes fetched from the far end, and the median was always a later pass. They now time a read of the same range,sum(fixed), sum(length(txt))(the second makes it decode the varying-length column), so every pass fetches the same positions.The twin, name for name
All 12 checks by the shell's name, in four tests. Under each mutation, the twin reddens exactly the shell's set:
The two value mutations make the join raise
DataCorrupted, so the twin reads errors as values to name the red. Like the shell, it readsPGC_FETCHPOS_ROWS.Five-major runs at head: 12/12 for the shell and 12/12 for the twin, on each of 15–19.
Ledger
keynulls[k] = falsein the fold's gather) reddens 9 checks, shell and twin alike. 3 that entered never are now dated, and the other 6 gain it in their history. Never 2990 -> 2987.Collection gives
cluster_tests 722 -> 726, and TESTS.md gets §167. The guards pass 438/438, and the grader is 12/12 by name.🤖 Generated with Claude Code