Repository navigation
test: row_triggers gains its pytest twin, and a trigger that never existed no longer passes its comparisons (#1341) - #1617
Conversation
OffgridwithJD
left a comment
There was a problem hiding this comment.
@jdatcmd Approved at a668381f. Measured on PG17 (~/pg/17-g) via runr (the shell, then the twin as shipped), plus a clean run on PG18:
| mutant | shell | twin |
|---|---|---|
| clean, PG17 and PG18 | 14/14 | 14/14 |
nobuffer (tuple_fetch_row_version without the write-buffer fallback, #179) |
10 red | SAME 10 |
nodisk (only the write buffer) |
10 red, a different ten | SAME 10 |
neighbor (mine: a buffered row answered with row n + 1's values first) |
4 red: the multi-row, 1500-row, after-update and insert-then-update comparisons | SAME 4 |
Ledger: each dated row's mutation set equals the measured red sets name for name: the 10 nobuffer rows, the 10 nodisk rows, and the before-row trigger never, which neither reddens.
The rest:
same_as_heaprefuses a heap side that fired nothing;succeededreads the first ERROR line, so any error fails the arm;- the twin's fixture reads every value through
runandq, so a failed statement reds by name rather than erroring the module; - the import's Arrow file goes under
server_dir; unported_suites.txtstem removed.
Non-blocking: under neighbor, the single-row arm ("the trigger saw the same row heap did") stays green, since a one-row buffer has no neighbour to read. The multi-row arms cover it, so nothing needs to change.
Conflicts with #1616 in the registries, as you said. Restate whichever merges second.
🤖 Generated with Claude Code
a668381 to
fe244ba
Compare
|
@OffgridwithJD Restated onto 🤖 Generated with Claude Code |
…nger vouches for the row set (#1341) test/pytest/test_recluster_gate.py ports all 13 checks of test/recluster_gate.sh by name. Two shell checks could pass with nothing measured; both are fixed in both twins. - "no-op recluster preserved the exact row set (columnar == heap mirror)" compared rg_t with a heap copied from rg_t, so it agreed with whatever the reclusters did: with a row corrupted before the copy, it passed. The mirror is now built from the generator (g = 1..20000, then 1..5000 again from the append). - "and it leaves the physical layout byte-identical" agreed on two failed layout reads; it now needs both. Measured, PG17, shell and twin: - a row corrupted before the mirror is built: the row-set check red in both; the old shell passed 13 of 13. - nogate (the #415 gate removed): 4 red. noextent (appended groups ignored) 2, nokey (the key ignored) 2. Each red set is identical in both twins. - clean: 13 of 13 on PG15 to PG19 in both twins. Ledger: 13 rows, 8 dated by the three mutations and 5 never. suites_not_covered 125 -> 124, cluster_tests 2169 -> 2182, TESTS.md section 243. Census 3650 -> 3651 never, 1930 -> 1942 dated, 5593 rows. The ledger gate and the guards pass over 9bd1850. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nothing no longer passes as bounded (#1341) test/pytest/test_recluster_extent.py ports all 17 checks of test/recluster_extent.sh by name, with the races run in threads as the shell runs them in background jobs. Three shell fixes, in both twins: - "the run does not claim rows written during the rewrite" read sorted_rows <= base, which a run that recorded nothing meets: with no extent recorded at all (norun), it passed. It now needs at least one sorted row, and carries its counts. - The race check compared two client clocks, but the rewrite ends 3-4 ms after the insert commits (measured twice), so the order flipped between the twins. It now asks the server, just before the insert, whether the recluster is running, and is renamed "the insert ran while the rewrite was still running" (the suite had no ledger rows). Both checks leave lossy_arms.tsv. - The projection loop's names were built from $arm; they are literal. Measured, PG17, shell and twin (mutations of record_online_sorted_extent): - norun (no extent recorded): 9 red; the old shell reddened 8 and passed the claim check. - noconsec (the run spans ids another writer took) 1, runall (the run claims every group) 4, noproj (a projection's stripe ids not merged) 2. Each red set is identical in both twins. - clean: 17 of 17 on PG15 to PG19 in both twins, and twice more on PG17. Ledger: 17 rows, 10 dated by the four mutations and 7 never. suites_not_covered 124 -> 123, cluster_tests 2182 -> 2200, TESTS.md section 244. Census 3651 -> 3658 never, 1942 -> 1953 dated, 5611 rows. The ledger gate and the guards pass over 1f03f62. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ors no longer vouch for the reader (#1341) test/pytest/test_reader_buffer_reuse.py ports all 22 checks of test/reader_buffer_reuse.sh by name. Three shell fixes, in both twins: - Both heap mirrors (rb_h, rbd_h) were copied from the tables under test, so every differential check agreed with whatever the reader returned: with a row corrupted before the copy, the shell passed 22 of 22. Both are now built from the generators, and the three full-scan comparisons refuse an empty side (check_text), as does the all-groups premise. - "premise: the differential fixture spans several row groups" and the fault-ratio check carry their numbers, so their rows leave lossy_arms.tsv. - The projection loop's eight names were built from $proj; they are literal. The twin pins max_parallel_workers_per_gather = 0, as the shell's cluster does: left at its default, a Gather split the groups and the leader's count read 2 of 4. Measured, PG17, shell and twin: - a row corrupted before the mirror is built: the full-scan and short-group checks red in both; the old shell passed 22 of 22. - noreuse (a fresh group buffer per row group): 1 red, the fault ratio. - staleproj (a projected read skips column 1's bytes and decodes the stale buffer): 8 red in the shell and the same names in the twin, but it also reddens three premises, so it dates nothing. - clean: 22 of 22 on PG15 to PG19 in both twins. Ledger: 22 rows, 1 dated and 21 never. suites_not_covered 123 -> 122, cluster_tests 2200 -> 2226, TESTS.md section 245. Census 3658 -> 3666 never, 1953 -> 1971 dated, 5637 rows. The ledger gate and the guards pass over 1ccaf4e. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) parallel_export_parquet, parallel_copy and the parallel flush each start background workers that connected as the calling role. PostgreSQL 15 and 16 refuse a background worker connection for a role without LOGIN; only 17 added BGWORKER_BYPASS_ROLELOGINCHECK. So after SET ROLE to a NOLOGIN role, the export failed "worker 0 failed: worker exited without reporting a result", parallel_copy failed "coordinator exited without reporting a result", and the parallel flush encoded every column serially. The worker's session user is now the caller on every major. On 17 and later it connects as the caller with the login check bypassed, as before. Below 17 it connects as the session's authenticated user, which logged in, and then makes the caller its session user. The caller's security context is taken inside the worker's transaction, as core's ParallelWorkerMain does, by the workers that run user code (the copy loader and the export worker). And a copy loader refuses to PREPARE as any role but the caller's. From the review (OffgridwithJD): connecting as the authenticated user and stopping there made it the session user, so a copy trigger's RESET ROLE became that user, the loader PREPAREd as it, and the coordinator could not finish it, leaving prepared transactions holding locks; and without the security context a SECURITY DEFINER copy's trigger could switch to the session user. Setting the context before the first transaction tripped StartTransaction's assert on an assert-enabled build. nologin_workers and its twin, 15 checks: the export, the copy and the flush as a NOLOGIN role; a copy trigger that resets the role still writes as the caller and leaves nothing prepared; one that switches to another role is refused before PREPARE and leaves nothing prepared; and a copy run by a SECURITY DEFINER function keeps its restriction in the workers. 15/15 shell and twin on each of PG15-19. 9 checks are dated by five mutations, with the same red sets in both harnesses: the pre-fix connect; no caller user id; no session user (the old head); no security context; and no PREPARE guard. Census 3492 -> 3498 never, 1715 -> 1724 dated, 5222 rows; suites_not_covered 150 -> 150; cluster_tests 1782 -> 1797; TESTS.md section 217. The ledger gate, guards (464), name grading and the self-test pass over 9e812a1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er passes its comparisons (#1341) test/pytest/test_scan_direction.py ports all 8 checks of test/scan_direction.sh by name. Two shell readers could pass with nothing measured; both are fixed in both twins. - state_of() printed 00000 whenever psql printed no ERROR line, and a failed connection prints none, so both controls passed with the server unreachable. It now reads "no reading (psql rc N: ...)" when psql exits non-zero without a SQLSTATE. - The two cursor comparisons read two identical outputs as "matches", two identical connection errors included. same_rows() reads an empty or failed side as "no reading". With every probe pointed at a dead port the old shell passed four checks (both comparisons and both controls); it now fails all six probes. The twin pins max_parallel_workers_per_gather = 0, as the shell's cluster does. Measured, PG17, shell and twin: - noback (the access method ends a backward scan instead of raising 0A000): 2 red, the backward fetch and FETCH LAST. - claimback (the custom scan claims backward support, so no Materialize sits under the SCROLL cursor): 1 red, the shipped-defaults comparison. - Each red set is identical in both twins; a dead port reddens the same 6 in both. - clean: 8 of 8 on PG15 to PG19 in both twins. Ledger: 8 rows, 3 dated by the two mutations and 5 never. suites_not_covered 122 -> 121, cluster_tests 2241 -> 2249, TESTS.md section 247. Census 3672 -> 3676 never, 1980 -> 1984 dated, 5660 rows. The ledger gate and the guards pass over 5370cf9. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…be count no longer reads as ordered (#1341) test/pytest/test_qual_order_selectivity.py ports all 11 checks of test/qual_order_selectivity.sh by name. Two shell fixes, in both twins: - "and it does not depend on the order the predicates are written in" read a missing probe count as 0, inside its bound: with the counter reporting nothing (zeroprobes), it passed. It now needs a positive count. - The heap mirror was copied from qos, so with a row corrupted before the copy both oracle checks passed. It is built from the generator. Measured, PG17, shell and twin: - zeroprobes: both probe checks red; the old shell reddened one. - notranspose (an excluding predicate is never moved ahead, #403 item 4): both probe checks red. Each red set is identical in both twins. - a row corrupted before the mirror: the oracle checks red; the old shell passed them. - clean: 11 of 11 on PG15 to PG19 in both twins. Ledger: 11 rows, 2 dated by the two mutations and 9 never. suites_not_covered 121 -> 120, cluster_tests 2249 -> 2260, TESTS.md section 248. Census 3676 -> 3682 never, 1984 -> 1989 dated, 5671 rows. The ledger gate and the guards pass over 895116d. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… both plans no longer agrees (#1341) test/pytest/test_pushdown_report.py ports all 51 checks of test/pushdown_report.sh by name. Three shell fixes, in both twins: - Five checks compared one EXPLAIN field read from two plans (the two #726 plain-EXPLAIN checks, the grouped/ungrouped agreement, and the two #493 scalar agreements) and agreed when the field was missing from both. With the vector-predicate line never printed (novecpreds), "plain EXPLAIN reports the ungrouped node's vector predicates (#726)" passed. They are check_num. - The six "exactly one emitter for ... (#495)" names were built in a loop and carried escaped quotes; they are literal and unquoted (the suite had no ledger rows). - "the executor filters far more rows with pushdown off" carries its counts and leaves lossy_arms.tsv. Measured, PG17, shell and twin: - novecpreds: 3 red; the old shell reddened 2 and passed the #726 check. - plan726 (plain EXPLAIN of the ungrouped node reports no pushed-down filters): 2 red. Each red set is identical in both twins. - clean: 51 of 51 on PG15 to PG19 in both twins. Ledger: 51 rows, 5 dated by the two mutations and 46 never. suites_not_covered 120 -> 119, cluster_tests 2260 -> 2311, TESTS.md section 249. Census 3682 -> 3713 never, 1989 -> 2009 dated, 5722 rows. The ledger gate and the guards pass over 4855564. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…isted no longer passes its comparisons (#1341) test/pytest/test_row_triggers.py ports all 14 checks of test/row_triggers.sh by name. Two kinds of shell check could pass with nothing measured; both are fixed in both twins. - The seven comparisons of a columnar table's trigger log with its heap twin's read "0/0/0" = "0/0/0" when neither trigger existed. With every pair's triggers created against a missing function, all of them passed. same_as_heap() now reads "no reading" unless the heap trigger fired, and the names are literal at each call site. - The two "succeeds" checks counted the message "failed to fetch", so any other error -- a missing table's included -- read as success. They now read the first ERROR line, or "succeeded". Measured, PG17, shell and twin: - every pair's triggers against a missing function: 7 red; the old shell reddened 1 ("the row itself landed") and passed the rest. - nobuffer (tuple_fetch_row_version without the write-buffer fallback, #179): 10 red. nodisk (only the write buffer): 10 red, a different ten. Each red set is identical in both twins. - clean: 14 of 14 on PG15 to PG19 in both twins. Ledger: 14 rows, 13 dated by the two mutations and 1 never. suites_not_covered 119 -> 118, cluster_tests 2311 -> 2325, TESTS.md section 250. Census 3713 -> 3714 never, 2009 -> 2022 dated, 5736 rows. The ledger gate and the guards pass over 44d3113. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fe244ba to
6e3609a
Compare
OffgridwithJD
left a comment
There was a problem hiding this comment.
@jdatcmd Re-approving the stacked head 6e3609a0 (I approved a668381f).
- Content unchanged: compared file by file against
a668381f, the PR's own +/- lines match. That covers every file except the budget, expected_tests and TESTS.md, which only restate counts;check_ledger.tsvandlossy_arms.tsvare identical. - Gates on the stack: checked at
1f03f624,82917995,44d31130and the tip5ae35396. Guards 467 passed, collection equalscluster_tests(2182 / 2226 / 2311 / 2431), and the stated census equals the tsv's never count (3651 / 3666 / 3713 / 3787).
🤖 Generated with Claude Code
@OffgridwithJD This ports
test/row_triggers.shto pytest for #1341 (14 checks). It's the second from the bottom of the list. It conflicts with #1616 in the registries, so whichever merges second, I'll restate.Two vacuity fixes, in both twins:
0/0/0 = 0/0/0when neither trigger existed.same_as_heap()now readsno readingunless the heap trigger fired, and each name is literal at its call site.succeeded.Measured on PG17, unless a row says otherwise:
nobuffer(tuple_fetch_row_versionwithout the write-buffer fallback, #179), shell and twinnodisk(only the write buffer)Gates over
82e9c506:suites_not_covered: 149 → 148;cluster_tests: 1800 → 1814, by collection;13 of 14 rows are dated.
🤖 Generated with Claude Code