Skip to content

test: row_triggers gains its pytest twin, and a trigger that never existed no longer passes its comparisons (#1341) - #1617

Merged
jdatcmd merged 8 commits into
mainfrom
test/1341-row-triggers-twin
Oct 21, 2026
Merged

jdatcmd merged 8 commits into
mainfrom
test/1341-row-triggers-twin

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

@OffgridwithJD This ports test/row_triggers.sh to pytest for #1341 (14 checks). It's the second from the bottom of the list. It conflicts with #1616 in the registries, so whichever merges second, I'll restate.

Two vacuity fixes, in both twins:

  • Two missing triggers agreed. The seven columnar-vs-heap trigger-log comparisons read 0/0/0 = 0/0/0 when neither trigger existed. same_as_heap() now reads no reading unless the heap trigger fired, and each name is literal at its call site.
  • Any error counted as success. The two "succeeds" checks counted the message "failed to fetch", so any other error, a missing table's included, read as success. They now read the first ERROR line, or succeeded.

Measured on PG17, unless a row says otherwise:

run result
clean, shell and twin, PG17 and PG15 14/14 in each
every trigger pointed at a missing function, main's shell 9 passed + 5 failed
the same, this shell 0 passed + 14 failed
nobuffer (tuple_fetch_row_version without the write-buffer fallback, #179), shell and twin 10 red in each
nodisk (only the write buffer) 10 red, a different ten (measured when this was written)

Gates over 82e9c506:

  • ledger gate rc=0; census 3506 never + 1731 dated = 5237;
  • suites_not_covered: 149 → 148;
  • cluster_tests: 1800 → 1814, by collection;
  • guards: 464;
  • TESTS.md: §218.

13 of 14 rows are dated.

🤖 Generated with Claude Code

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Approved at a668381f. Measured on PG17 (~/pg/17-g) via runr (the shell, then the twin as shipped), plus a clean run on PG18:

mutant shell twin
clean, PG17 and PG18 14/14 14/14
nobuffer (tuple_fetch_row_version without the write-buffer fallback, #179) 10 red SAME 10
nodisk (only the write buffer) 10 red, a different ten SAME 10
neighbor (mine: a buffered row answered with row n + 1's values first) 4 red: the multi-row, 1500-row, after-update and insert-then-update comparisons SAME 4

Ledger: each dated row's mutation set equals the measured red sets name for name: the 10 nobuffer rows, the 10 nodisk rows, and the before-row trigger never, which neither reddens.

The rest:

  • same_as_heap refuses a heap side that fired nothing;
  • succeeded reads the first ERROR line, so any error fails the arm;
  • the twin's fixture reads every value through run and q, so a failed statement reds by name rather than erroring the module;
  • the import's Arrow file goes under server_dir;
  • unported_suites.txt stem removed.

Non-blocking: under neighbor, the single-row arm ("the trigger saw the same row heap did") stays green, since a one-row buffer has no neighbour to read. The multi-row arms cover it, so nothing needs to change.

Conflicts with #1616 in the registries, as you said. Restate whichever merges second.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd force-pushed the test/1341-row-triggers-twin branch from a668381 to fe244ba Compare October 21, 2026 18:45
@jdatcmd

jdatcmd commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD Restated onto dbd5f443 as fe244bac, which voids your approval here. The suite's shell, twin and ledger rows are the content you approved; only the counts moved, re-derived from the merged tsv. Over this tree the guards pass and the ledger gate is rc=0. A re-approval of fe244bac lets me merge it.

🤖 Generated with Claude Code

jdatcmd and others added 8 commits October 21, 2026 19:44
…nger vouches for the row set (#1341)

test/pytest/test_recluster_gate.py ports all 13 checks of
test/recluster_gate.sh by name. Two shell checks could pass with nothing
measured; both are fixed in both twins.

- "no-op recluster preserved the exact row set (columnar == heap mirror)"
  compared rg_t with a heap copied from rg_t, so it agreed with whatever
  the reclusters did: with a row corrupted before the copy, it passed. The
  mirror is now built from the generator (g = 1..20000, then 1..5000
  again from the append).
- "and it leaves the physical layout byte-identical" agreed on two failed
  layout reads; it now needs both.

Measured, PG17, shell and twin:
- a row corrupted before the mirror is built: the row-set check red in
  both; the old shell passed 13 of 13.
- nogate (the #415 gate removed): 4 red. noextent (appended groups
  ignored) 2, nokey (the key ignored) 2. Each red set is identical in both
  twins.
- clean: 13 of 13 on PG15 to PG19 in both twins.

Ledger: 13 rows, 8 dated by the three mutations and 5 never.
suites_not_covered 125 -> 124, cluster_tests 2169 -> 2182, TESTS.md
section 243. Census 3650 -> 3651 never, 1930 -> 1942 dated, 5593 rows. The ledger
gate and the guards pass over 9bd1850.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nothing no longer passes as bounded (#1341)

test/pytest/test_recluster_extent.py ports all 17 checks of
test/recluster_extent.sh by name, with the races run in threads as the
shell runs them in background jobs. Three shell fixes, in both twins:

- "the run does not claim rows written during the rewrite" read
  sorted_rows <= base, which a run that recorded nothing meets: with no
  extent recorded at all (norun), it passed. It now needs at least one
  sorted row, and carries its counts.
- The race check compared two client clocks, but the rewrite ends 3-4 ms
  after the insert commits (measured twice), so the order flipped between
  the twins. It now asks the server, just before the insert, whether the
  recluster is running, and is renamed "the insert ran while the rewrite
  was still running" (the suite had no ledger rows). Both checks leave
  lossy_arms.tsv.
- The projection loop's names were built from $arm; they are literal.

Measured, PG17, shell and twin (mutations of record_online_sorted_extent):
- norun (no extent recorded): 9 red; the old shell reddened 8 and passed
  the claim check.
- noconsec (the run spans ids another writer took) 1, runall (the run
  claims every group) 4, noproj (a projection's stripe ids not merged) 2.
  Each red set is identical in both twins.
- clean: 17 of 17 on PG15 to PG19 in both twins, and twice more on PG17.

Ledger: 17 rows, 10 dated by the four mutations and 7 never.
suites_not_covered 124 -> 123, cluster_tests 2182 -> 2200, TESTS.md
section 244. Census 3651 -> 3658 never, 1942 -> 1953 dated, 5611 rows. The ledger
gate and the guards pass over 1f03f62.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ors no longer vouch for the reader (#1341)

test/pytest/test_reader_buffer_reuse.py ports all 22 checks of
test/reader_buffer_reuse.sh by name. Three shell fixes, in both twins:

- Both heap mirrors (rb_h, rbd_h) were copied from the tables under test,
  so every differential check agreed with whatever the reader returned:
  with a row corrupted before the copy, the shell passed 22 of 22. Both
  are now built from the generators, and the three full-scan comparisons
  refuse an empty side (check_text), as does the all-groups premise.
- "premise: the differential fixture spans several row groups" and the
  fault-ratio check carry their numbers, so their rows leave
  lossy_arms.tsv.
- The projection loop's eight names were built from $proj; they are
  literal.

The twin pins max_parallel_workers_per_gather = 0, as the shell's
cluster does: left at its default, a Gather split the groups and the
leader's count read 2 of 4.

Measured, PG17, shell and twin:
- a row corrupted before the mirror is built: the full-scan and
  short-group checks red in both; the old shell passed 22 of 22.
- noreuse (a fresh group buffer per row group): 1 red, the fault ratio.
- staleproj (a projected read skips column 1's bytes and decodes the
  stale buffer): 8 red in the shell and the same names in the twin, but
  it also reddens three premises, so it dates nothing.
- clean: 22 of 22 on PG15 to PG19 in both twins.

Ledger: 22 rows, 1 dated and 21 never.
suites_not_covered 123 -> 122, cluster_tests 2200 -> 2226, TESTS.md
section 245. Census 3658 -> 3666 never, 1953 -> 1971 dated, 5637 rows. The ledger
gate and the guards pass over 1ccaf4e.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

parallel_export_parquet, parallel_copy and the parallel flush each start
background workers that connected as the calling role. PostgreSQL 15 and
16 refuse a background worker connection for a role without LOGIN; only
17 added BGWORKER_BYPASS_ROLELOGINCHECK. So after SET ROLE to a NOLOGIN
role, the export failed "worker 0 failed: worker exited without reporting
a result", parallel_copy failed "coordinator exited without reporting a
result", and the parallel flush encoded every column serially.

The worker's session user is now the caller on every major. On 17 and
later it connects as the caller with the login check bypassed, as before.
Below 17 it connects as the session's authenticated user, which logged
in, and then makes the caller its session user. The caller's security
context is taken inside the worker's transaction, as core's
ParallelWorkerMain does, by the workers that run user code (the copy
loader and the export worker). And a copy loader refuses to PREPARE as
any role but the caller's.

From the review (OffgridwithJD): connecting as the authenticated user and
stopping there made it the session user, so a copy trigger's RESET ROLE
became that user, the loader PREPAREd as it, and the coordinator could
not finish it, leaving prepared transactions holding locks; and without
the security context a SECURITY DEFINER copy's trigger could switch to
the session user. Setting the context before the first transaction
tripped StartTransaction's assert on an assert-enabled build.

nologin_workers and its twin, 15 checks: the export, the copy and the
flush as a NOLOGIN role; a copy trigger that resets the role still writes
as the caller and leaves nothing prepared; one that switches to another
role is refused before PREPARE and leaves nothing prepared; and a copy
run by a SECURITY DEFINER function keeps its restriction in the workers.

15/15 shell and twin on each of PG15-19. 9 checks are dated by five
mutations, with the same red sets in both harnesses: the pre-fix connect;
no caller user id; no session user (the old head); no security context;
and no PREPARE guard. Census 3492 -> 3498 never, 1715 -> 1724 dated, 5222
rows; suites_not_covered 150 -> 150; cluster_tests 1782 -> 1797; TESTS.md
section 217. The ledger gate, guards (464), name grading and the
self-test pass over 9e812a1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er passes its comparisons (#1341)

test/pytest/test_scan_direction.py ports all 8 checks of
test/scan_direction.sh by name. Two shell readers could pass with nothing
measured; both are fixed in both twins.

- state_of() printed 00000 whenever psql printed no ERROR line, and a
  failed connection prints none, so both controls passed with the server
  unreachable. It now reads "no reading (psql rc N: ...)" when psql exits
  non-zero without a SQLSTATE.
- The two cursor comparisons read two identical outputs as "matches", two
  identical connection errors included. same_rows() reads an empty or
  failed side as "no reading".

With every probe pointed at a dead port the old shell passed four checks
(both comparisons and both controls); it now fails all six probes.

The twin pins max_parallel_workers_per_gather = 0, as the shell's cluster
does.

Measured, PG17, shell and twin:
- noback (the access method ends a backward scan instead of raising
  0A000): 2 red, the backward fetch and FETCH LAST.
- claimback (the custom scan claims backward support, so no Materialize
  sits under the SCROLL cursor): 1 red, the shipped-defaults comparison.
- Each red set is identical in both twins; a dead port reddens the same 6
  in both.
- clean: 8 of 8 on PG15 to PG19 in both twins.

Ledger: 8 rows, 3 dated by the two mutations and 5 never.
suites_not_covered 122 -> 121, cluster_tests 2241 -> 2249, TESTS.md
section 247. Census 3672 -> 3676 never, 1980 -> 1984 dated, 5660 rows. The ledger
gate and the guards pass over 5370cf9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…be count no longer reads as ordered (#1341)

test/pytest/test_qual_order_selectivity.py ports all 11 checks of
test/qual_order_selectivity.sh by name. Two shell fixes, in both twins:

- "and it does not depend on the order the predicates are written in"
  read a missing probe count as 0, inside its bound: with the counter
  reporting nothing (zeroprobes), it passed. It now needs a positive count.
- The heap mirror was copied from qos, so with a row corrupted before the
  copy both oracle checks passed. It is built from the generator.

Measured, PG17, shell and twin:
- zeroprobes: both probe checks red; the old shell reddened one.
- notranspose (an excluding predicate is never moved ahead, #403 item
  4): both probe checks red. Each red set is identical in both twins.
- a row corrupted before the mirror: the oracle checks red; the old shell
  passed them.
- clean: 11 of 11 on PG15 to PG19 in both twins.

Ledger: 11 rows, 2 dated by the two mutations and 9 never.
suites_not_covered 121 -> 120, cluster_tests 2249 -> 2260, TESTS.md
section 248. Census 3676 -> 3682 never, 1984 -> 1989 dated, 5671 rows. The ledger
gate and the guards pass over 895116d.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… both plans no longer agrees (#1341)

test/pytest/test_pushdown_report.py ports all 51 checks of
test/pushdown_report.sh by name. Three shell fixes, in both twins:

- Five checks compared one EXPLAIN field read from two plans (the two #726
  plain-EXPLAIN checks, the grouped/ungrouped agreement, and the two
  #493 scalar agreements) and agreed when the field was missing from both.
  With the vector-predicate line never printed (novecpreds), "plain
  EXPLAIN reports the ungrouped node's vector predicates (#726)" passed.
  They are check_num.
- The six "exactly one emitter for ... (#495)" names were built in a
  loop and carried escaped quotes; they are literal and unquoted (the
  suite had no ledger rows).
- "the executor filters far more rows with pushdown off" carries its
  counts and leaves lossy_arms.tsv.

Measured, PG17, shell and twin:
- novecpreds: 3 red; the old shell reddened 2 and passed the #726 check.
- plan726 (plain EXPLAIN of the ungrouped node reports no pushed-down
  filters): 2 red. Each red set is identical in both twins.
- clean: 51 of 51 on PG15 to PG19 in both twins.

Ledger: 51 rows, 5 dated by the two mutations and 46 never.
suites_not_covered 120 -> 119, cluster_tests 2260 -> 2311, TESTS.md
section 249. Census 3682 -> 3713 never, 1989 -> 2009 dated, 5722 rows. The ledger
gate and the guards pass over 4855564.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…isted no longer passes its comparisons (#1341)

test/pytest/test_row_triggers.py ports all 14 checks of test/row_triggers.sh
by name. Two kinds of shell check could pass with nothing measured; both
are fixed in both twins.

- The seven comparisons of a columnar table's trigger log with its heap
  twin's read "0/0/0" = "0/0/0" when neither trigger existed. With every
  pair's triggers created against a missing function, all of them passed.
  same_as_heap() now reads "no reading" unless the heap trigger fired,
  and the names are literal at each call site.
- The two "succeeds" checks counted the message "failed to fetch", so any
  other error -- a missing table's included -- read as success. They now
  read the first ERROR line, or "succeeded".

Measured, PG17, shell and twin:
- every pair's triggers against a missing function: 7 red; the old shell
  reddened 1 ("the row itself landed") and passed the rest.
- nobuffer (tuple_fetch_row_version without the write-buffer fallback,
  #179): 10 red. nodisk (only the write buffer): 10 red, a different ten.
  Each red set is identical in both twins.
- clean: 14 of 14 on PG15 to PG19 in both twins.

Ledger: 14 rows, 13 dated by the two mutations and 1 never.
suites_not_covered 119 -> 118, cluster_tests 2311 -> 2325, TESTS.md
section 250. Census 3713 -> 3714 never, 2009 -> 2022 dated, 5736 rows. The ledger
gate and the guards pass over 44d3113.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Re-approving the stacked head 6e3609a0 (I approved a668381f).

  • Content unchanged: compared file by file against a668381f, the PR's own +/- lines match. That covers every file except the budget, expected_tests and TESTS.md, which only restate counts; check_ledger.tsv and lossy_arms.tsv are identical.
  • Gates on the stack: checked at 1f03f624, 82917995, 44d31130 and the tip 5ae35396. Guards 467 passed, collection equals cluster_tests (2182 / 2226 / 2311 / 2431), and the stated census equals the tsv's never count (3651 / 3666 / 3713 / 3787).

🤖 Generated with Claude Code

@jdatcmd
jdatcmd merged commit 908ebb7 into main Oct 21, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants