Repository navigation
test: parallel_vector_agg gains its pytest twin, and two failed arms no longer agree (#1341) - #1673
Merged
Merged
Conversation
OffgridwithJD
approved these changes
Oct 22, 2026
OffgridwithJD
left a comment
OffgridwithJD
left a comment
Collaborator
There was a problem hiding this comment.
@jdatcmd Approving e112590e. Measured on ~/pg/17-w, plus a clean run on 18; mutants in ~/pgc-tools/rv1673/.
| run | main's shell (5a947572) |
this shell | twin |
|---|---|---|---|
| clean, PG17 / PG18 | — | 29/29 / 29/29 | 29 / 29 passed |
gnorecheck (yours: the grouped row path skips its WHERE recheck, pgcolumnar_groupagg_build) |
29/29: the hole | 2 red: grouped count+sum(int), grouped multi-key + WHERE | SAME 2 |
harness: server stopped right after t is built |
11 pass, 18 red | 29 red | — |
unopartial (the ungrouped Gather path not added) |
3 red, all premise: |
the same 3, relabelled | SAME 3 |
unorecheck (mine, drop: the ungrouped scan-fold skips its recheck) |
— | 2 red: "not-batch-foldable filter: count(*) exact", "…: all-null avg is NULL" | SAME 2 |
gallreject (mine, over-apply: the grouped recheck rejects every row) |
— | 2 red: the two grouped oracles | SAME 2 |
unopartial: my version removes the whole Gather path rather than only the partial, so main's premise-labelled checks catch it too. Your narrower mutant is the one the newParallel Custom Scanrequirement is for, and I didn't rebuild it.- Non-blocking: "not-batch-foldable filter: all-null avg is NULL" ships
never, butunorecheckreddens it in both twins, so it could be dated.
🤖 Generated with Claude Code
jdatcmd
force-pushed
the
test/1341-parallel-vector-agg-twin
branch
from
October 22, 2026 03:23
e112590 to
d73aa5c
Compare
OffgridwithJD
approved these changes
Oct 22, 2026
OffgridwithJD
left a comment
OffgridwithJD
left a comment
Collaborator
There was a problem hiding this comment.
@jdatcmd Re-approving the restated head d73aa5cd, stacked on #1672's d5f9ade1 (approved earlier at e112590e). The PR's added lines are identical in parallel_vector_agg.sh (114), test_parallel_vector_agg.py (305), check_ledger.tsv (29) and test_compare_to_bash.py (1). Only the budget and counters moved. The earlier measurements stand.
🤖 Generated with Claude Code
Collaborator
Author
87 tasks done
…1339) Fixes #1339 test_projection_drop_column.py:113 and :119, test_projection_parallel.py:217, test_projection_privilege.py:259, test_projection_scan_io.py:260 and test_projections.py:459 and :600, each in its shell twin too. For each finding, a one-line mutation of pgcolumnar was first run on main, where both twins stayed green (or reddened the wrong name). After the fix, the same mutation reddens the target by the same name in both twins. The pytest twin stops at a test's first red, so it can show fewer names than the shell suite. - #1339 test_projection_drop_column.py:113 (weak). The stranger arms read only 42501, so deleting the unlocked ownership check in columnar_tableam.c passed (main: 8/8 shell, twin green). New arm in both: while the owner holds SHARE, a stranger is refused at once. Under the mutation it waits out lock_timeout: "a non-owner is refused before it waits for the table lock" got 55P03. - #1339 test_projection_drop_column.py:119 (mislabelled, twin mismatch). "premise: and the two refusals are the same error" came after the two constant arms, so it could never run red; the shell suite did not have it. With both ownership checks deleted, main's twin reddened "a non-owner learns nothing". The check loses the prefix, moves first, and joins the shell suite. Under that mutation both twins now redden "and the two refusals are the same error, so neither leaks" (2BP01 against 42501). - #1339 test_projection_parallel.py:217 (mislabelled). With parallel_workers = 1 on the projection path, main reddened "premise: EXPLAIN ANALYZE launched two workers" in both twins (and the rows= premise in the shell suite). Both lose the prefix; the same mutation reddens them under their new names. - #1339 test_projection_privilege.py:259 (twin mismatch). The twin asserted 42501 (and 0A000 under the policy) twice on one error. With read_projection's aclcheck_error replaced by elog, main's shell reddened "and THAT refusal is the C check" and the twin reddened "a role with EXECUTE but no SELECT is refused read_projection". The twin's first arm now asks only "refused", as the shell does. The shell's reconstruct arms now read the SQLSTATE, as the twin did: main's shell stayed green with reconstruct's aclcheck_error replaced. Under each of the three mutations (both elogs, and the RLS refusal raised as 42501), the two twins now redden identical names. - #1339 test_projection_scan_io.py:260 (weak). With the one-page floor deleted, an empty projection is priced at 0 pages and is still chosen (main green in both). New arm in both: "and its scan is not priced at zero pages" (PG17: total cost 5.00, and 0.00 under the mutation). - #1339 test_projections.py:459 (weak). With the executor never opening the projection while EXPLAIN still names it, main was green in both. New arm in both reads EXPLAIN ANALYZE: the filter sees fewer rows than the base holds (5980 against 17980 on PG17; 17980 of 17980 under the mutation). - #1339 test_projections.py:600 (inaccurate). EXPLAIN shows that the count plans pmp (Columnar Projection: pmp), so the triage's guess that it reads the base did not hold. But nothing asserted it. With projection pages multiplied by 1000000, so the count plans the base, every :600 arm stayed green on main. New arm in both: "the old-snapshot count is planned as a projection scan". The triage's candidate mutation (GetLatestSnapshot() for the projection read only) stays green: the base liveness cache still uses the query snapshot. With both on GetLatestSnapshot(), main's shell reddened "old snapshot projection scan does not see post-snapshot rows" but the twin reddened "session A responded post-commit". The twin's two timeout arms now read whether session A answered. The shell's poll no longer accepts an ERROR that echoes the token as an answer: with session A unable to see pm, main's shell reddened the counts but never "session A opened snapshot". It does now, in both twins. Ledger: projection_drop_column is seeded (11 rows; suites_not_covered 104 -> 103). 19 rows are dated by ten mutations, 2 are renamed, and 14 are new. checks_never_observed_red 3868 -> 3863 (6141 rows = 3863 never + 2278 dated). Clean on 15-19 for every touched suite and twin. Gates: the ledger gate passes for all five suites, compare_to_bash finds every bash property covered for all five pairs, and the guard set passes (476 passed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… stale stored digest no longer passes (#1339) #1339 listed "premise: a tdigest stored in a columnar table still answers" as mislabelled: nothing compared the stored digests, so a pgColumnar storage change could redden that premise alone, or pass it. hll's stored sketch has had a byte-for-byte arm; tdigest had none. New arm, in both harnesses: "a tdigest stored in a columnar table comes back byte for byte". The daily digests are built once into a heap table and stored from there, because a tdigest depends on the order its inputs arrive in; digests rebuilt from heap could differ for reasons that are not storage. The union-within-0.1% check keeps its premise label: a storage change that altered a digest now reddens the arm first, the suite's rule for its bounds. Measured on PG17: - Mutant: a reader returning each long value (over 512 bytes) as the previous long value read (columnar_reader.c). Main's suite: 11 + 1 failed, only "an hll stored in a columnar table comes back byte for byte"; every tdigest check passed. This branch: 11 + 2 failed, the new arm "got [f] want [t]" and the hll arm; the twin fails the same two names (read from its own records). - Rejected mutants: flipping the last byte of a long value, or zeroing its second half, makes tdigest's output function take the server down, so every later arm fails as collateral. Not used for dating. - Clean: shell 13/13 on PG15 to PG18, 7 + 6 skipped on PG19 (no hll there); the twin passes on each; the new arm passes on all five. - compare_to_bash: 15 literal matches, 0 missing. Guards 474 passed. Ledger: 6141 + 1 = 6142 rows, 3863 never (unchanged), 2278 -> 2279 dated. suites_not_covered and cluster_tests unchanged. Stacked on #1674; the ledger gate passes, and guards 476 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the file's first read no longer passes (#1341) test/pytest/test_parallel_copy_dedup.py ports all 17 checks of test/parallel_copy_dedup.sh by name. Two properties the shell's header promises could break with main's shell still green; both are fixed in both twins. - The changed file differed from the first in its first bytes, and the whole file fitted in the coordinator's first 256 KiB read. With the fingerprint taken from the first read alone, main's shell passed 14 of 14. The changed file now differs only in its last row, 25,000 rows put that row past the first read (a premise checks this), and a new check, "the records hold the SHA-256 of each loaded file's bytes", compares the table's records with the server's sha256() of each file. - The header says a refused load "says what it skipped rather than returning a silent 0", but no check read the NOTICE. With the NOTICE removed, main's shell passed 14 of 14. A new check reads it. - "premise: without dedup a repeated load doubles the table, ..." and "premise: and each of those loads reported its rows" lose their prefix under #1339: dedup is pgcolumnar's own default, and forcing it on reddens both. Measured, PG17, shell and twin: - fingerprint of the first read alone: 4 red; main's shell 0. - the skip's NOTICE never raised: 1 red; main's shell 0. - dedup forced on: 4 red. The refused load committed anyway: 2 red. A fingerprint matched in any table: 3 red. The refused load reporting the rows it discarded: 1 red. Each red set is identical in both twins. - the fixture shrunk to 20,000 rows: 1 red, the size premise. - clean: 17 of 17 on PG15 to PG19 in both twins. Ledger: 17 rows, 13 dated by the 6 mutations and 4 never. suites_not_covered 74 -> 73, cluster_tests 3222 -> 3239, TESTS.md section 296. Census 4132 -> 4136 never, 2818 -> 2831 dated, 6967 rows. The ledger gate and the guards pass over eb4aa69. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… no longer passes as one that survived (#1341) test/pytest/test_parallel_copy.py ports all 81 checks of test/parallel_copy.sh by name. Ten shell checks could pass with the property broken; all are fixed in both twins. - Five survival checks read "SELECT 1", which answers again after crash recovery. With one backend killed by SIGSEGV before each, main's shell passed 81 of 81. They now also need the server log's count of killed backends unmoved since the suite began (crashes() and survived(), from avro_manifest.sh). - "atomic: a loader failure fails the whole load" and "single table: a bad row fails the whole load" took any message with "failed" in it, and every worker failure reads "worker N failed: ...". With the loader's message dropped, main's shell passed 81 of 81. They now need the loader's own cause after "failed: ". - The nine reconstruct checks compared the per-range loads with a single columnar COPY of the same file, which shares every defect of the COPY path. With multi_insert dropping the row whose id is 1, all nine passed on main. They now compare with the generator that wrote the file. - Check names built from loop variables and helper labels are literal at the call site, so the three suite-local check_ helpers are gone. test_compare_to_bash.py asserted those three existed; that arm now asserts no suite defines its own check_ helper, and it reddens on main's parallel_copy.sh. Measured, PG17, shell and twin, each red set identical in both: - a backend killed before each survival check: 5 red; main's shell 0. - the loader's message dropped: 2 red; main's shell 0. - multi_insert dropping id 1: 19 red, 9 of them reconstruct checks; main's shell 10. - a split offset on the newline: 23. The split degraded to one range: 5. The S_ISREG refusal removed: 2. The guard on max_prepared_transactions removed, bulk_parallel_writer booting on, the FIFO opener blocking, workers < 1 accepted, the key-type restriction removed: 1 each. - clean: 81 of 81 on PG15 to PG19 in both twins. Ledger: 81 rows, 49 dated by the 11 mutations and 32 never. suites_not_covered 73 -> 72, cluster_tests 3239 -> 3320, TESTS.md section 297. Census 4136 -> 4168 never, 2831 -> 2880 dated, 7048 rows. The ledger gate and the guards pass over 7e8c10f. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erially no longer passes (#1341) test/pytest/test_parallel.py ports all 13 checks of test/parallel.sh by name. Three shell checks could pass with the property broken; each is fixed in both twins, and each fix adds one named check. - "par no-filter" was a bare count(*). The vectorized aggregate answers that with no Gather, so the check ran serially. With every participant reading every row group, main's shell reddened the other four par checks and left this one green. It now returns rows, and "par queries all plan a Gather" holds all five par queries to a Gather. - The table was assumed to have 25 row groups. With the writer ignoring stripe_row_limit it had 1, and a claim that skips every other group passed main's shell 10 of 10. "par table has 25 row groups to claim" now counts them. - setg swallows its error, so a serial setting that did not take ran the three serial checks in parallel, and main's shell passed 10 of 10. "serial queries plan no Gather" now says so. The queries are named once, so the plan checks read the queries the comparisons ran. Measured, PG17, shell and twin: - every participant reads every row group: 5 red (the par checks); main's shell 4, without "par no-filter". - the writer ignores stripe_row_limit: 1 red (the row-group count); main's shell 0. With a claim that skips every other group as well: the same 1 red; main's shell 0. - the serial setting misspelled: 1 red (the serial plan check); main's shell 0. "par no-filter" put back to count(*): 1 red (the par plan check). - a claim that skips every other group: 5 red. The partial path never added: 2 red. The scan named ColumnarScan, as before #428: 3 red. A serial scan reading every other row group: 3 red. Each red set is identical in both twins. - clean: 13 of 13 on PG15 to PG19 in both twins. Ledger: 13 rows, 13 dated by the 5 mutations and 0 never. suites_not_covered 72 -> 71, cluster_tests 3320 -> 3333, TESTS.md section 298. Census 4168 -> 4168 never, 2880 -> 2893 dated, 7061 rows. The ledger gate and the guards pass over 9ae8951. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…no longer agree (#1341) test/pytest/test_parallel_vector_agg.py ports all 29 checks of test/parallel_vector_agg.sh by name. Four kinds of shell check could pass with the property broken; all are fixed in both twins. - With the server stopped after the first table, main's shell passed 11 of 29: each two-arm check compared psql's "Connection refused" text with the same text from the other arm. q now reads psql's exit status (ON_ERROR_STOP) and prints nothing on failure, and every two-arm comparison is check_text. With the edit, the fixed shell reds all 29. - The two H2 checks read rows out of one session and ignored its exit status. With both parallel InitializeDSM callbacks raising an ERROR in a transaction holding an XID, the transaction aborted, both arms read empty, and main's shell passed 29 of 29. Both checks now need psql to succeed and both SELECTs to answer. - The grouped "serial oracle" was pgcolumnar's own grouped fold run serially (pgcolumnar.enable_group_vectorization is on by default). With the grouped row path skipping its WHERE recheck, both arms agreed on the wrong answer and main's shell passed 29 of 29. The grouped oracles are now core's grouped aggregate. - "server still up" was SELECT 1, which answers after crash recovery. With a parallel worker dying on SIGSEGV, main's shell passed it. It now also needs the server log's "terminated by signal" count unmoved. - "the partial columnar agg node present" now needs the Parallel Custom Scan: the serial node prints the same aggregate line, so with the parallel partial path never added main's version passed. - Nine checks lose their premise: prefix under #1339. Each reads pgcolumnar's own plan or writes, and a one-line mutant reddens it. Measured, PG17, shell and twin: - both InitializeDSM callbacks raise an ERROR in a writing transaction: 2 red (both H2 checks); main's shell 0. - the grouped row path skips its WHERE recheck: 2 red; main's shell 0. - a grouped parallel worker SIGSEGVs in a writing transaction: 2 red (grouped H2, server still up); main's shell 1, "server still up", and only because SELECT 1 arrived during recovery. - the ungrouped parallel partial path never added: 4 red; main's shell 3. The grouped one: 3 red. Each worker with a private counter: 7 red ungrouped, 4 red grouped. The batch-fold gate always refusing: 2 red. An insert that drops one row: 1 red. Each red set is identical in both twins. - a worker that folded no row group SIGSEGVs: the shell 19 red, the twin 17, both including "server still up"; the rest is timing-dependent collateral, so the run is not in the ledger. - clean: 29 of 29 on PG15 to PG19 in both twins. Ledger: 29 rows, 21 dated by the nine mutations and 8 never. suites_not_covered 71 -> 70, cluster_tests 3333 -> 3362, TESTS.md section 299. Census 4168 -> 4176 never, 2893 -> 2914 dated, 7090 rows. The ledger gate and the guards pass over f953f5a. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Restated again, onto #1672's new head (main 9ab6a4c underneath), because the registries conflicted with main. The registries are regenerated over that head by the same tooling as before; the suite and twin are unchanged; the ledger gate and the guards pass (see the commit message). Re-approval needed. Head a04a5a7. |
jdatcmd
force-pushed
the
test/1341-parallel-vector-agg-twin
branch
from
October 22, 2026 19:06
d73aa5c to
a04a5a7
Compare
OffgridwithJD
approved these changes
Oct 22, 2026
OffgridwithJD
left a comment
OffgridwithJD
left a comment
Collaborator
There was a problem hiding this comment.
@jdatcmd Re-approved at a04a5a79, on #1672's f953f5a6. The diff outside the bookkeeping hashes the same as the approved d73aa5cd's. The tsv is sorted, and its census (7090 / 4176 / 2914) matches the budget.
🤖 Generated with Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@OffgridwithJD, this is the pytest twin of
test/parallel_vector_agg.sh(#1341). It also fixes the shell checks that could pass with the property broken. One commit,e112590e, on5a947572.This base is behind main.
origin/mainmoved tof0b43802while I worked, and #1638 (the iceberg_malformed twin) also changedunported_suites.txtand the ledger registries. Restate this commit onto main and regenerate the registries before you push. The counts below are from5a947572.What was fixed
Connection refusedtext with the same text from the other arm. Nowqreads psql's exit status (withON_ERROR_STOP) and prints nothing when psql fails, and every two-arm comparison usescheck_text. With the same edit, the fixed shell fails all 29 checks.InitializeDSMcallbacks raise an ERROR in a transaction that holds an XID. That aborted the transaction, both arms read empty, and main's shell passed 29 of 29. Both H2 checks now need psql to succeed and both SELECTs to answer.pgcolumnar.enable_group_vectorizationis on by default, so the "serial" arm ran the grouped node serially. In my mutant, the grouped row path skips its WHERE recheck. Both arms then agreed on the wrong answer, and main's shell passed 29 of 29. The grouped oracles are now core's grouped aggregate.SELECT 1. It answers again after crash recovery. With a parallel worker dying on SIGSEGV, main's shell passed this check. The check now also needs the server log'sterminated by signalcount to stay unchanged. I copied that logic fromavro_manifest.sh.Parallel Custom Scan (PgColumnarScan). The serial node prints the same aggregate line, so with the parallel partial path never added, main's version passed.premise:prefix under pytest audit: 260 confirmed checks that cannot fail, prove something else, or carry the wrong label #1339. Each one reads pgcolumnar's own plan or writes, and a one-line mutant makes it fail (see the table).Main's shell, this shell and the twin under each edit and mutant (PG17)
t(twin: its database dropped)columnar_vector.c: both parallel InitializeDSM callbacks raise ERROR in a transaction holding an XIDcolumnar_vector.c: the grouped row path skips its WHERE recheckcolumnar_vector.c: a grouped parallel worker SIGSEGVs in a transaction holding an XIDcolumnar_vector.c: a parallel worker that folded no row group SIGSEGVscolumnar_vector.c: the ungrouped parallel partial path never addedcolumnar_vector.c: the grouped parallel partial path never addedcolumnar_vector.c: each ungrouped worker claims through a private countercolumnar_vector.c: each grouped worker claims through a private countercolumnar_vector.c: the batch-fold shape gate always refusescolumnar_tableam.c:tuple_insertdrops the row whose first column is 1The crash mutants:
I also tried a mutant where the grouped batch fold ignores the delete mask. It failed nothing in main's shell or in this shell (I did not run the twin under it), because every grouped query here has a text key or an expression key, so the batch fold never runs. This suite does not cover that fold. It is not in the ledger.
Gates (over
5a947572)census stated 3861, ledger holds 3861: they agree,ceiling against origin/main ...: 105 -> 104, which does not rise.474 passed.test_harness_deps.py,test_capability_decline.pyand the twin with--pg-config(PG17):59 passed._capability_import_sitesprints[].compare_to_bash.py:VERDICT: every bash property is covered(29 literal matches).cluster_tests 2706(was 2677).harness_selftest.sh:PASSED.unported_suites.txtloses the suite.Logs
The logs are in
/tmp/claude-1006/-home-jdatcmd/ed97498f-58ee-4453-98bc-cbf3cfaf41b1/scratchpad/portlogs/parallel_vector_agg/:mainshell-clean-pg17.log,mainshell-deadserver.logandmainshell-<mutant>.log.shell-<mutant>.log,shell-deadserver.log, and the five clean logs merged into the ledger,shell-clean-pg15.logtoshell-clean-pg19.log.twin-<mutant>.log,twin-deadserver.logandtwin-clean-pg15.logtotwin-clean-pg19.log.gate-pg*.log,guards.log,deps-decline.log,compare.log,selftest.logandmerge-*.txt.mut-<mutant>.patch,edit-deadserver.patchandedit-deadserver-twin.patch.Re-checked by me before pushing (PG17, main f0b4380), with the agent's own patches:
mut-gnorecheck: main's shell 29/29; this shell 2 failed, the two grouped "parallel fold == serial oracle (#349)" checks; the twin the same 2 tests.mut-nopartial: main 3 failed underpremise:names; this shell 4, the same three relabelled plus "the partial columnar agg node present"; the twin the same 4.A gap this port found, not closed here: the grouped batch fold ignoring the delete mask fails nothing, in main's suite or this one, because every grouped query in the suite has a text or expression key and never reaches that fold. I'll file it.
🤖 Generated with Claude Code