Repository navigation
test: parallel_export_parquet gains its pytest twin, and an export that loses a row no longer passes (#1341) - #1678
Merged
Conversation
jdatcmd
force-pushed
the
test/1341-parallel-export-parquet-twin
branch
from
October 22, 2026 03:29
d20439e to
490358a
Compare
Collaborator
Author
OffgridwithJD
approved these changes
Oct 22, 2026
OffgridwithJD
left a comment
OffgridwithJD
left a comment
Collaborator
There was a problem hiding this comment.
@jdatcmd Approving 490358aa. I measured d20439e2. 490358aa is its restate onto #1673. The file set is the same, and the PR's own added lines in parallel_export_parquet.sh (+134), test_parallel_export_parquet.py (+416), check_ledger.tsv (+54) and test_compare_to_bash.py (+1) are identical. The budget, TESTS.md and expected_tests differ only in numbers. Gates at 490358aa: guards 476 passed, collection 2904 = cluster_tests, census 3923 = the tsv's never count, suites_not_covered 98.
Measured on PG17, ~/pg/17-u, shell then twin as shipped (per mutant make clean):
| run | main's shell | this shell | twin |
|---|---|---|---|
| head | — | 54/54 (PG18 too) | 54 passed (PG18 too) |
M1 (yours): tuple_insert drops the row whose first column is 1 |
54/54 pass | 16 red | SAME 16 |
| M2 (yours): the non-columnar target is not refused | 54/54 pass | 1 red | SAME 1 |
dup1 (mine, over-apply): tuple_insert writes the id-1 row twice |
1 red | 15 red | SAME 15 |
expskip (mine): PgColumnarWriteParquetFile never writes row number 5 of each file |
16 red | 16 red | SAME 16 |
harness: server stopped (pg_ctl -m immediate) just before the length guard, in a temporary copy of the twin |
— | — | 11 failed, the same 11 your table gives for H1's shell: the 984-byte and 985-byte boundaries, the cleanup-scan window, the non-empty directory, the non-columnar target and survival among them. The one ERROR is the fixture teardown's DROP DATABASE on the dead server |
dup1 shows the generator oracle's value: main's shell caught a duplicated row in only 1 check, this head in 15, in both harnesses.
Static:
- every oracle hashes the generator (
GEN_COL,GEN_PART,GEN_TX_COMMITTED), never the table under test; _capability_import_sitesprints[];- failed readings come back as
query_error()values; - a refusal needs a SQLSTATE and its own message, and a connection failure is a failed reading, not "accepted".
🤖 Generated with Claude Code
87 tasks done
…1339) Fixes #1339 test_projection_drop_column.py:113 and :119, test_projection_parallel.py:217, test_projection_privilege.py:259, test_projection_scan_io.py:260 and test_projections.py:459 and :600, each in its shell twin too. For each finding, a one-line mutation of pgcolumnar was first run on main, where both twins stayed green (or reddened the wrong name). After the fix, the same mutation reddens the target by the same name in both twins. The pytest twin stops at a test's first red, so it can show fewer names than the shell suite. - #1339 test_projection_drop_column.py:113 (weak). The stranger arms read only 42501, so deleting the unlocked ownership check in columnar_tableam.c passed (main: 8/8 shell, twin green). New arm in both: while the owner holds SHARE, a stranger is refused at once. Under the mutation it waits out lock_timeout: "a non-owner is refused before it waits for the table lock" got 55P03. - #1339 test_projection_drop_column.py:119 (mislabelled, twin mismatch). "premise: and the two refusals are the same error" came after the two constant arms, so it could never run red; the shell suite did not have it. With both ownership checks deleted, main's twin reddened "a non-owner learns nothing". The check loses the prefix, moves first, and joins the shell suite. Under that mutation both twins now redden "and the two refusals are the same error, so neither leaks" (2BP01 against 42501). - #1339 test_projection_parallel.py:217 (mislabelled). With parallel_workers = 1 on the projection path, main reddened "premise: EXPLAIN ANALYZE launched two workers" in both twins (and the rows= premise in the shell suite). Both lose the prefix; the same mutation reddens them under their new names. - #1339 test_projection_privilege.py:259 (twin mismatch). The twin asserted 42501 (and 0A000 under the policy) twice on one error. With read_projection's aclcheck_error replaced by elog, main's shell reddened "and THAT refusal is the C check" and the twin reddened "a role with EXECUTE but no SELECT is refused read_projection". The twin's first arm now asks only "refused", as the shell does. The shell's reconstruct arms now read the SQLSTATE, as the twin did: main's shell stayed green with reconstruct's aclcheck_error replaced. Under each of the three mutations (both elogs, and the RLS refusal raised as 42501), the two twins now redden identical names. - #1339 test_projection_scan_io.py:260 (weak). With the one-page floor deleted, an empty projection is priced at 0 pages and is still chosen (main green in both). New arm in both: "and its scan is not priced at zero pages" (PG17: total cost 5.00, and 0.00 under the mutation). - #1339 test_projections.py:459 (weak). With the executor never opening the projection while EXPLAIN still names it, main was green in both. New arm in both reads EXPLAIN ANALYZE: the filter sees fewer rows than the base holds (5980 against 17980 on PG17; 17980 of 17980 under the mutation). - #1339 test_projections.py:600 (inaccurate). EXPLAIN shows that the count plans pmp (Columnar Projection: pmp), so the triage's guess that it reads the base did not hold. But nothing asserted it. With projection pages multiplied by 1000000, so the count plans the base, every :600 arm stayed green on main. New arm in both: "the old-snapshot count is planned as a projection scan". The triage's candidate mutation (GetLatestSnapshot() for the projection read only) stays green: the base liveness cache still uses the query snapshot. With both on GetLatestSnapshot(), main's shell reddened "old snapshot projection scan does not see post-snapshot rows" but the twin reddened "session A responded post-commit". The twin's two timeout arms now read whether session A answered. The shell's poll no longer accepts an ERROR that echoes the token as an answer: with session A unable to see pm, main's shell reddened the counts but never "session A opened snapshot". It does now, in both twins. Ledger: projection_drop_column is seeded (11 rows; suites_not_covered 104 -> 103). 19 rows are dated by ten mutations, 2 are renamed, and 14 are new. checks_never_observed_red 3868 -> 3863 (6141 rows = 3863 never + 2278 dated). Clean on 15-19 for every touched suite and twin. Gates: the ledger gate passes for all five suites, compare_to_bash finds every bash property covered for all five pairs, and the guard set passes (476 passed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… stale stored digest no longer passes (#1339) #1339 listed "premise: a tdigest stored in a columnar table still answers" as mislabelled: nothing compared the stored digests, so a pgColumnar storage change could redden that premise alone, or pass it. hll's stored sketch has had a byte-for-byte arm; tdigest had none. New arm, in both harnesses: "a tdigest stored in a columnar table comes back byte for byte". The daily digests are built once into a heap table and stored from there, because a tdigest depends on the order its inputs arrive in; digests rebuilt from heap could differ for reasons that are not storage. The union-within-0.1% check keeps its premise label: a storage change that altered a digest now reddens the arm first, the suite's rule for its bounds. Measured on PG17: - Mutant: a reader returning each long value (over 512 bytes) as the previous long value read (columnar_reader.c). Main's suite: 11 + 1 failed, only "an hll stored in a columnar table comes back byte for byte"; every tdigest check passed. This branch: 11 + 2 failed, the new arm "got [f] want [t]" and the hll arm; the twin fails the same two names (read from its own records). - Rejected mutants: flipping the last byte of a long value, or zeroing its second half, makes tdigest's output function take the server down, so every later arm fails as collateral. Not used for dating. - Clean: shell 13/13 on PG15 to PG18, 7 + 6 skipped on PG19 (no hll there); the twin passes on each; the new arm passes on all five. - compare_to_bash: 15 literal matches, 0 missing. Guards 474 passed. Ledger: 6141 + 1 = 6142 rows, 3863 never (unchanged), 2278 -> 2279 dated. suites_not_covered and cluster_tests unchanged. Stacked on #1674; the ledger gate passes, and guards 476 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the file's first read no longer passes (#1341) test/pytest/test_parallel_copy_dedup.py ports all 17 checks of test/parallel_copy_dedup.sh by name. Two properties the shell's header promises could break with main's shell still green; both are fixed in both twins. - The changed file differed from the first in its first bytes, and the whole file fitted in the coordinator's first 256 KiB read. With the fingerprint taken from the first read alone, main's shell passed 14 of 14. The changed file now differs only in its last row, 25,000 rows put that row past the first read (a premise checks this), and a new check, "the records hold the SHA-256 of each loaded file's bytes", compares the table's records with the server's sha256() of each file. - The header says a refused load "says what it skipped rather than returning a silent 0", but no check read the NOTICE. With the NOTICE removed, main's shell passed 14 of 14. A new check reads it. - "premise: without dedup a repeated load doubles the table, ..." and "premise: and each of those loads reported its rows" lose their prefix under #1339: dedup is pgcolumnar's own default, and forcing it on reddens both. Measured, PG17, shell and twin: - fingerprint of the first read alone: 4 red; main's shell 0. - the skip's NOTICE never raised: 1 red; main's shell 0. - dedup forced on: 4 red. The refused load committed anyway: 2 red. A fingerprint matched in any table: 3 red. The refused load reporting the rows it discarded: 1 red. Each red set is identical in both twins. - the fixture shrunk to 20,000 rows: 1 red, the size premise. - clean: 17 of 17 on PG15 to PG19 in both twins. Ledger: 17 rows, 13 dated by the 6 mutations and 4 never. suites_not_covered 74 -> 73, cluster_tests 3222 -> 3239, TESTS.md section 296. Census 4132 -> 4136 never, 2818 -> 2831 dated, 6967 rows. The ledger gate and the guards pass over eb4aa69. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… no longer passes as one that survived (#1341) test/pytest/test_parallel_copy.py ports all 81 checks of test/parallel_copy.sh by name. Ten shell checks could pass with the property broken; all are fixed in both twins. - Five survival checks read "SELECT 1", which answers again after crash recovery. With one backend killed by SIGSEGV before each, main's shell passed 81 of 81. They now also need the server log's count of killed backends unmoved since the suite began (crashes() and survived(), from avro_manifest.sh). - "atomic: a loader failure fails the whole load" and "single table: a bad row fails the whole load" took any message with "failed" in it, and every worker failure reads "worker N failed: ...". With the loader's message dropped, main's shell passed 81 of 81. They now need the loader's own cause after "failed: ". - The nine reconstruct checks compared the per-range loads with a single columnar COPY of the same file, which shares every defect of the COPY path. With multi_insert dropping the row whose id is 1, all nine passed on main. They now compare with the generator that wrote the file. - Check names built from loop variables and helper labels are literal at the call site, so the three suite-local check_ helpers are gone. test_compare_to_bash.py asserted those three existed; that arm now asserts no suite defines its own check_ helper, and it reddens on main's parallel_copy.sh. Measured, PG17, shell and twin, each red set identical in both: - a backend killed before each survival check: 5 red; main's shell 0. - the loader's message dropped: 2 red; main's shell 0. - multi_insert dropping id 1: 19 red, 9 of them reconstruct checks; main's shell 10. - a split offset on the newline: 23. The split degraded to one range: 5. The S_ISREG refusal removed: 2. The guard on max_prepared_transactions removed, bulk_parallel_writer booting on, the FIFO opener blocking, workers < 1 accepted, the key-type restriction removed: 1 each. - clean: 81 of 81 on PG15 to PG19 in both twins. Ledger: 81 rows, 49 dated by the 11 mutations and 32 never. suites_not_covered 73 -> 72, cluster_tests 3239 -> 3320, TESTS.md section 297. Census 4136 -> 4168 never, 2831 -> 2880 dated, 7048 rows. The ledger gate and the guards pass over 7e8c10f. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erially no longer passes (#1341) test/pytest/test_parallel.py ports all 13 checks of test/parallel.sh by name. Three shell checks could pass with the property broken; each is fixed in both twins, and each fix adds one named check. - "par no-filter" was a bare count(*). The vectorized aggregate answers that with no Gather, so the check ran serially. With every participant reading every row group, main's shell reddened the other four par checks and left this one green. It now returns rows, and "par queries all plan a Gather" holds all five par queries to a Gather. - The table was assumed to have 25 row groups. With the writer ignoring stripe_row_limit it had 1, and a claim that skips every other group passed main's shell 10 of 10. "par table has 25 row groups to claim" now counts them. - setg swallows its error, so a serial setting that did not take ran the three serial checks in parallel, and main's shell passed 10 of 10. "serial queries plan no Gather" now says so. The queries are named once, so the plan checks read the queries the comparisons ran. Measured, PG17, shell and twin: - every participant reads every row group: 5 red (the par checks); main's shell 4, without "par no-filter". - the writer ignores stripe_row_limit: 1 red (the row-group count); main's shell 0. With a claim that skips every other group as well: the same 1 red; main's shell 0. - the serial setting misspelled: 1 red (the serial plan check); main's shell 0. "par no-filter" put back to count(*): 1 red (the par plan check). - a claim that skips every other group: 5 red. The partial path never added: 2 red. The scan named ColumnarScan, as before #428: 3 red. A serial scan reading every other row group: 3 red. Each red set is identical in both twins. - clean: 13 of 13 on PG15 to PG19 in both twins. Ledger: 13 rows, 13 dated by the 5 mutations and 0 never. suites_not_covered 72 -> 71, cluster_tests 3320 -> 3333, TESTS.md section 298. Census 4168 -> 4168 never, 2880 -> 2893 dated, 7061 rows. The ledger gate and the guards pass over 9ae8951. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…no longer agree (#1341) test/pytest/test_parallel_vector_agg.py ports all 29 checks of test/parallel_vector_agg.sh by name. Four kinds of shell check could pass with the property broken; all are fixed in both twins. - With the server stopped after the first table, main's shell passed 11 of 29: each two-arm check compared psql's "Connection refused" text with the same text from the other arm. q now reads psql's exit status (ON_ERROR_STOP) and prints nothing on failure, and every two-arm comparison is check_text. With the edit, the fixed shell reds all 29. - The two H2 checks read rows out of one session and ignored its exit status. With both parallel InitializeDSM callbacks raising an ERROR in a transaction holding an XID, the transaction aborted, both arms read empty, and main's shell passed 29 of 29. Both checks now need psql to succeed and both SELECTs to answer. - The grouped "serial oracle" was pgcolumnar's own grouped fold run serially (pgcolumnar.enable_group_vectorization is on by default). With the grouped row path skipping its WHERE recheck, both arms agreed on the wrong answer and main's shell passed 29 of 29. The grouped oracles are now core's grouped aggregate. - "server still up" was SELECT 1, which answers after crash recovery. With a parallel worker dying on SIGSEGV, main's shell passed it. It now also needs the server log's "terminated by signal" count unmoved. - "the partial columnar agg node present" now needs the Parallel Custom Scan: the serial node prints the same aggregate line, so with the parallel partial path never added main's version passed. - Nine checks lose their premise: prefix under #1339. Each reads pgcolumnar's own plan or writes, and a one-line mutant reddens it. Measured, PG17, shell and twin: - both InitializeDSM callbacks raise an ERROR in a writing transaction: 2 red (both H2 checks); main's shell 0. - the grouped row path skips its WHERE recheck: 2 red; main's shell 0. - a grouped parallel worker SIGSEGVs in a writing transaction: 2 red (grouped H2, server still up); main's shell 1, "server still up", and only because SELECT 1 arrived during recovery. - the ungrouped parallel partial path never added: 4 red; main's shell 3. The grouped one: 3 red. Each worker with a private counter: 7 red ungrouped, 4 red grouped. The batch-fold gate always refusing: 2 red. An insert that drops one row: 1 red. Each red set is identical in both twins. - a worker that folded no row group SIGSEGVs: the shell 19 red, the twin 17, both including "server still up"; the rest is timing-dependent collateral, so the run is not in the ledger. - clean: 29 of 29 on PG15 to PG19 in both twins. Ledger: 29 rows, 21 dated by the nine mutations and 8 never. suites_not_covered 71 -> 70, cluster_tests 3333 -> 3362, TESTS.md section 299. Census 4168 -> 4176 never, 2893 -> 2914 dated, 7090 rows. The ledger gate and the guards pass over f953f5a. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…at loses a row no longer passes (#1341) test/pytest/test_parallel_export_parquet.py ports all 54 checks of test/parallel_export_parquet.sh by name. The shell's two loops are unrolled, so every name is literal; the twin parametrizes nine functions over literal tables of the shell's names. Four holes are fixed in both twins. - The source hashes and counts read the columnar table being exported. With tuple_insert dropping the row whose id is 1, the table, its serial export and every parallel export agreed, and all 54 passed. They now read the generator that fills each table. - The two refusals grepped psql's output for "error", which a dead server's "psql: error: connection ... failed" also matches, and took any error as the refusal. With the non-columnar branch removed, a heap target failed on "could not read blocks" and the check passed. Each now reads psql's exit status and matches its own message. - run_sqlstate read a dead server as "none", so "a 984-byte destination still exports" passed with the server stopped, and the two "not created" checks read it as absent. Each now needs the statement to have reached the server. - "server still up" took SELECT 1, which answers after crash recovery. It now also needs the server log's count of backends killed by a signal unmoved since setup, as avro_manifest.sh does. Measured, PG17: - tuple_insert drops id 1: main's shell passed 54 of 54; 16 are red now. - the non-columnar refusal removed: main's shell passed; "reject a non-columnar target" is red now. - server stopped before the length guard, a harness edit: main's shell passed 5 checks on nothing; 11 are red now, the 5 among them. - one backend killed by SIGSEGV before the last check, a harness edit: main's shell passed; "server still up after the exports" is red now. - seven mutations, each reddening the same names in both twins: row dropped 16, non-columnar not refused 1, cancel cleanup removed 3, no _SUCCESS 8, length probe of the part name only 5, non-empty directory accepted 1, every worker writing part-0000 10. - clean: 54 of 54 on PG15 to PG19 in both twins. Ledger: 54 rows, 38 dated by the 7 mutations and 16 never. suites_not_covered 70 -> 69, cluster_tests 3362 -> 3416, TESTS.md section 300. Census 4176 -> 4192 never, 2914 -> 2952 dated, 7144 rows. The ledger gate and the guards pass over a04a5a7. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Restated again, onto #1673's new head (main 9ab6a4c underneath), because the registries conflicted with main. The registries are regenerated over that head by the same tooling as before; the suite and twin are unchanged; the ledger gate and the guards pass (see the commit message). Re-approval needed. Head a0fac05. |
jdatcmd
force-pushed
the
test/1341-parallel-export-parquet-twin
branch
from
October 22, 2026 19:11
490358a to
a0fac05
Compare
OffgridwithJD
approved these changes
Oct 22, 2026
OffgridwithJD
left a comment
OffgridwithJD
left a comment
Collaborator
There was a problem hiding this comment.
@jdatcmd Re-approved at a0fac054, on #1673's a04a5a79. The diff outside the bookkeeping hashes the same as the approved 490358aa's. The tsv is sorted, and its census (7144 / 4192 / 2952) matches the budget.
🤖 Generated with Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@OffgridwithJD, this ports
test/parallel_export_parquet.shtotest/pytest/test_parallel_export_parquet.py(#1341) and fixes four holes in the shell. Main's shell passed all 54 checks with an export that lost a row.What was fixed
All four fixes are in both the shell and the twin.
tuple_insertdropping the row whose id is 1, the table, its serial export and every parallel export agreed, and all 54 checks passed. The oracles now read the generator that fills each table.expect_errorgrepped psql's output for "error". A dead server'spsql: error: connection ... failedmatches that, and so does any unrelated error. With the non-columnar branch removed, a heap target failed oncould not read blocksand the check passed. Each refusal now reads psql's exit status and matches its own message.run_sqlstatereturned "none" (accepted) when psql could not connect. So "a 984-byte destination still exports" passed with the server stopped. The two "the destination was not created" checks also passed, because nothing could create the directory. Each check now needs the statement to have reached the server.SELECT 1as "still up". It answers after crash recovery. "server still up after the exports" now also needs the server log's count of backends killed by a signal to be unchanged since setup (crashes()/survived()fromtest/avro_manifest.sh).Check names that were built from loop variables are now literal: the loops are unrolled, and the length checks name 984, 985 and 994 bytes. No check was renamed or removed.
Each edit and mutant, by run
Every run is on PG17. "Red" counts FAIL records. Edits H1 and H2 change only the test harness. M1 to M7 are product mutants.
columnar_tableam.c:tuple_insertdrops the row whose first column is 1columnar_parallel_export.c: a non-columnar target is not refused (else if (true))_SUCCESSmarker is never writtenpart-0000.parquetIn H1, the 5 checks that main's shell passed on nothing are: "accepted boundary: a 984-byte destination still exports", "rejected boundary: the destination was not created", "cleanup-scan window: the destination was not created", "reject a non-empty output directory" and "reject a non-columnar target".
For each product mutant, the shell's FAIL names and the twin's failed assertion names were compared by script, and they are identical. None of the mutants crashed a backend.
Gates
All gates ran on the one commit over
f6794929:--registeredand--against origin/main(ceiling 103 -> 102). The orphan scan finds 0 orphans.test_harness_deps.py,test_capability_decline.pyand the twin, run with--pg-config: 84 passed.compare_to_bash.py: 54 literal matches, 0 missing.[]. The twin needs no pyarrow: the read-back usespgcolumnar.read_parquet.harness_selftest.shpasses, and so doesshellcheck -S error.cluster_tests2732 -> 2786, measured by collection.suites_not_covered103 -> 102. The TESTS.md section is 267.H2 is a harness edit, so it is not in the ledger, and "server still up after the exports" arrives as never.
Re-checked by me before pushing (PG17, main f679492): M2, the non-columnar refusal unreachable (
else if (PgColumnarIsColumnarRelation(relid))madeelse if (true)incolumnar_parallel_export.c): main's shell 54/54; this shell 1 failed, "reject a non-columnar target: got [another error: …could not read blocks 0..0…] want [refused]"; the twin fails the same test.🤖 Generated with Claude Code