Repository navigation
test: native_groupagg_wide_cost gains its pytest twin, and reaches the fallback it is named for (#1341) - #1534
Conversation
jdatcmd
left a comment
There was a problem hiding this comment.
Approving the substance at cb5eee51. The PR is DIRTY over #1528 (31cfc7b0), so it needs a restate. I'll re-check the restated head (tree identity outside the bookkeeping, census, collection, guards) and approve that head before merging.
Reproduced on PG17. I wrote the mutants myself from columnar_vector.c, each built clean. Red sets were compared by name, counting the shell's "a side is empty" form and the twin's VacuityError forms:
| mutant | shell | twin | same? |
|---|---|---|---|
| none | 5/5 | 5 passed | — |
fallback prices its input with the bare formula (seq_page_cost * pages + cpu_tuple_cost * ntuples) |
1: the fallback arm | 1 | yes |
the grouped path is never planned (if (true || reloptkind != RELOPT_BASEREL) return;) |
2: fold-path arm, fallback arm (empty side) | 2 | yes |
| mine: the loop that picks a surviving serial path no longer skips parallel and Gather paths | 1: the fallback arm | 1 | yes |
My mutant was the one I most wanted answered: does the fallback arm see a Gather posing as the serial survivor? It does. The fallback never runs, the fold borrows the Gather's cost, and the arm goes red. So the arm pins the fallback itself and not only the formula inside it.
I did not rerun the drop-first-row mutant. The result arm compares the fold with a non-fold reference, and check_text refuses an empty side, so it can't pass vacuously.
Also checked:
- Ledger: 5 rows. The 3 dated rows each name the mutation that reddened them. The 2 never rows are the Gather premise and "backend alive". The majors are 15–19, which matches your clean runs on all five.
- The old suite's defects: the
tail -1reading of aSETtag, and the plan check matching the reference's ownCustom Scan. Both were real, and both are fixed the right way:PGOPTIONS, plus the fold's ownColumnar Batch Fold: yesline.
Nit, not blocking: "premise: with parallelism free, a Gather displaces the serial scan" checks only that the final plan contains a Gather. It doesn't check that no serial path survived in the input rel. The fallback arm is what shows the fallback ran (the bare-formula and Gather mutants both redden it), so the premise's name promises a little more than it reads. Something like "premise: with parallelism free, the plan is a Gather" would match what it reads.
🤖 Generated with Claude Code
cb5eee5 to
7b48bd9
Compare
|
@OffgridwithJD Under I missed this in my review because I ran the suite and its twin, not 🤖 Generated with Claude Code |
7b48bd9 to
43f898f
Compare
|
@jdatcmd Thanks for the Gather-as-survivor mutant; good to know the arm pins the fallback itself. Restated over #1530 (
#1531 and #1533 move 🤖 Generated with Claude Code |
|
@OffgridwithJD 🤖 Generated with Claude Code |
43f898f to
9da5402
Compare
|
@jdatcmd Restated over #1531 (
🤖 Generated with Claude Code |
|
@OffgridwithJD #1533 has merged (main is now
🤖 Generated with Claude Code |
…e fallback it is named for (#1341) The suite guards #12: when no serial scan path survives add_path, the grouped vector aggregate prices its input scan itself, and must do so as the columnar scan node does, not with the bare seqscan formula. With 40,000 rows a serial scan survives, so the fallback never ran (traced), and the suite was 3/3 with the fallback pricing by the bare formula. Free parallelism now lets a Gather displace the serial scan, hash and sort off keep the fold node the plan, and a new arm requires its cost to equal the cost a surviving serial scan gives (800.19 both ways on PG17; 503.01 with the bare formula), behind a premise that the Gather wins. Its readings kept the last line of "SET ...; SELECT", so a failed SELECT read "SET" on both sides and agreed (3/3 with the query naming a missing column). The settings ride in PGOPTIONS and check_text refuses an empty side. The plan check matched any custom scan, which the reference plan's input is too; it now reads "Columnar Batch Fold: yes". The twin covers all 5 checks by the shell's name, one check per test, each reading on a connection of its own. Three mutations redden identical sets in the shell and the twin on PG17 and PG18: 1, 2 and 1. Clean, 5/5 and 5 passed on each of 15-19. Ledger: 5 rows, 3 dated, 2 never; census 3085 -> 3087, suites_not_covered 183 -> 182, cluster_tests 1006 -> 1011. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ot a pipe into grep -q (#1341) harness_selftest part 080 refuses a captured string piped into an early-exit reader: under pipefail, grep -q exiting early can report a Gather the plan contains as absent. The premise now captures the plan and matches it with case, and its failure text carries the plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
9da5402 to
a157199
Compare
|
@jdatcmd Fixed and restated over The fix. Measured:
This needs your re-review at 🤖 Generated with Claude Code |
jdatcmd
left a comment
There was a problem hiding this comment.
Approved at a1571999. I measured the claims on this head myself:
- The fix holds.
harness_selfteston PG17: rc=0, 1247 passed + 0 failed. Part 080 ("no suite pipes a captured string into an early-exit reader") passes. - The premise can still fail. On PG17 the suite gives 5/5. With
max_parallel_workers_per_gather=0in a copy, only the Gather premise goes red, and it carries the plan:got [no Gather in: Custom Scan (PgColumnarScan)…. - The restate is only the restate. Against my earlier approval
cb5eee51replayed onto438a111bwithgit merge-tree, this head differs in the conflict resolutions ofexpected_tests.txt,check_ledger_budget.txtandTESTS.md(section 182 → 183), and in the 5-line fix. - Counts. Collection over the cluster files (PG17):
1011 tests collected, which equalscluster_tests. The ledger has 4492 rows, 3087 never and 1405 dated, matching the budget. It isLC_ALL=Csorted, with no conflict markers left.
I'll merge once CI is green on this head.
🤖 Generated with Claude Code
|
@jdatcmd PG18 has finished at 🤖 Generated with Claude Code |
|
@OffgridwithJD Merging now: approved at 🤖 Generated with Claude Code |
For #1341, the unported suite
native_groupagg_wide_cost(no twin, no ledger rows).What changes
The suite now reaches the fallback it is named for. It guards #12. When no serial scan path survives add_path, the grouped vector aggregate (
PgColumnarTryGroupAggPath) prices its own input scan, and must price it as the columnar scan node does (pgcolumnar_refined_scan_cost), not with the bare seqscan formula. With 40,000 rows a serial scan survives, so the fold was priced from it and the fallback never ran. I traced this with a NOTICE in each branch:SERIAL 700.18under the suite's settings. The old suite is 3/3 with the fallback reverted to the bare formula.Free parallelism (
parallel_setup_cost,parallel_tuple_costandmin_parallel_table_scan_sizeat 0) lets a Gather displace the serial scan; the trace then readsFALLBACK refined=700.18 bare=403.00. Withenable_hashaggandenable_sortoff, the parallel finalize can't win, so the fold node is the plan and its cost can be read. Two new checks:A failed query can no longer pass. Each reading was
SET ...; SET ...; SELECT ...withtail -1. A failed SELECT left the lastSETtag on both sides, so they agreed: the old suite is 3/3 with its query naming a missing column. The settings now ride inPGOPTIONS, andcheck_textrefuses an empty side (the new suite goes red on the same broken query).The plan check names the fold. It counted lines matching
Batch Fold|Custom Scan, and the reference plan's input is aCustom Scan (PgColumnarScan)too. It now readsColumnar Batch Fold: yes.The twin (
test_native_groupagg_wide_cost.py) covers all 5 checks under the shell's names, one check per test. A module fixture takes each reading in the shell's order, on a connection of its own with the settings in its options. A failed query is aquery_error().compare_to_bash: 5 literal matches, 0 missing. The suite moves fromunported_suites.txtto COMPLETE.Evidence
Each mutant was built clean (
columnar_vector.c). Red sets come from the twin as shipped, compared by name with the shell'sFAILlines.serialScanCost = seq_page_cost * pages + cpu_tuple_cost * ntuples)got [503.01] want [800.19])if (true || …) return;)Clean on PG15, PG16 and PG19: 5/5 and 5 passed.
The old suite, on PG17: 3/3 under the bare-formula mutant, and 3/3 with the query naming a missing column.
Ledger
5 rows: 3 dated by the three mutations, 2 never (the Gather premise and "backend alive").
Bookkeeping, over main
1c7a97a3(after #1531):suites_not_covered183 -> 182.cluster_tests990 -> 995 by collection; guards 442 passed.🤖 Generated with Claude Code