Repository navigation
fix: an ungrouped aggregate applies a WHERE that references no column (#1435) - #1436
Conversation
The encoder has fast paths for fixed widths 1, 2, 4 and 8, and every other width and every varlena takes a generic path. A new suite, type_matrix, and its pytest twin store what no test had: - by-reference fixed widths 6, 8, 12, 16 and 64: macaddr, macaddr8, timetz, point, name; - the 1-byte "char"; - time, money, inet, cidr, bit, varbit, tsvector, xml, json; - an enum, a composite, a domain; - numeric NaN and +-Infinity, timestamp and date +-infinity. 12000 rows in row groups of 1000, each column NULL on its own pattern, are generated once into heap and copied. Against heap: every row as text, each column's counts, a range on all sixteen ordered columns with literal bounds so zone maps can prune, json, point and tsvector matches, the special-value aggregates, and the composite's fields. 24/24 on each of PG15-19, and the twin on each. Five checks are dated on PG17 by a mutant that stores a by-reference value over 8 bytes as its first 8 bytes. cluster_tests 565 -> 566. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N Squashed for the rebase onto main; it replaces: - test: type_matrix's pinned cidr arm names #1433, the cause OffgridwithJD found (#1341) - test: type_matrix's rows re-dated, restated on main (#1341) - test: type_matrix varies timetz's zone, asserts each range arm's pruning, and names a missing libxml (#1341) - test: types the differential matrix never stored answer as heap (#1341) Squashed for the rebase; it replaces: - test: type_matrix's rows re-dated, restated on main 6f0bd12 (#1341) - test: type_matrix's bit and varbit columns increase with the row, its bounds keep their typmod, and vb must skip (#1341) - test: types the differential matrix never stored answer as heap (#1341)
OffgridwithJD
left a comment
There was a problem hiding this comment.
Red first, then green, on PG18:
| tree | native_agg.sh |
test_native_agg.py |
|---|---|---|
| bc40b67 (the tests only) | 37/40: the three gating arms red, got [10000/1/10000] want [0/-/-] |
1 failed, 1 passed: got '5000/1/5000' want '0/-/-' |
| 01640fd (the fix) | 40/40 | 2 passed |
The .so differs between the two runs (9c1993a0 and 58d3e7f6).
Probes against heap on the fixed build, 200000 rows, with count/min/max/sum compared each time:
- Gates: a false gate, a true gate (
200000/1/200000/20000100000on both sides) and a STABLEcurrent_settinggate. - A gate beside a column qual:
v > 10 AND (SELECT false), and the same with a truenow()gate. - Other plan shapes: GROUP BY under a false gate (0 groups),
HAVING (SELECT false)(0 rows), and a preparedWHERE $1withfalse, both custom and forced-generic (0). - Settings:
enable_ungrouped_vector_agg = on, and forced parallel with 4 workers, for false and true gates and GROUP BY. The parallel plan isGather -> Partial Aggregate -> Result (One-Time Filter) -> Parallel Custom Scan.
All agree with heap. A query with no WHERE still plans Columnar Vectorized Aggregates: 3, so the refusal is no wider than the gate. Returning before any ungrouped path is built is the right place for it.
Found while probing, a separate bug, on main: #1437. Any join with a columnar fact and a gating qual fails with pgcolumnar runtime filter expected a core Hash Join, true gates and EXPLAIN included. enable_join_runtime_filter = off avoids it. It doesn't block this PR, which doesn't touch that path.
🤖 Generated with Claude Code
01640fd to
79f514a
Compare
|
@OffgridwithJD, this is now 79f514a, stacked on #1426 (996413c), which is on #1429, so the three merge in order without restating.
🤖 Generated with Claude Code |
OffgridwithJD
left a comment
There was a problem hiding this comment.
@jdatcmd Approved at 79f514a, stacked on #1426 (996413c).
- The patch: identical to 01640fd's apart from one blank line.
- The conflict: resolved as it should be. In
columnar_vector.cthe gating refusal comes first and #1429'spgcolumnar_quals_could_leakcheck follows it, and both just return. - The combined tree on PG17: native_agg 40/40 and rls_query_paths 24/24 in the shell, and both twins (
test_native_agg.pyandtest_rls_query_paths.py) pass.
My PG18 red-first runs and probes from the 01640fd approval still cover the fix.
🤖 Generated with Claude Code
… so bounds that come out wrong cannot pass on nothing (#1341) Review of #1426 by OffgridwithJD: with the old pg_typeof bounds put back, bt's range became BETWEEN '0' AND '0', found no rows on either table, and both its arms passed on two empty answers. Each of the 16 range arms now first checks heap's answer is not empty, in both twins. That mutation turns bt's new arm red (0 rows), and nothing else. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n, red first (#1435) Three gating WHEREs, (SELECT false), now() < '2000-01-01' and current_user = 'pgc_nobody', each against heap, with a premise that the same aggregate with no WHERE is the vectorized node. On main each returns every row's aggregates (10000/1/10000 in the shell, 5000/1/5000 in the twin) where heap gives 0/-/-. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… qual (#1435) A WHERE that references no column is a pseudoconstant qual, which extract_actual_clauses(..., false) leaves out of the node's quals. With no other qual, needsScan was false, the node answered from the zone maps, and the gating refusal in the scan-fold branch was never reached, so a false gate returned every row's aggregates. The refusal now runs before needsScan is decided, as the grouped node's does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@OffgridwithJD, restated as 3e0a7c0 onto #1426's new head (c1441b4), which added the heap-not-empty arms. Only the budget conflicted, and it's re-derived (3466 rows, 2598 never, 868 dated, matching the tsv). The patch is identical to 79f514a, which you approved, outside the budget and 🤖 Generated with Claude Code |
79f514a to
3e0a7c0
Compare
OffgridwithJD
left a comment
There was a problem hiding this comment.
@jdatcmd Approved at 3e0a7c0, on #1426's c1441b4.
- The patch: the src, the tests and the CHANGELOG are identical to 79f514a's, and only the budget was re-derived.
- Runs: my PG17 run of the combined tree and the PG18 red-first runs still cover it, since c1441b4 only adds type_matrix arms.
🤖 Generated with Claude Code
Fixes #1435, found by @OffgridwithJD reviewing #1429.
The bug: a wrong answer
On a columnar table, an ungrouped aggregate ignored a WHERE that references no column:
SELECT count(*), min(id), max(id) FROM t WHERE (SELECT false)... WHERE now() < '2000-01-01'... WHERE current_user = 'x'Cause: a column-free WHERE is a pseudoconstant (gating) qual, and
extract_actual_clauses(..., false)leaves it out of the node's quals. With no other qual,needsScanwas false and the node answered from the zone maps. The node's own gating refusal sits in the scan-fold branch, which this case never reached.The fix
In
src/columnar_vector.c, the ungrouped node refuses a gating qual beforeneedsScanis decided, as the grouped node does. The query goes to the ordinary plan, which applies the one-time filter. A query with no WHERE, or with a WHERE on a column, is unchanged.Tests (red first)
test/native_agg.shand a newtest_a_where_with_no_column_in_it_is_appliedintest_native_agg.py:0/-/-, so the comparison can't pass on two wrong answers.On main the three arms are red in both twins:
got [10000/1/10000] want [0/-/-]in the shell, andgot '5000/1/5000'in the twin.Evidence
cluster_testsis 601 by collection, the guard set passes 438/438, and docs_style passes.🤖 Generated with Claude Code