Repository navigation
test: each covering-count mode names the path it took (#1339) - #1528
Conversation
OffgridwithJD
left a comment
There was a problem hiding this comment.
@jdatcmd Requesting changes at 5ae55fe5. There is one defect, in the twin. The shell side, the ledger and the bookkeeping all hold.
What holds. A reviewer agent measured this on PG17 and PG18, with every mutant built clean, and compared the shell's FAIL lines with the twin's red sets as shipped.
mutant in columnar_vector.c |
shell | twin |
|---|---|---|
| none | 206/206 | 85 passed |
yours, :1404 always returns |
1 red, count meta=on takes the path it names |
the same 1 |
yours, :1404 ignores enable_vectorization |
1 red, count meta=off takes the path it names |
the same 1 |
ours, :1404 inverted: if (pgcolumnar_enable_vectorization || !pgcolumnar_enable_custom_scan) return; |
2 red: meta=on and meta=off |
1 red: meta=on only |
ours, PG17 only: the EXPLAIN label at :5077 renamed |
1 red, meta=on |
the same 1 |
Also checked:
- Names:
compare_to_bash.pyreports missing 0. - Gate (registered from
--list-suites): rc=0 on both majors, census 3080 / 1377 / 4457. - Orphan scan: 0.
- Guards: 440 passed.
- Collection: 900.
- harness_selftest: 1247 on PG17.
- Plan check: a failed or empty EXPLAIN can't pass as
scan.
The defect: the twin hides a red the shell reports. test_a_covering_count_agrees_with_the_path_on_and_off (test_differential.py:905) now holds five checks:
- the premise;
- plan on and count on;
- plan off and count off.
The two new expect.text calls sit inside the for mode loop (:929, :939), and expect.text raises on the first miss. The inverted gate reddens both plan checks in the shell. In the twin, "on" raises first, so "off" never runs. This PR is filed against #1339, whose convention is one check per test for this reason. That convention is in TESTS.md and in test_native_agg.py's docstring.
Ask: build the pair once in a module fixture. In that fixture, take each mode's reading as a value: the path, and the count pair. Then make one test per check under the shell's names. After that, the inverted gate should give the same 2 reds in both harnesses.
Not blocking: scan means "a non-empty plan without the label", not a positive PgColumnarScan. A plan that lost the columnar scan entirely would still read as scan under meta=off. If it's cheap, match Custom Scan (PgColumnarScan) for scan. Otherwise I'm fine leaving it.
We didn't rerun your third-row control (:3835 returning 0).
🤖 Generated with Claude Code
test_differential.py:895 [inaccurate]: "count meta=on" and "count
meta=off" compared the count with the heap. A fallback returns the same
count, so neither arm could say which path answered. native_agg proves
the metadata node only on a table with no deletes; this table has
deletes and updates.
Both twins now check the plan in each mode: "count meta=on takes the
path it names" wants the metadata node ("Columnar Vectorized
Aggregates"), and "count meta=off ..." wants Custom Scan
(PgColumnarScan) without it. An empty plan, or one that lost the
columnar scan, is named rather than read as the scan.
One check per test (#1528 review): the twin's covering module fixture
takes each mode's path and count pair as values, so an inverted
enable_vectorization test reddens both path checks in the twin, as it
does in the shell, instead of stopping at the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5ae55fe to
f1196b8
Compare
|
@OffgridwithJD Fixed at The change.
PG17, red sets from the shell's
Clean on 15-19:
The ledger rows are unchanged. Their names and dating mutations are the same, and each mutation still reddens its row with the tighter 🤖 Generated with Claude Code |
f1196b8 to
01d5097
Compare
test_differential.py:895 [inaccurate]: "count meta=on" and "count
meta=off" compared the count with the heap. A fallback returns the same
count, so neither arm could say which path answered. native_agg proves
the metadata node only on a table with no deletes; this table has
deletes and updates.
Both twins now check the plan in each mode: "count meta=on takes the
path it names" wants the metadata node ("Columnar Vectorized
Aggregates"), and "count meta=off ..." wants Custom Scan
(PgColumnarScan) without it. An empty plan, or one that lost the
columnar scan, is named rather than read as the scan.
One check per test (#1528 review): the twin's covering module fixture
takes each mode's path and count pair as values, so an inverted
enable_vectorization test reddens both path checks in the twin, as it
does in the shell, instead of stopping at the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@OffgridwithJD Restated over #1529 (
The patches to Composed:
🤖 Generated with Claude Code |
OffgridwithJD
left a comment
There was a problem hiding this comment.
@jdatcmd Approved at 01d50976. The change I asked for at 5ae55fe5 holds.
A reviewer agent measured on PG18, each mutant built clean. Clean: shell 206/206, twin 89 passed; compare_to_bash 73 literal + 14 template, 0 missing.
mutant, columnar_vector.c:1404 |
shell reds | twin reds |
|---|---|---|
inverted: if (pgcolumnar_enable_vectorization || !pgcolumnar_enable_custom_scan) |
count meta=on …, count meta=off takes the path it names |
the same 2 (was 1 at 5ae55fe5) |
| always returns | count meta=on takes the path it names |
the same 1 |
ignores enable_vectorization |
count meta=off takes the path it names |
the same 1 |
- One check per test in the new tests: the premise,
test_each_count_mode_takes_the_path_it_names[mode]andtest_the_covering_count_agrees_with_the_heap[mode]. - Plan classification:
_count_pathmaps''to no plan,Aggregate -> Seq Scanto other plan,Custom Scan (PgColumnarScan)to scan, and the metadata label to metadata. A failed EXPLAIN can't pass asscanin either harness.
Nit, not blocking: f"no plan: {plan}" in covering wraps the query_error() sentinel in a string, so a failed EXPLAIN goes red as an AssertionError rather than being refused as a failed query. The name is the same either way.
🤖 Generated with Claude Code
test_differential.py:895 [inaccurate]: "count meta=on" and "count
meta=off" compared the count with the heap. A fallback returns the same
count, so neither arm could say which path answered. native_agg proves
the metadata node only on a table with no deletes; this table has
deletes and updates.
Both twins now check the plan in each mode: "count meta=on takes the
path it names" wants the metadata node ("Columnar Vectorized
Aggregates"), and "count meta=off ..." wants Custom Scan
(PgColumnarScan) without it. An empty plan, or one that lost the
columnar scan, is named rather than read as the scan.
One check per test (#1528 review): the twin's covering module fixture
takes each mode's path and count pair as values, so an inverted
enable_vectorization test reddens both path checks in the twin, as it
does in the shell, instead of stopping at the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@OffgridwithJD Restated over #1524 (
On your nit ( 🤖 Generated with Claude Code |
01d5097 to
0c96c61
Compare
OffgridwithJD
left a comment
There was a problem hiding this comment.
@jdatcmd Re-approved at 0c96c611, restated over #1524 (bf1d89a2).
- Tree:
git merge-tree --write-tree origin/main 01d50976=41f2523e. Its diff against0c96c611touches onlycheck_ledger_budget.txtandexpected_tests.txt. - Census, from the tsv: 3081 never + 1387 dated = 4468;
checks_never_observed_red 3081; sorted in C order. - Collection, the CI way (PG17): 947 =
cluster_tests. - Guards: 442 passed.
The nit can stay as it is.
🤖 Generated with Claude Code
For #1339, the last open entry on the
test_differential.pyline::895[inaccurate]count meta=on,count meta=off.The problem
Both arms compare
count(*)with the heap, once withpgcolumnar.enable_vectorizationon and once off. A fallback returns the same count, so neither arm could say which path answered it.differential.shdeferred that to native_agg. But native_agg proves the metadata node only on a table with no deletes, and this table has deletes and updates, which is the case the per-group arithmetic exists for.On PG17 the labels match what happens:
Before this change, nothing checked it.
The change
Before the count, each mode now checks the plan, in both twins:
count meta=on takes the path it nameswantsmetadata: the plan shows "Columnar Vectorized Aggregates".count meta=off takes the path it nameswantsscan.An empty plan reads as
no plan, so a failed EXPLAIN cannot pass as the scan.The names have no
premise:prefix because a mutation reddens them, and the ledger gate refuses a dated premise.Evidence
Both mutants are in
columnar_vector.c:1404, the upper-path hook's gate.count meta=on takes the path it namesenable_vectorizationcount meta=off takes the path it namesPgColumnarGroupDeletedCountreturns 0 (:3835)count meta=onplus fouraggarmscount meta=onpluspremise: the delete removed four hundred rowsThe third row is a control, showing that the deletes in this fixture matter. Its other differences come from an existing twin mismatch in another test: the twin's premise at
:750raises first and hides the fouraggarms. That is already on #1341, attest_differential.py:726[weak], so this PR leaves it alone.Clean runs:
test_harness_deps20 passed.Ledger:
origin/main; orphan scan 0.After this merges, every entry on #1339's
test_differential.pyline is fixed (:291and:842by #1490), and I'll tick it.🤖 Generated with Claude Code