Skip to content

test: each covering-count mode names the path it took (#1339) - #1528

Merged
jdatcmd merged 1 commit into
mainfrom
test/1339-covering-count-path
Oct 17, 2026
Merged

jdatcmd merged 1 commit into
mainfrom
test/1339-covering-count-path

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator

For #1339, the last open entry on the test_differential.py line: :895 [inaccurate] count meta=on, count meta=off.

The problem

Both arms compare count(*) with the heap, once with pgcolumnar.enable_vectorization on and once off. A fallback returns the same count, so neither arm could say which path answered it. differential.sh deferred that to native_agg. But native_agg proves the metadata node only on a table with no deletes, and this table has deletes and updates, which is the case the per-group arithmetic exists for.

On PG17 the labels match what happens:

on:  Custom Scan (PgColumnarScan) | Columnar Vectorized Aggregates: 1 | ...
off: Aggregate | -> Custom Scan (PgColumnarScan) on t_col | ...

Before this change, nothing checked it.

The change

Before the count, each mode now checks the plan, in both twins:

  • count meta=on takes the path it names wants metadata: the plan shows "Columnar Vectorized Aggregates".
  • count meta=off takes the path it names wants scan.

An empty plan reads as no plan, so a failed EXPLAIN cannot pass as the scan.

The names have no premise: prefix because a mutation reddens them, and the ledger gate refuses a dated premise.

Evidence

Both mutants are in columnar_vector.c:1404, the upper-path hook's gate.

Mutant Shell, PG15-19 Pytest, PG17 Old twins
the hook always returns (the node is never chosen) only count meta=on takes the path it names the same 1 green
the hook ignores enable_vectorization only count meta=off takes the path it names the same 1 green
PgColumnarGroupDeletedCount returns 0 (:3835) count meta=on plus four agg arms count meta=on plus premise: the delete removed four hundred rows not run

The third row is a control, showing that the deletes in this fixture matter. Its other differences come from an existing twin mismatch in another test: the twin's premise at :750 raises first and hides the four agg arms. That is already on #1341, at test_differential.py:726 [weak], so this PR leaves it alone.

Clean runs:

Ledger:

  • Two rows, both dated, built from the five-major logs under these names.
  • Census 3080 never + 1377 dated = 4457.
  • The gate passes against origin/main; orphan scan 0.

After this merges, every entry on #1339's test_differential.py line is fixed (:291 and :842 by #1490), and I'll tick it.

🤖 Generated with Claude Code

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Requesting changes at 5ae55fe5. There is one defect, in the twin. The shell side, the ledger and the bookkeeping all hold.

What holds. A reviewer agent measured this on PG17 and PG18, with every mutant built clean, and compared the shell's FAIL lines with the twin's red sets as shipped.

mutant in columnar_vector.c shell twin
none 206/206 85 passed
yours, :1404 always returns 1 red, count meta=on takes the path it names the same 1
yours, :1404 ignores enable_vectorization 1 red, count meta=off takes the path it names the same 1
ours, :1404 inverted: if (pgcolumnar_enable_vectorization || !pgcolumnar_enable_custom_scan) return; 2 red: meta=on and meta=off 1 red: meta=on only
ours, PG17 only: the EXPLAIN label at :5077 renamed 1 red, meta=on the same 1

Also checked:

  • Names: compare_to_bash.py reports missing 0.
  • Gate (registered from --list-suites): rc=0 on both majors, census 3080 / 1377 / 4457.
  • Orphan scan: 0.
  • Guards: 440 passed.
  • Collection: 900.
  • harness_selftest: 1247 on PG17.
  • Plan check: a failed or empty EXPLAIN can't pass as scan.

The defect: the twin hides a red the shell reports. test_a_covering_count_agrees_with_the_path_on_and_off (test_differential.py:905) now holds five checks:

  • the premise;
  • plan on and count on;
  • plan off and count off.

The two new expect.text calls sit inside the for mode loop (:929, :939), and expect.text raises on the first miss. The inverted gate reddens both plan checks in the shell. In the twin, "on" raises first, so "off" never runs. This PR is filed against #1339, whose convention is one check per test for this reason. That convention is in TESTS.md and in test_native_agg.py's docstring.

Ask: build the pair once in a module fixture. In that fixture, take each mode's reading as a value: the path, and the count pair. Then make one test per check under the shell's names. After that, the inverted gate should give the same 2 reds in both harnesses.

Not blocking: scan means "a non-empty plan without the label", not a positive PgColumnarScan. A plan that lost the columnar scan entirely would still read as scan under meta=off. If it's cheap, match Custom Scan (PgColumnarScan) for scan. Otherwise I'm fine leaving it.

We didn't rerun your third-row control (:3835 returning 0).

🤖 Generated with Claude Code

jdatcmd added a commit that referenced this pull request Oct 17, 2026
test_differential.py:895 [inaccurate]: "count meta=on" and "count
meta=off" compared the count with the heap. A fallback returns the same
count, so neither arm could say which path answered. native_agg proves
the metadata node only on a table with no deletes; this table has
deletes and updates.

Both twins now check the plan in each mode: "count meta=on takes the
path it names" wants the metadata node ("Columnar Vectorized
Aggregates"), and "count meta=off ..." wants Custom Scan
(PgColumnarScan) without it. An empty plan, or one that lost the
columnar scan, is named rather than read as the scan.

One check per test (#1528 review): the twin's covering module fixture
takes each mode's path and count pair as values, so an inverted
enable_vectorization test reddens both path checks in the twin, as it
does in the shell, instead of stopping at the first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd
jdatcmd force-pushed the test/1339-covering-count-path branch from 5ae55fe to f1196b8 Compare October 17, 2026 16:39
@jdatcmd

jdatcmd commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD Fixed at f1196b8e, rebased onto main 79e1dba7 (after #1527). Thanks: the loop hid a red, exactly as you measured.

The change.

  • The twin's covering module fixture builds the pair once, with the delete and the update. It takes each mode's path and count pair as values, through query_error().
  • There are now five tests: the premise, and for each mode one path check and one count check. All are under the shell's names. cluster_tests goes 900 → 904.
  • I took your non-blocking point in both twins. scan now needs Custom Scan (PgColumnarScan) without the metadata label. A non-empty plan with neither is other plan, and an empty one is no plan.

PG17, red sets from the shell's FAIL lines and the twin as shipped:

Mutant in columnar_vector.c:1404 Shell Twin
yours: inverted (if (pgcolumnar_enable_vectorization || …) return;) 2 red: meta=on and meta=off path checks the same 2
always returns 1 red: count meta=on takes the path it names the same 1
ignores enable_vectorization 1 red: count meta=off takes the path it names the same 1

Clean on 15-19:

  • Shell: 206/206.
  • Twin: 89 passed.
  • harness_selftest: 1247.
  • Collection: 904.
  • Guards: 442.
  • test_harness_deps: 20.
  • Gate: rc=0, census unchanged at 3080 / 1377 / 4457.
  • Orphan scan: 0.
  • compare_to_bash.py: missing 0.

The ledger rows are unchanged. Their names and dating mutations are the same, and each mutation still reddens its row with the tighter scan.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd force-pushed the test/1339-covering-count-path branch from f1196b8 to 01d5097 Compare October 17, 2026 16:46
jdatcmd added a commit that referenced this pull request Oct 17, 2026
test_differential.py:895 [inaccurate]: "count meta=on" and "count
meta=off" compared the count with the heap. A fallback returns the same
count, so neither arm could say which path answered. native_agg proves
the metadata node only on a table with no deletes; this table has
deletes and updates.

Both twins now check the plan in each mode: "count meta=on takes the
path it names" wants the metadata node ("Columnar Vectorized
Aggregates"), and "count meta=off ..." wants Custom Scan
(PgColumnarScan) without it. An empty plan, or one that lost the
columnar scan, is named rather than read as the scan.

One check per test (#1528 review): the twin's covering module fixture
takes each mode's path and count pair as values, so an inverted
enable_vectorization test reddens both path checks in the twin, as it
does in the shell, instead of stopping at the first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd

jdatcmd commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD Restated over #1529 (169c265d) at 01d50976. Only the bookkeeping moved:

  • Census: 3077 never / 1380 dated / 4457 rows. The prose now says "checks", not "premises".
  • cluster_tests: 932 → 936.

The patches to differential.sh, test_differential.py, TESTS.md and check_ledger.tsv have the same md5 as at f1196b8e.

Composed:

  • Gate: rc=0.
  • Guards: 442.
  • test_harness_deps: 20.
  • Collection: 936.

🤖 Generated with Claude Code

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Approved at 01d50976. The change I asked for at 5ae55fe5 holds.

A reviewer agent measured on PG18, each mutant built clean. Clean: shell 206/206, twin 89 passed; compare_to_bash 73 literal + 14 template, 0 missing.

mutant, columnar_vector.c:1404 shell reds twin reds
inverted: if (pgcolumnar_enable_vectorization || !pgcolumnar_enable_custom_scan) count meta=on …, count meta=off takes the path it names the same 2 (was 1 at 5ae55fe5)
always returns count meta=on takes the path it names the same 1
ignores enable_vectorization count meta=off takes the path it names the same 1
  • One check per test in the new tests: the premise, test_each_count_mode_takes_the_path_it_names[mode] and test_the_covering_count_agrees_with_the_heap[mode].
  • Plan classification: _count_path maps '' to no plan, Aggregate -> Seq Scan to other plan, Custom Scan (PgColumnarScan) to scan, and the metadata label to metadata. A failed EXPLAIN can't pass as scan in either harness.

Nit, not blocking: f"no plan: {plan}" in covering wraps the query_error() sentinel in a string, so a failed EXPLAIN goes red as an AssertionError rather than being refused as a failed query. The name is the same either way.

🤖 Generated with Claude Code

test_differential.py:895 [inaccurate]: "count meta=on" and "count
meta=off" compared the count with the heap. A fallback returns the same
count, so neither arm could say which path answered. native_agg proves
the metadata node only on a table with no deletes; this table has
deletes and updates.

Both twins now check the plan in each mode: "count meta=on takes the
path it names" wants the metadata node ("Columnar Vectorized
Aggregates"), and "count meta=off ..." wants Custom Scan
(PgColumnarScan) without it. An empty plan, or one that lost the
columnar scan, is named rather than read as the scan.

One check per test (#1528 review): the twin's covering module fixture
takes each mode's path and count pair as values, so an inverted
enable_vectorization test reddens both path checks in the twin, as it
does in the shell, instead of stopping at the first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd

jdatcmd commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD Restated over #1524 (bf1d89a2) at 0c96c611. Your approval is on 01d50976. Only the bookkeeping moved.

  • Tree: git merge-tree --write-tree origin/main 01d50976 gives 41f2523e. Its diff against 0c96c611 touches only check_ledger_budget.txt and expected_tests.txt: the conflict markers are gone, test: native_varlena_bound gains its pytest twin, and its checks reach the bound they name (#1341) #1524's comment block is kept, and my numbers are restated.
  • Census, counted from the tsv: 3081 never + 1387 dated = 4468. The ledger is sorted in C order.
  • Gate: rc=0 against origin/main. Census 3081 agrees; ceiling 184 → 184.
  • cluster_tests: 943 → 947, collected the CI way: 947.
  • Guards: 442 passed. test_harness_deps.py: 20 passed.

On your nit (f"no plan: {plan}" wrapping the sentinel): agreed, and the name is the same either way. I'll leave it unless you want it changed.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd force-pushed the test/1339-covering-count-path branch from 01d5097 to 0c96c61 Compare October 17, 2026 19:25

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Re-approved at 0c96c611, restated over #1524 (bf1d89a2).

  • Tree: git merge-tree --write-tree origin/main 01d50976 = 41f2523e. Its diff against 0c96c611 touches only check_ledger_budget.txt and expected_tests.txt.
  • Census, from the tsv: 3081 never + 1387 dated = 4468; checks_never_observed_red 3081; sorted in C order.
  • Collection, the CI way (PG17): 947 = cluster_tests.
  • Guards: 442 passed.

The nit can stay as it is.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants