Skip to content

test: native_agg's pytest twin covers every shell arm (#1341) - #1514

Merged
jdatcmd merged 2 commits into
mainfrom
test/1341-native-agg-twin
Oct 16, 2026
Merged

jdatcmd merged 2 commits into
mainfrom
test/1341-native-agg-twin

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

For #1341's twin-parity work: native_agg was declared INCOMPLETE, with 28 shell names missing.

The port

A new test, every name the shell's:

  • The node first: the metadata node is asserted (count, sum/min/max) before any answer, because a correct fallback gives the same answers.
  • Parity: the eleven int4, text and sum/avg arms against heap.
  • Plan choice (pgcolumnar.enable_metadata_count is an orphaned GUC, and count(*) loses to a parallel scan by default #133): the node beats a parallel plan, with no Gather.
  • Zone-map reads: count(*) reads none, against an aggregate that must read them, so the zero isn't a dead counter.
  • Fallbacks: filtered, sum(bigint) and the empty table.

The three gating arms (#1435) were already there, but built their names with a conditional f-string the parity tool can't read. They're written out now, which also puts native_agg on the loop-reader list.

compare_to_bash now reports 40 literal matches, 0 missing, so the pair moves to COMPLETE.

Red first

With count(col) folded from the row group's row count instead of its value count (columnar_vector.c), both twins redden "count(col,nulls)":

  • shell: 10000 vs 8000;
  • pytest: 12000 vs 9600, on its own fixture.

Measured

  • Twins: shell 40/40 and pytest 3 tests (40 checks) on each of PG15–19. The PG17 legs were rerun on a clean install, because a mutant build of mine overlapped.
  • Harness: harness_selftest 1246; collection 750 (cluster_tests 749 → 750); guards 439/439.
  • Ledger: gate rc 0. The shell suite is unchanged, so its 40 ledger rows are too.

It shares expected_tests.txt / TESTS.md / test_compare_to_bash.py with #1510, #1512 and #1513; whichever lands later restates those lines.

🤖 Generated with Claude Code

test_native_agg.py ported only the timestamptz arms (#1293) and the
gating arms (#1435); native_agg was declared INCOMPLETE with 28 shell
names missing. A new test ports the rest by name: the metadata node
asserted before any answer (count, sum/min/max), the eleven int4/text/
sum/avg parity arms, the node beating a parallel plan with no Gather
(#133), count(*) reading no zone maps against an aggregate that must,
and the filtered, sum(bigint) and empty-table fallbacks.

The three gating arms existed, but built their names with a conditional
f-string the parity tool cannot read; they are written out now, which
also puts native_agg on the loop-reader list. compare_to_bash: 40
literal matches, 0 missing, so the pair moves to COMPLETE.

Red first: count(col) folded from the row-group row count instead of the
value count (columnar_vector.c) reddens "count(col,nulls)" in both twins
(shell 10000 vs 8000, pytest 12000 vs 9600 on its own fixture).

Matrix: shell 40/40 and pytest 3 tests (40 checks) on PG15-19, the PG17
legs rerun on a clean install; harness_selftest 1246; collection 750
(cluster_tests 749 -> 750); guards 439/439; ledger gate rc 0 (the shell
suite is unchanged, so its 40 ledger rows are too).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd
jdatcmd force-pushed the test/1341-native-agg-twin branch from a522565 to 2718d4a Compare October 16, 2026 20:12
The restate onto main kept both main's 'cluster_tests 750' and this
branch's 751. CI's awk read then returned two lines and pytest refused
'751\n750' as --pgc-expect-tests. Collection on this tree is 751.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd

jdatcmd commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator Author

Head is now 5e413ba6. My restate of 2718d4ae left two cluster_tests lines in expected_tests.txt (751 and main's 750), so CI's awk read returned both and pytest refused '751\n750'. The new commit drops the stale 750 line, and nothing else changes. Collection on this tree is 751.

My local check missed it because it runs only the NO_CLUSTER guards, and test_harness_deps.py (which holds the one-line-per-key check) is not among them. My restate script now runs that file too.

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd approve, at 5e413ba.

Measured

  • PG17 and PG18: the shell is 40/40, and the twin passes its 3 tests with 40 of 40 checks.
  • Pairing: compare_to_bash.py matches all 40 names, with 0 missing.
  • Guards: 439 passed.
  • Collection: 751, which matches cluster_tests 751.

Mutants (columnar_vector.c, PG18)

The shell and the twin go red under the same names:

mutant red
needZones = true from the start "count(*) reads no zone maps"
avg's count adds rg->rowCount, not z->valueCount "avg(int4,nulls)"
the metadata path costed at the cheapest scan's cost (#133 restored) shell: the 4 parallel/Gather arms; twin: "count(*) beats a parallel plan" only (see 4 below)

Non-blocking

  1. test_native_agg.py:244 turns the zone-map delta into yes/no. expect.at_least(after - before, 1, "an aggregate that needs zone maps reads them") would carry the measurement itself, and native_agg.sh:137 drops the number entirely.
  2. The RESETs in _plan (:152-167) aren't in try/finally. If an EXPLAIN raises, the parallel settings stay on the shared connection.
  3. The sum(bigint) fallback and empty-table arms check only the answer. Nothing asserts that the fallback plan ran. Same as the shell, so a paired follow-up rather than this PR.
  4. The third test holds about 27 checks under fail-fast. Under the #133 mutant the twin reports one red where the shell reports four. Splitting it into parity, plan-choice, zone-map and fallback tests would report the whole red set.
  5. The body still says collection 750, which your comment already corrects. The module docstring's first line describes only the timestamptz arms.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd merged commit 4490755 into main Oct 16, 2026
15 checks passed
jdatcmd added a commit that referenced this pull request Oct 17, 2026
…1339)

In CI's harness the first red in a test raises, so a twin holding many
checks in one test reports fewer reds than its shell. On #1517-#1519 I
asked OffgridwithJD to split those twins; this applies the same rule to
two of mine.

test_native_agg.py went from 3 tests holding 40 checks to 40 tests.
Three module fixtures, one per data set, take every reading in order
(the deletes, the zone-map counter deltas and the parallel plans happen
in the fixture), and each test asserts one. With #133 restored (the
metadata path priced at the scan's cost), on PG17:

  shell         4 red: both "beats a parallel plan", both "no Gather"
  old twin      1 red: "count(*) beats a parallel plan"
  new twin      the same 4 as the shell

test_latest_reading.py went from 1 test holding 9 checks to 9 tests,
the same way. With the document's index statement replaced by SELECT 1:

  shell         2 red: "the document's index exists", "under a tenth"
  old twin      1 red: the index premise
  new twin      the same 2 as the shell

Also from #1514's review: the zone-map read arm takes the measured delta
(at_least(delta, 1)) rather than a yes/no, and _plan's RESETs run in a
finally. The twin grader still matches every name: 40 of 40, and 9 of 9.
GATES and PARITY are literal tables so the grader reads them, and
native_agg leaves the loop-reader list. cluster_tests 779 -> 824 by
collection. test_native_skip.py, which has the same shape, waits for

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jdatcmd added a commit that referenced this pull request Oct 17, 2026
…1339)

In CI's harness the first red in a test raises, so a twin holding many
checks in one test reports fewer reds than its shell. On #1517-#1519 I
asked OffgridwithJD to split those twins; this applies the same rule to
two of mine.

test_native_agg.py went from 3 tests holding 40 checks to 40 tests.
Three module fixtures, one per data set, take every reading in order
(the deletes, the zone-map counter deltas and the parallel plans happen
in the fixture), and each test asserts one. With #133 restored (the
metadata path priced at the scan's cost), on PG17:

  shell         4 red: both "beats a parallel plan", both "no Gather"
  old twin      1 red: "count(*) beats a parallel plan"
  new twin      the same 4 as the shell

test_latest_reading.py went from 1 test holding 9 checks to 9 tests,
the same way. With the document's index statement replaced by SELECT 1:

  shell         2 red: "the document's index exists", "under a tenth"
  old twin      1 red: the index premise
  new twin      the same 2 as the shell

Also from #1514's review: the zone-map read arm takes the measured delta
(at_least(delta, 1)) rather than a yes/no, and _plan's RESETs run in a
finally. The twin grader still matches every name: 40 of 40, and 9 of 9.
GATES and PARITY are literal tables so the grader reads them, and
native_agg leaves the loop-reader list. cluster_tests 792 -> 837 by
collection. test_native_skip.py, which has the same shape, is next, now
that #1510 has merged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants