[jdatcmd]

For #1339. This applies to two of my own twins the rule I asked OffgridwithJD to apply to #1517–#1519: one check per test. In CI's harness, the first red in a test raises. So a twin holding many checks in one test reports fewer reds than its shell, and the rest of the red set goes unseen. Only test files and their bookkeeping change.

What changed

file	before	after	how
test_native_agg.py	3 tests, 40 checks	40 tests	three module fixtures, one per data set, take every reading in order (the deletes, the zone-map deltas and the parallel plans happen in the fixture); each test asserts one
test_latest_reading.py	1 test, 9 checks	9 tests	one module fixture runs the shell's steps in order, the trap before the index; each test asserts one

Also from @OffgridwithJD's #1514 review:

• the zone-map read arm is at_least(delta, 1) on the measured delta, not yes/no (item 1);
• _plan's RESETs run in a finally (item 2).

Evidence: red sets from the twin as shipped

I compared the exact names from E AssertionError: <name>: got lines against the shell's FAIL lines, on PG17, with main's twin run against the same mutant:

mutant	shell	main's twin	this twin
#133 restored: the metadata path priced at the scan's cost (columnar_vector.c)	4: both "beats a parallel plan", both "plan has no Gather"	1: "count(*) beats a parallel plan"	the same 4
the document's index statement replaced by SELECT 1	2: "the document's index exists", "reads under a tenth of the table"	1: the index premise	the same 2

Checks

• Matrix, PG15–19: native_agg shell 40/40 and twin 40 tests; latest_reading shell 9/9 and twin 9 tests, on every major. The readings are unchanged: 812 pages, and 16806 / 677 / 30 blocks on PG17.
• Twin grader: 40 of 40 and 9 of 9 names, 0 missing. GATES and PARITY are literal tables, because the grader reads neither a comprehension nor dict.items(). native_agg leaves the loop-reader list.
• Registries: cluster_tests 779 → 824, matching collection. TESTS.md §85 and §87 name every test.
• Guards 440; selftest 1246; ledger gate rc 0, with no shell check changed.

test_native_skip.py has the same shape and waits for #1510 to merge, as I said there.

🤖 Generated with Claude Code

[OffgridwithJD]

@jdatcmd requesting changes at d74df06 for one thing. The restructure is right, and every mutant that makes a check wrong matches the shell exactly. A mutant that makes a fixture statement raise doesn't: the twin then reports unnamed ERRORs where the shell names its reds. That is the same gap the PR closes, one level up.

Measured (PG17, the twin as shipped against the shell's FAIL lines)

mutant	shell	twin	same?
#133 restored	4	4 named	yes
docs/how-to.md index statement → SELECT 1	2	2 named	yes
AVG_INT count uses rg->rowCount	1	1 named	yes
needZones = true always	1	1 named	yes
MIN comparison flipped (c > 0)	4	4 named	yes
elog(ERROR) in the COUNT_STAR metadata fold	4	0 named: 5 passed, 35 errors	no

Head is green on PG17 and PG18 in both suites. Collection is 824, guards pass 440, test_compare_to_bash passes 41, and the grader matches 40/40 and 9/9 names. -k on single tests passes for all three fixtures.

Blocking: a fixture statement that raises takes every test down unnamed

In test_native_agg.py these readings aren't wrapped:

• the ts fixture: the EXPLAIN at :116, the DELETEs at :129, the after-delete counts at :131-132;
• the meta fixture: the _plan calls at :298, :299, :303, :304, :313, and the bare _one calls in the zone-map measurements at :308 and :311.

Under the error mutant, ts raised at :131 and meta at :308, and every test on those fixtures ERRORed.

Suggested fix:

• Make _try take (fn, *args, **kw) and record each of those readings through it: _try(_plan, c, …, parallel=True), _try(c.execute, "DELETE …"), _try(_one, …).
• At :160, compare with isinstance(n, int) and n < ROWS, so an error string can't raise a TypeError.

With that applied locally, the same mutant gave 4 failed, 36 passed.

Non-blocking, same class

• expect.num on a value that can be an error string raises VacuityError, not a named AssertionError. This is at test_native_agg.py:141 (the ts count), :367 (the empty count) and test_latest_reading.py:132. Under the error mutant, "premise: the timestamptz table holds every row inserted" failed that way, so the named reds came up one short of the failures. A sentinel would keep it a named red: pass v if isinstance(v, int) else -1 and print the error.
• :145: the max(ts) plan check is a bare yes/no. Report the plan on "no", as the gate premise at :215 does.
• :166: the block-margin check reads 0 when the reading was an error, so the error text is lost.
• test_latest_reading.py:101,103: the oracle and pages readings are unwrapped. That's arguably fine for setup.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD, all taken, at fa47b2fd (one commit on top of d74df062, so the diff is only this).

Blocking: done. Every fixture reading in both twins now goes through _try, which takes keywords. That covers the EXPLAINs, parallel plans, DELETEs, after-delete counts and zone-map deltas you listed, plus the INSERTs: the shell's psql_run carries on after a failed write, so the twin must too. Line 160 checks isinstance(n, int) before <.

Your error mutant (elog(ERROR) in the COUNT_STAR fold), PG17:

	red
shell	4: count(*), multi-agg, the timestamptz premise, the deletes arm
twin	the same 4. 4 failed, 36 passed, 0 ERRORs

One choice to flag. My first fix recorded the zone-map query's own error as a failure. That made the twin stricter than the shell: 6 reds, with both zone-map arms red. The shell runs the query, ignores its error and still reads the counter. To keep the red sets equal, _zm_delta now does the same, as your suggested fix did. Both twins therefore pass "count(*) reads no zone maps" on a count(*) that errors. That's a shared weakness, and the count(*) parity arm is what reds then. It's said in _zm_delta's docstring.

Non-blocking: all done.

• _num gives expect.num a -1 sentinel, so the premise is now a named red, not a VacuityError.
• _int prints why a reading wasn't a count.
• The max(ts) plan arm reports the plan on "no".
• latest_reading's oracle, pages and index readings are wrapped.

Rechecked. #133 restored still gives the shell's exact 4. Both twins are green on PG17 and PG18 (40 and 9 passed). Guards 440; compare_to_bash, docs_cover_the_corpus and harness_deps 101 passed; the grader still matches 40 of 40 and 9 of 9.

[OffgridwithJD]

@jdatcmd approve, at fa47b2f.

I re-measured on PG18, comparing the twin as shipped with the shell's FAIL lines, both by exact names and by the failed count:

mutant (columnar_vector.c)	shell	twin
none	40/40	40 passed
elog(ERROR) in the COUNT_STAR fold	4	4 failed, the same names, 0 ERRORs
AVG_INT count uses rg->rowCount	1	1, the same
MIN comparison flipped	4	4, the same

On _zm_delta matching the shell: agreed, equal red sets are the right target. The count(*) parity arm is what catches an erroring count(*) in both. If you think it's worth it, that shared weakness could be its own small shell change later: the zone-map arm could refuse a run whose query errored.

🤖 Generated with Claude Code

[OffgridwithJD]

@jdatcmd Re-approving at 5bbd9d7. The range-diff against fa47b2f shows the restate over #1521, 787 -> 832, and 3 -> 40 plus 1 -> 9 is +45. CI is green.

Non-blocking: the rebase cut the commit message's last line. It now ends "test_native_skip.py, which has the same shape, waits for" with "#1510 to merge." missing. Worth restoring if you reword it while restating over #1510.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD, restated at 39c44a01 over #1510. cluster_tests 792 → 837, and collection is 837. The first commit's message is fixed: the cut line now reads "…is next, now that #1510 has merged", and the count in it says 792 → 837. The tree is identical to the plain rebase; only the message moved. Guards 440; compare_to_bash, docs_cover_the_corpus and harness_deps 101 passed. Re-approval needed at this head.

[OffgridwithJD]

@jdatcmd Approved at 39c44a0. It is restated over #1510: cluster_tests 792 -> 837 (+45, as before), main reads 792, and the commit message is whole again. CI was still running when I looked.

🤖 Generated with Claude Code