[OffgridwithJD]

Stacked on #1477 (merge after it). Two commits: 3b4bff5a (the twin and the shell fix) and 82a31a21 (the ledger seed).

What was wrong

The oracles compare an index scan with a sequential scan, but their "index" side only set enable_seqscan=off, which doesn't disable the columnar custom scan. Measured plans (PG18):

oracle	"index" side ran as
late-column	Custom Scan
expression	Custom Scan
text expression	Custom Scan
expression point lookup	Custom Scan
plain, partial	an index

So three oracles compared a sequential scan with itself. With w_expr dropped before the oracles, the old shell stayed green on the expression oracle. The fixes:

• The index side now also sets pgcolumnar.enable_custom_scan=off.
• Each oracle asserts, as a premise, "premise: -- the index side reads ".

Turning the custom scan off exposed a second problem. The aggregate's ORDER BY sent the late-column oracle down a full scan of w_k12 with its predicate as a filter, fetching every row through the index: over 120 s at 60,000 rows, and the 300,000-row run blew its 30-minute limit. The premise therefore reads the oracle query itself, fenced with OFFSET 0, and every oracle reads its own index in milliseconds. The suite runs in 18 s.

The md5-shape failure now records under the oracle's own name. A separate failure-only name left an orphan ledger row.

Measured (PG18)

mutation	old shell (35)	new shell (41)	twin
DROP INDEX w_expr before the oracles (harness)	green on the expression oracle; only amcheck red	red: premise: expression index agrees -- the index side reads w_expr	—
projection omits ii_Expressions' columns	red: the 3 expression projection arms	same	same
projection omits ii_Predicate's columns	red: the 2 partial projections, the lookup, the oracle	same	same

The twin

test_native_index_projection.py carries all 41 checks by the shell's name (compare_to_bash: every property covered), in four tests. As in the shell, the parallel indexes are built after the serial oracles. When they came first, the planner took w_par for the plain oracle. The build lines come from psycopg notice handlers at client_min_messages=debug1. The oracle table joins the loop-reader guard's list.

Runs

Head is 41/41 in both twins on each of 15, 16, 17, 18 and 19, at the full 300,000 rows.

Ledger

• The suite had no rows. All 41 enter, and 7 are dated by the two projection mutations.
• Thirty-four arrive never:
  • the six oracle premises, which only the harness change reddens;
  • the fixture and branch arms;
  • the plain and late-column projections;
  • the oracles a projection mutation can't reach;
  • the eleven amcheck arms.
• suites_not_covered 203 → 202. The census is 4047 / 2929 / 1118.
• pgc_ledger.py gate returns rc 0, and orphan-scan reports 0 orphans.

Counts and guards

• cluster_tests 698 → 702, and pytest collects 702.
• COMPLETE gains the suite, and TESTS.md gains §163.
• The NO_CLUSTER guards pass (438), harness_selftest.sh passes, and docs_style.sh passes.

A note for the product, outside this PR: a full index scan of a 20-column columnar table fetches at roughly 2 ms a row (60,000 rows in over 120 s), while the same query on the columnar scan takes 15 ms.

🤖 Generated with Claude Code

[OffgridwithJD]

@jdatcmd #1451, #1459 and #1475 landed under this, so I've restated it onto main e3e9654f and force-pushed with a lease on the old head 82a31a21. New head: a08464a2. It's MERGEABLE again.

• Census: main is now 4007 / 2889 never / 1118 dated, so this PR's 41 rows (34 never, 7 dated) give 4048 / 2923 / 1125. That's in the budget paragraph and the second commit's message. pgc_ledger.py gate --against origin/main on the five-major logs says "census stated 2923, ledger holds 2923", with suites_not_covered 203 -> 202 and rc 0.
• Code: outside the bookkeeping files, the new tip differs from the old one by exactly main's own diff (git diff 82a31a21 a08464a2 equals git diff <old base> e3e9654f).
• Counts: main's three PRs added no tests. Collection still gives cluster_tests 702, and TESTS.md §163 stands. The no-cluster guard set passes, 438/438.

The PR body still quotes the pre-restate census (4047 / 2929 / 1118). The commit and the files are right.

One thing on main, not from this PR: #1475 renamed its five rows in place, so check_ledger.tsv is no longer sorted in two places. "a table that was not rewritten still points at itself" sits after "premise: CREATE TABLE ... AS …", and "premise: the storage row points at the live table…" sits after "the rewritten table still holds every row". The gate doesn't check order, but the next pgc_ledger.py merge re-sorts them, so they'll show up as churn in whichever PR touches the ledger next. Tell me if you want a one-commit fix, or fold it into your next ledger PR.

🤖 Generated with Claude Code

[jdatcmd]

Approve, at a08464a. Against 82a31a2, your restate leaves the patches of all five test files and the ledger rows unchanged; only the budget paragraph is re-anchored. So this evidence, measured at 82a31a2, holds here.

My own mutant, different from yours: in pgcolumnar_index_build_range_scan, skip every 997th row, but only for an expression index (if (index_info->ii_Expressions != NIL && rowNumber % 997 == 0) continue;). The projection count stays right, so only a real oracle can see it. PG17:

shell	result under the mutant
main's native_index_projection.sh	35/35 green
yours	2 red: expression index agrees and text expression index agrees (md5 mismatch)
your pytest twin	1 red: expression index agrees

The clean build afterwards is 41/41. So the custom-scan-off change and the per-oracle plan premise are what give these oracles teeth.

One note, not blocking. The twin found 1 red where the shell found 2, because expect raises on the first failing arm and all five oracles sit in one test function. A red there hides the oracles after it. If you want the twins' failure output to match, give each oracle its own test (or parametrize it).

#1479 also adds one ledger row (census 2889 -> 2890), so whichever of us merges second re-anchors.

[jdatcmd]

@OffgridwithJD, merging #1480 now: approved at a08464a, 15/15 green, CLEAN.