[jdatcmd]

Closes #1298. For @OffgridwithJD to review. This is the plan posted on #1298, with one change: native_fetch_coalesce needs no conversion (see below).

The problem

catalog_delete_index and catalog_plan_index compare catalog buffer counts that are a few buffers apart. Examples: the small drop's 24 vs 27, and the storage arm's single block. Both read the counts from pg_statio_all_tables. That view is shared across sessions, so noise on the reference side of a comparison lets a regression pass.

Measured on PG18, one compact() of a 40-group table per reading. Each pair has the same catalog size, and in the noisy run a second session scans the catalogs:

pair	quiet: shared / transaction	noisy: shared / transaction
1	570 / 554	598 / 556
3	679 / 659	776 / 656
5	743 / 721	863 / 724

(All six pairs are on #1298.)

The change

• Readings: taken from pg_stat_get_xact_blocks_fetched, before and after the measured statements, in one transaction. Nothing is flushed while a transaction is open, so the difference is the statements' own work. Pending counts from the fixture's writes cancel out, and other sessions can't reach the reading.
  • Shell: pgc_xact_blocks in lib.sh, used by cat_blocks (compaction, drop), stor_blocks and plan_work.
  • pytest: _cat_blocks and _xact_work, with the storage scan counts from pg_stat_get_xact_numscans.
• cat_detail prints the breakdown of the same reading, so it always adds up to the total.
• Not converted: the vacuum reading. VACUUM can't run in a transaction block. Its margin is large (714 permille).
• The seq_scan = 0 arms keep their shared-view reading, as agreed on Planning-buffer counts wobbled in CI and the cause was never found: audit the other arms that count buffers #1298.

Evidence

• Red first, in both twins, both suites. The new arm measures a statement that reads no catalog. Inside it, a second session started by COPY (SELECT 1) TO PROGRAM 'psql …' scans the catalogs and exits. The reading must be 0.

  suite	shell, old helper	pytest, old helper
  catalog_delete_index	got [31] want [0]	got 4 want 0
  catalog_plan_index	got [10] want [0]	got 2 want 0

  A control beside each arm shows the second session's reads do reach the shared view.

• Green:

  • catalog_delete_index 36/36 and catalog_plan_index 29/29 on each of PG15–19;
  • both pytest files: 10 passed, 67 checks, on each of PG15–19.

• The comparative arms still catch what they guard, under the new reading:

  • Size check removed (pgcolumnar_scan_index_oid always probes): all seven comparative arms fail in the shell. The small drop gives got [0] want [100]. pytest fails P1, the vacuum arm and the drop.
  • The written-limit lookup scans pgcolumnar.storage sequentially (Every planned query scans pgcolumnar.storage twice; relation_oid has no index #1210 reverted): FAIL the newest columnar table costs no more catalog work than the oldest: got [-3] want [1] (newest 9 blocks, oldest 4). pytest fails the seq-scan arm (newest 6, oldest 2).
  • A correction to my own run: I first aimed this mutation at storage_pkey through the size check. That isn't the written-limit path, and the arms rightly stayed green.

• native_fetch_coalesce: no conversion. Its reading is EXPLAIN (ANALYZE, BUFFERS), backend-local and warmed. Over 20 readings, 10 quiet and 10 with another session scanning the same table and the catalogs, the narrow fetch read 77 every time and the wide one 77 (once 79). Its failure is 15 pins over a 3-pin bound.

• Counts:

  • Ledger: four rows. The two arms are dated by their red-first runs; the two controls arrive never. 1893 → 1895 never. No row of main's changed.
  • The gate against origin/main: rc=0, census agrees, ceiling 244 → 244.
  • Database-free guard set: 429 passed. cluster_tests 538 → 540 by collection.

• Conflicts: this collides with ci: the nightly's coverage job installs pg_partman, hll and tdigest (#1382) #1385 and test: the row-identity lock and its two settings, in both twins (#1341) #1368 on the budget and cluster_tests. Whichever lands later restates.

🤖 Generated with Claude Code

https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N

[jdatcmd]

Rebased onto f601281 (#1368 merged). The new head is 879305d. cluster_tests is restated 539 → 541 by collection (541 collected). Nothing else changed; the ledger rows and budget were already on d7cd8f3's figures, and #1368 has no ledger rows.

[OffgridwithJD]

Approved at 879305d, with full CI green.

I checked the one thing that could make the new reading measure less than the old. The reading is taken before the transaction commits, so catalog work deferred to pre-commit would drop out. pgcolumnar_xact_callback does run PgColumnarExtentPreCommit there, with extent_recompute_storage for storages whose groups went, which is what a compaction does. Measured on PG17 against this head: the same compaction read in-transaction (your helper) and after commit through the shared view, on a quiet cluster, three pairs:

pair	in-transaction	after commit
1	86 (delete_vector 81, row_group 5)	86
2	89	88
3	91	88

Nothing in CATS is left for pre-commit, so the conversion loses no work. The drop's catalog cleanup runs in the drop hook, synchronously.

Reproduced (PG18): with the size check removed (pgcolumnar_scan_index_oid always probes), all seven comparative arms fail in the shell, from P1 … got [12] want [100] to the drop's default … got [-23]. pytest fails P1, the vacuum arm and the drop. Clean on PG15 and PG19 (CI covers 17/18): catalog_delete_index 36/36, catalog_plan_index 29/29, both pytest files 10 passed.

Nit, not blocking: the body says cluster_tests 538 → 540, but the branch is on f601281 and says 541 (539 + 2), which is right.

🤖 Generated with Claude Code