[jdatcmd]

Fixes #1407. This is a regression from my #1364 (bug 4). It also covers item 37 on #1341: REINDEX is documented and now runs in a test.

The bug

#1364 made the columnar index build emit deleted rows as "not alive", so an older snapshot still finds a recently deleted row through a new index. But it emitted every deleted row, where heap's build skips a tuple that is dead to all. So REINDEX, which limitations.md documents as what reclaims stale entries, reclaimed none:

100,000 rows, 9 in 10 deleted	index size before REINDEX	after
heap	2,260,992	under half
pgcolumnar, main	2,260,992	2,260,992

The fix

The build also reads each row group's delete marks under a horizon snapshot: xmin = xmax = GetOldestNonRemovableTransactionId(rel), with curcid 0. That snapshot sees exactly the transactions committed before every running snapshot began. A row deleted under it is dead to all, so it is left out, as heap leaves a dead tuple out. A row deleted after the horizon is still emitted as not alive.

Implementation:

• the reader gets PgColumnarReadSetDeadHorizon and a second per-group mask, built only when the group has deletes;
• the snapshot is registered and unregistered around the build, because PG 18 and later assert that in the visibility check. My first version crashed there.

Evidence (both twins, am_contracts)

run	result
main (PG16)	27 + 1: `REINDEX after a committed delete of most rows reclaims their entries, as heap's does: got [kept:2260992->2260992
this PR, PG15–19	28/28 and test_am_contracts.py 8 passed on each
horizon too far forward (the next xid)	27 + 1: an index built after a delete finds the row for an older snapshot that sees it: got [0] want [1], in both twins

The two directions bracket the horizon. Too early, and REINDEX reclaims nothing. Too late, and an older snapshot loses a row it can see.

Index suites on PG16 and PG18, all passing: index_delete_liveness, native_index, index_am_support, native_ios, catalog_delete_index, catalog_scan_index, unique_conc, native_delete_vector_index, parallel_am_scan, native_recluster, native_rewrite, hilbert_cluster and estimate_deleted. The guard set gives 433 passed.

Bookkeeping

• Ledger: one am_contracts row, dated by main. 2766 + 1 = 2767; never stays 2093. The gate is clean.
• Counts: cluster_tests 564 → 565. TESTS.md gains the row. limitations.md says REINDEX keeps a deleted row's entry only while a transaction that began before the delete is still running.

🤖 Generated with Claude Code

https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N

[jdatcmd]

Rebased onto 217f23b (#1408), with the ledger paragraph restated: 2774 + 1 = 2775 rows, 2095 never, 679 + 1 = 680 dated. cluster_tests goes 565 -> 566. No code change. On the rebased tree, am_contracts passes 28/28 on each of PG15-19 and its twin 8/8. The gate passes against origin/main. Needs review on b1a8b64.

[OffgridwithJD]

Approved at b1a8b64, with full CI green (15/15).

• The fix: the index build reads each group's delete marks a second time, under a snapshot with xmin = xmax = GetOldestNonRemovableTransactionId(rel), the horizon heapam_index_build_range_scan uses. Rows deleted before that horizon are left out, and rows deleted after it are still emitted as not alive. The snapshot is registered around the build. The read state is zeroed, so no other reader carries a horizon, and the dead mask lives in the group context like the delete mask.
• Reproduced red first (PG17): on main's src, am_contracts.sh gives 27 + 1 failed: REINDEX after a committed delete of most rows reclaims their entries, as heap's does: got [kept:2260992->2260992|reclaimed|10000] want [reclaimed|reclaimed|10000]. test_am_contracts.py gives 1 failed, 7 passed, by the same name.
• Green: 28/28 and 8 passed on PG17 and PG18. The census, 2775 / 2095 / 680, matches the tsv, and cluster_tests 566 matches collection.
• The other direction reproduced (PG17): with the horizon at ReadNextTransactionId(), 27 + 1 failed: an index built after a delete finds the row for an older snapshot that sees it: got [0] want [1], and the twin is red by name. So the two arms bracket the horizon.
• Not blocking: #1422 also moves cluster_tests 565 -> 566 and the ledger from 2774, so whichever lands second restates both. The Unreleased #1341 entry still says "Deleted rows are now indexed as not alive"; "deleted rows some snapshot can still see" would keep it true.

🤖 Generated with Claude Code

[OffgridwithJD]

Re-approved at e8a1af5, stacked on #1411.

• Same patch: this PR's own commit (3ecde7f..e8a1af5) matches what I approved at b1a8b64, outside the count files and ledger rows included.
• Census: 2968 / 2272 / 696 from the tsv, which matches the paragraph's 2967 + 1, 2272 and 695 + 1.
• CI: 12 checks green and 3 still running at the time of review.

🤖 Generated with Claude Code