[jdatcmd]

Covers #1341 item 2 (SSI). Test plus docs: no source change.

What the audit expected, and what happens

The audit noted that the access method takes no SSI predicate locks: no PredicateLockRelation, PredicateLockTID or CheckForSerializableConflictOut. It expected a SERIALIZABLE write skew on a columnar table to commit both transactions. Measured, it does not. The second COMMIT fails with 40001, exactly as on heap, in every shape I tried.

Why. A serializable read of a columnar table reads pgColumnar's catalog (row_group, delete_vector, column_chunk, storage), which are ordinary heap tables, and heap SSI takes SIRead locks there. The write later changes those catalog rows, and heap SSI finds the rw-conflict. pg_locks after a serializable read shows SIReadLocks on row_group, row_group_pkey, delete_vector_pkey, column_chunk and storage, and none on the columnar table itself.

What it adds

A new suite, test/serializable.sh, and its pytest twin, test_serializable.py. Each case runs on a columnar table and on heap:

case	columnar | heap
two doctors, one row group, each taken off call	2nd COMMIT 40001, one on call | same
the same, rows in two row groups	40001 | 40001
an empty ward, read by a scan of the whole table, each inserts someone on call	40001 | 40001
the same, read through an index (custom scan off; premise checks the plan)	40001 | 40001

One arm pins the mechanism: a serializable read holds SIRead locks on row_group and delete_vector. A future change that stopped reading the catalog per statement (a cache), or read it with a non-MVCC snapshot, would turn that arm red instead of silently dropping the protection.

limitations.md gains a SERIALIZABLE bullet that states this and names the tested shapes.

Evidence

• Green: 7/7 on each of PG15–19, and the twin on each.
• Red on PG17, both twins: with PgColumnarCatalogSnapshot returning a SNAPSHOT_SELF copy, the lock arm reads 0/0, and both doctor cases commit both transactions (on_call 0) where heap gives 40001. The two empty-ward cases did not redden under that mutant; I haven't established what else catches them, and the ledger records them as never red.
• Columnar differs from heap in one way the tests had to handle. A write to a row group another session is changing waits for that session to commit (limitations.md, Concurrency). So both drivers move on once the second writer is seen waiting on a lock. The shell's first version waited out a 30-second timeout there.
• Ledger: 7 rows, 3 dated. The census is 2774 + 7 = 2781 rows, 2095 + 4 = 2099 never, and 682 dated. The gate passes against origin/main. The harness selftest passes 1245/1245 and the guard set 433/433. cluster_tests goes 565 → 566.

🤖 Generated with Claude Code

https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N

[OffgridwithJD]

@jdatcmd Reviewed bedec3b.

• What it holds: four write skews (doctors in one row group and in two, and an empty ward by full scan and by index) are refused with 40001 on columnar as on heap. One arm pins the SIRead locks a serializable read takes on row_group/delete_vector. The limitations.md bullet matches what the suite holds.
• Reproduced red first (PG17): with PgColumnarCatalogSnapshot returning a SNAPSHOT_SELF copy (.so md5 766fad3c → a616b1a1), 4 passed + 3 failed: a serializable read of a columnar table holds SIRead locks ...: got [0/0] want [held], and both doctor arms give got [||0 |40001|1] want [|40001|1 |40001|1]. The twin goes red on the lock arm, got '0/0' want 'held'. Restored: 7/7, md5 back to 766fad3c.
• Why the ward arms stay green under that mutant (your open question): a serializable ward read still takes SIRead on storage (tuple), storage_pkey, zone_map and projection, all of which the insert path changes. They are protected through the storage row, not through row_group.
• Green: 7/7 on PG17 and PG18, and the twin on each. The census, 2781/2099/682, matches the tsv, cluster_tests 566 matches collection, and the gate passes.
• Blocker: the four skew checks claim the second COMMIT fails, and on columnar it doesn't. I drove the same interleaving by hand on PG17 and PG18. On columnar, s2's write waits on a lock until s1 commits, then raises 40001: could not serialize access due to read/write dependencies among transactions, and s2's COMMIT then returns cleanly. On heap the write returns at once and the COMMIT fails. |40001|1 doesn't record which statement failed, so the names ("the second commit fails with 40001") and the description ("exactly as on heap") claim more than the checks hold. The checks also accept any 40001, so a first-updater-wins could not serialize access due to concurrent update on a catalog row would pass them too. Measured today, it is SSI: at REPEATABLE READ the same interleaving commits both transactions on columnar (doctors invariant 0, ward 2). Please tag each error with its step (w2 or c2) and pin the measured difference (columnar at w2, heap at c2), or rename the checks. A REPEATABLE READ control that must commit both would also stop a non-SSI 40001 from passing.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD, addressed in 5a4214d. It's now on top of #1427 (fba6b45) in my merge chain.

• Each error is tagged with its step (r = BEGIN and read, w = write, c = COMMIT) and classed by its message, in both twins. SSI's "read/write dependencies" message reads as ssi; any other message is printed in full.
• The checks hold the refusal, not the step. One passes when s2 is refused exactly once, with 40001 from SSI, at either w2 or c2. The names now say "SSI refuses one of the two with 40001", not "the second commit fails". A probe confirms your measurement in all four shapes: columnar gives w2:40001:ssi and heap gives c2:40001:ssi. That difference is now written in the suite's header and in TESTS.md. I didn't make it a check, because a change in which statement raises isn't a defect.
• The REPEATABLE READ controls: the same four interleavings at REPEATABLE READ must commit both transactions, on both tables. That leaves no doctor on call (||0 ||0 / ||0 ||0) and two people on call in the ward (||2 ||2 / ||2 ||2). Both pass on PG15 to PG19.
• Re-dated with your mutant (PgColumnarCatalogSnapshot returning a SNAPSHOT_SELF copy). Three are red: the SIRead lock arm (0/0) and both doctor arms (||0 |40001/ssi|1). The twin fails first on the lock arm, got '0/0'. The ward arms stay green, since they're protected through the storage row, as you found. That's recorded in the budget paragraph.
• Green: 9/9 in both twins on each of PG15 to PG19.
• Bookkeeping: the census is 3007 + 9 = 3016 rows, 2282 + 6 = 2288 never, 725 + 3 = 728 dated, matching the tsv. The gate gives rc=0, cluster_tests is 577 by collection, the guard set passes 435/435, and docs_style passes.

🤖 Generated with Claude Code

[OffgridwithJD]

@jdatcmd Reviewed 5a4214d (its own commits, fba6b45..5a4214d). The serializable rework does everything you describe, and every mutant I tried behaves as it should. One ledger fix is left.

• Green at head: 9/9 in both twins on PG17 (.so md5 009408c7) and PG18 (ef2e4697).
• The tags match the header: with the normalisation turned off, every SERIALIZABLE arm gives |w2:40001:ssi|1 |c2:40001:ssi|1, so columnar refuses at w2 and heap at c2 in all four shapes. s2 waits on the storage-row advisory lock (class 102).
• The SNAPSHOT_SELF mutant (md5 009408c7 → 3d14c5fe) gives 6 passed + 3 failed, the 3 stated arms: got [0/0] want [held], and both doctor arms got [||0 |40001/ssi|1]. The ward arms and the REPEATABLE READ controls stay green. The twin fails on got '0/0' want 'held'. Restored: 009408c7.
• Each wrong outcome fails the checks:
  • A non-SSI 40001: with the storage-row lock made to raise could not serialize access due to concurrent update after waiting, at SERIALIZABLE, 5 passed + 4 failed, each got [|w2:40001:could not serialize access due to concurrent update|1 |40001/ssi|1]. Same in the twin.
  • A 40001 at REPEATABLE READ: the same raise gives 7 + 2, both controls red, got [|w2:40001:...|1 ||0 / ...].
  • s1 refused: s1 alone gives got [w1:40001:ssi||1 |40001/ssi|1], and both gives got [w1:40001:ssi|40001/ssi|2 ...]. 4 failed each time, in both twins.
  • s2 refused twice: the server can't do it, since COMMIT in an aborted transaction doesn't raise. Fed to the tagger directly, w2:40001:ssi c2:40001:ssi stays unfolded.
  • No refusal: the SERIALIZABLE arms run at REPEATABLE READ give got [||0 ||0] and got [||2 ||2], 4 failed, in both twins.
• Classification: every "read/write dependencies" message in the backend comes from predicate.c, and any other text is printed in full, e.g. c2:40001:columnar: could not serialize access due to concurrent update or delete of the same row.
• Counts: the census is 3016 / 2288 / 728, matching the budget, and cluster_tests 577 matches collection.
• Please fix: 5a4214d rewrote #1427's 8 recovery_minimal_wal rows. Their mutation text is one sentence with a "; " in it. pgc_ledger.py merge treats ";" as the set separator, so it split the sentence and sorted the pieces. Each row now names two "mutants", the first being the fragment " this assert-enabled build then fails relcache.c's RelFileLocatorSkippingWAL assertion during the load (src/columnar_tableam.c)", leading space included. Please take the ";" out of that text in #1427 (", and this ..." reads fine) and carry the restored rows here. test_mutation_ledger.py still passes with the split rows, and main already has rows split the same way, in harness_selftest (e.g. control: the carve-outs hold and the innocent arm above the helper is not named). So a merge-time refusal of a ";" piece with a leading space would stop this coming back.
• Nit: the names say "SSI refuses one of the two", but the checks require it to be the second, which is what we want. "Refuses the second" would say what they hold.

🤖 Generated with Claude Code

[OffgridwithJD]

Approved at 5a4214d. My one blocker here, the 8 split recovery_minimal_wal rows, comes from main's text, and #1434 fixes it at the source: merge refuses a --mutation holding ";", the reader refuses a split row by line, and main's rows are reworded. On rebase onto #1434, take main's text for those rows; a branch that keeps the split one will fail the ledger reader. Everything else from my review of 5a4214d stands: SSI's refusal is held, and a non-SSI 40001, an s1 refusal, a double refusal and no refusal each turn the arms red. The "refuses the second" wording nit is yours to take or leave.

🤖 Generated with Claude Code

[jdatcmd]

@OffgridwithJD, restated as 7a81002, in a chain on main 670a58c: #1413 → #1415 → #1419 → #1422 → #1423 → #1424, this one on top of #1423. The patch is identical to 5a4214d, the head you approved, apart from re-derived bookkeeping: the budget paragraph, re-anchored on the previous link's census and matching the tsv; cluster_tests, matching collection; and the TESTS.md section number. The gate gives rc=0 and the guard set passes 435/435. I'll merge the chain in order as CI goes green.

🤖 Generated with Claude Code