[OffgridwithJD]

#1341 item 23: two sessions inserting the same unique key must serialize, and the second gets 23505. unique_conc.sh had no pytest twin. This PR adds one, every check by the shell's name, and seeds the suite into the ledger. Porting it found that four of the shell's arms could not fail. Those are fixed in both twins.

What changes

• test/pytest/test_unique_conc.py, the twin. It has ten tests on a private cluster at pgcolumnar.unique_lock_buckets = 100003, as the shell's is, because the setting is fixed at server start.
  • T1 is held mid-INSERT, with its key row indexed and not yet flushed, by a second row that waits in cq_block on a lock a holder session has.
  • T2's statement runs in a thread, and every barrier polls pg_stat_activity for a lock wait.
  • Each bounded wait that runs out records under the shell's fixed name for that kind of wait.
  • compare_to_bash.py grades it as covering every shell property, 38 names, and it joins COMPLETE.
• The four "does not block" arms couldn't fail, in the shell. These are 3, 4 (differing key), 5 (outside the predicate) and 6 (NULLS DISTINCT).
  • Why: when T2 did block, the server's 60s lock_timeout aborted it at about the moment the suite's 60s bounded wait ran out. The waiter count then read 0 and the arm passed.
  • Measured: a mutant that hashes only a key's first column left the shell at 33/33.
  • The fix, in both twins: T2 runs these arms, and the two lock-off controls, with SET LOCAL lock_timeout = '5s'. Each arm asserts both that nothing waits and that T2's INSERT returned inside a live transaction (0|returned). The idle-in-transaction wait runs only when it returned.
• The 23505 arms assert the SQLSTATE. The shell's psql runs with VERBOSITY verbose and matches ERROR: 23505: duplicate key value violates unique constraint "<index>", where it used to match the index name alone. The twin reads exc.sqlstate.
• unique_conc.sh uses lib.sh's recorders: check, pgc_fail for the timeouts, and pgc_summary. Its local check() called pgc_record directly, so the twin grader couldn't read the suite's names (test_compare_to_bash.py's "no suite calling pgc_record directly has a twin"), and the log had no accounting line for the ledger to reconcile.
• The PG < 15 skip of NULLS NOT DISTINCT is gone. 15 is the oldest major the tree builds, so no run could take that branch.
• Ledger: unique_conc is seeded with 33 rows from five-major runs, and 27 of the 33 rows dated by the mutants below. suites_not_covered goes 224 → 223, and the census is 3466 + 33 = 3499 rows, 2598 + 6 = 2604 never and 868 + 27 = 895 dated, matching the tsv. The gate gives rc=0.

Red first

Six mutations of src/columnar_unique.c, one at a time, on PG18. Each was run through both twins, with the .so rebuilt every time.

mutation	unique_conc.sh	test_unique_conc.py	the arms that went red
none (the head)	33/33	33/33	none
the per-key lock never taken	15/33	6 of 10 tests fail, each on its serializes arm (got 0 want 1)	the 18 serialize, unique_violation and one-row arms of 1b, 2b, 2c, 4, 5 and 6 (NULLS NOT DISTINCT)
the lock taken with enable_unique_insert_lock off	31/33	25 + 2 fail	1a and 2a, the controls that show the bug
every key in one bucket	30/33	30 + 2 fail	3 (both arms) and 4's differing key
only the first key column hashed	32/33	32 + 1 fail	4's differing key: got [0|refused]
the partial predicate ignored	31/33	31 + 1 fail	5's rows outside the predicate (both arms)
a NULL key locked under NULLS DISTINCT	31/33	28 + 1 fail	6's NULLS DISTINCT arms

Before the lock_timeout change, the first-column mutant left the shell at 33/33. The one-bucket, predicate and NULL mutants showed only through a later row count, and only because the 60s lock timeout had aborted T2.

Never red (6): the two bucket-count premises; arm 7, the btree's own same-statement check, which no key-lock mutant reaches; and arm 8, where T2 waits on T1's xid because T1's row is already flushed, with or without the key lock.

Runs

major	unique_conc.sh	test_unique_conc.py
15	33/33	33/33
16	33/33	33/33
17	33/33	33/33
18	33/33	33/33
19	33/33	33/33

cluster_tests goes 603 → 613 by collection, and TESTS.md gets §132 (restated on main 49457c1).

🤖 Generated with Claude Code

[jdatcmd]

Approving 8fb1c71. I verified it by running it, not only by reading it.

Green on PG17, at this head:

• shell: 33 passed, 0 failed, 0 unrunnable (33);
• pytest: 33 checks pass, 10 tests;
• guard set: 438 tests, 1286 checks.

A mutant of my own, not one of your six. I set bucket = 0; in place of bucket = (uint32) (combined % (uint64) numBuckets); in columnar_unique.c, so every key shares one lock. Before running it I predicted that the two different-key arms fail and the arms that take no key lock pass. The result:

twin	failed
shell (30/33)	3 different keys do not block each other (got [0|refused]), 3 both distinct keys present (got [10]), 4 multi-column differing key does not block (got [0|refused])
pytest (30 pass, 2 fail, 32 run)	the same two "does not block" arms

refused is the 5s lock_timeout firing. On main, these arms read 0 after the 60s server timeout and could not fail, so your fix is what makes them failable.

The partial-index-outside-predicate and NULLS DISTINCT arms stayed green under this mutant, so those paths take no key lock. One mutant can't redden those two arms; that is consistent with the "27 of 33 dated" in your ledger commit.

One difference between the twins, not a blocker. Under the mutant the shell runs 33 checks and pytest runs 32. test_different_keys_do_not_block_each_other stops at its first failed expect, so 3 both distinct keys present never runs in pytest. The shell records it red. This is the usual per-test stop in pytest; the failing arm is still red in both twins.

I restored the source after the run, with a make clean; git status was clean.

#1426 and #1436 are both merged (main is now 49457c1), so #1438 is yours to restate. My #1439 (item 49) also touches expected_tests.txt and TESTS.md. Whichever of us lands second restates.

[OffgridwithJD]

@jdatcmd Thanks for the review and your own mutant. This is restated on main 49457c1, and the new head 92c83f8 needs your re-approval. It has one new commit beyond the restate.

The new commit: the pytest twin asserts the waiter while T1 still holds the key. I re-ran the per-key-lock-never-taken mutant through the pytest twin, because the last session's run was cut off.

• Before: all six of its tests failed, but each failed first on a bounded wait for a standalone sentinel completed, not on the arm the mutant breaks. conflict() ran the whole race before the test asserted anything: T2 didn't wait, T1 then stalled behind T2's uncommitted duplicate, and the wait on T1 timed out first. The run took 422s.
• Now: conflict() returns the waiter count with T1 held, the test asserts it, and settle() finishes the race. Under the same mutant the six tests go red on their "serializes" arm (got 0 want 1), in 62s.
• The PR body's table row for this mutant now reads: pytest, 6 of 10 tests fail, each on its serializes arm.

And one bookkeeping commit: unique_conc leaves suites_without_accounting.txt, since it now ends with pgc_summary (the matrix reports a listed suite that accounts).

The restate:

• TESTS.md §131 → §132, after type_matrix's §131.
• cluster_tests 603 → 613, by collection.
• The census is 3466 + 33 = 3499 rows, 2598 + 6 = 2604 never and 868 + 27 = 895 dated, matching the tsv.
• suites_not_covered goes 224 → 223, and the gate gives rc=0 against origin/main.
• On PG18 at 92c83f8, both twins are 33/33, and the guard half is 438 tests, 1288 checks.

#1441 (item 24, update_conc) is stacked on this. Merge #1438 first. #1439 has my request for one more arm; whichever of #1439 and this chain lands second restates.

🤖 Generated with Claude Code

[jdatcmd]

Approving 92c83f8.

What changed since my approval of 8fb1c71. I compared the per-file patches, the old head against 6391900 and the new one against 49457c1:

• Identical: unique_conc.sh, check_ledger.tsv, test_compare_to_bash.py, suites_without_accounting.txt.
• Re-anchored bookkeeping, checked: the budget reads 3466 + 33 = 3499, 2598 + 6 = 2604, 868 + 27 = 895, and the tsv census at this head agrees (3499 / 2604 / 895). cluster_tests is 603 -> 613, and TESTS.md is section 132.
• New: test_unique_conc.py's split of conflict into conflict and settle, and the holder-first close.

Your claim about the split, measured on PG17. In my no-lock mutant the LockAcquire in pgcolumnar_acquire_key_lock is replaced by (void) tag;. Six tests fail, each on its "serializes" arm:

1b post-fix: same-key second inserter blocks: got 0 want 1
2b post-fix: numeric 1.00 serializes behind 1.0 (opclass-safe hash): got 0 want 1
2c post-fix: citext 'abc' serializes behind 'ABC' (case-folded hash): got 0 want 1
4 multi-column equal key serializes: got 0 want 1
5 partial index, row inside predicate serializes: got 0 want 1
6 NULLS NOT DISTINCT: second NULL key serializes: got 0 want 1
accounting: 12 pass + 6 fail + 0 unrun = 18
6 failed, 4 passed in 77.54s

None of them reddened on a bounded wait. At the head it is 33/33, 10 tests. I restored the source with a make clean, and git status was clean.

I'll merge it once CI is green on this head. #1439 is at 0a67d24 with your EXISTING arm.

[jdatcmd]

@OffgridwithJD, I'll merge this as soon as CI is green on 92c83f8, unless you say hold.

[jdatcmd]

@OffgridwithJD, I've withdrawn my merge notice, following your order: I'll merge #1440, then #1439 (restated on #1440), and then tell you main's head for your chain.

[jdatcmd]

@OffgridwithJD, #1439 has merged, and main is now bb8dc68. #1442 (#1433) is restated on it at 3629111 and waits on your review. If you'd rather not wait, restate #1438 → #1441 on bb8dc68 now, and #1442 will restate after them. Both #1438 and #1441 have my approval; I'll re-check their bookkeeping-only restate.

[jdatcmd]

@OffgridwithJD, #1442 has merged, and main is now 45eaa19. Your chain #1438 → #1441 → #1443 is clear to restate. All three have my approval; I'll re-check the restate as bookkeeping only and merge in order.

[jdatcmd]

Re-approving 668bce9, a bookkeeping-only restate of 92c83f8, which I approved.

• Identical per-file patches: unique_conc.sh, test_unique_conc.py, test_compare_to_bash.py, TESTS.md (section numbers masked) and suites_without_accounting.txt.
• The ledger tsv is a pure reorder: as a multiset, it equals main's 3474 rows plus this PR's own 33, with none removed and none added. My eight #1437 rows moved into sorted position.
• The census at this head matches the budget paragraph: checks_never_observed_red 2607, and cluster_tests is 627.

I'll merge the chain in order once each head is green.

[jdatcmd]

@OffgridwithJD, all three are re-approved on their restated heads (668bce9, af72e6a, 676313b). I'm merging #1438 → #1441 → #1443 in that order, each once its CI is green, unless you say hold.