test: update_conc gets its pytest twin, and a savepoint arm that reaches the subtransaction discard (#1341 item 24)

test/check_ledger_budget.txt

@@ -15,7 +15,7 @@
 #
 #   Adding a check to a suite that is already covered does not move it, which is
 #   what makes it safe to bound.
-suites_not_covered 224
+suites_not_covered 222
 #
 # checks_never_observed_red -- A CENSUS. NOT a ceiling, and it must not become
 #   one.
@@ -2815,4 +2815,43 @@ suites_not_covered 224
 #     never         2602 - 1     = 2601
 #     dated          872 + 1     = 873
 #     2601 + 873 == 3474
-checks_never_observed_red 2601
+#
+# #1341, unique_conc seeded, with its twin (OffgridwithJD): 2601 -> 2607, and
+# suites_not_covered 224 -> 223. The suite had no rows: its local check() recorded
+# through pgc_record with no accounting line, and it now uses lib.sh's recorders.
+# It enters with all 33 checks from five-major runs (33/33 on each of 15-19). 27 are
+# dated by six mutations of columnar_unique.c: the per-key lock never taken (the 18
+# serialize, unique_violation and one-row arms), the lock taken with the setting off
+# (1a, 2a), every key in one bucket (3, and 4's differing key), only the first key
+# column hashed (4's differing key), the partial predicate ignored (5 outside it),
+# and a NULL key locked under NULLS DISTINCT (6). The "does not block" arms could not
+# fail before this change, so their dates are new evidence. Six arrive never: the two
+# bucket-count premises, the same-statement duplicate (7, the btree's own check), and
+# the aborted first inserter (8), which waits on the first's xid with or without the
+# key lock.
+#
+#     ledger rows   3474 + 33 = 3507
+#     never         2601 +  6 = 2607
+#     dated          873 + 27 = 900
+#     2607 + 900 == 3507
+#
+# #1341, update_conc seeded, with its twin (OffgridwithJD): 2607 -> 2621, and
+# suites_not_covered 223 -> 222. The suite had no rows: its local check() recorded
+# through pgc_record with no accounting line, and it now uses lib.sh's recorders.
+# It enters with all 28 checks from five-major runs (28/28 on each of 15-19). 14 are
+# dated by seven mutations: in columnar_row_lock.c, the flush-time serialization
+# skipped, the committed-delete conflict never raised, and the re-read taken from
+# the statement snapshot (each the same 10 duplicate-row and 40001 arms at RC and
+# RR, update-vs-delete, delete-vs-update and multirow); the row lock skipped during
+# the UPDATE of a table with a unique index (the unique/RC arm); the self-modify
+# guard removed (the self-update arm); and in pgcolumnar_subxact_callback, either
+# discard skipped (the new failed-statement savepoint arms, both). Fourteen arrive
+# never: the seven heap-oracle arms, the two SERIALIZABLE guards, which held before
+# the row lock existed, the two serial controls, the scan-order premise, and the
+# completed-UPDATE savepoint arm's two, which neither discard alone reddens.
+#
+#     ledger rows   3507 + 28 = 3535
+#     never         2607 + 14 = 2621
+#     dated          900 + 14 = 914
+#     2621 + 914 == 3535
+checks_never_observed_red 2621

test/pytest/TESTS.md

@@ -179,6 +179,8 @@ behaviour, the source of that number is named.
 - [131. test_type_matrix.py: types the differential matrix never stored answer as heap](#131-test_type_matrixpy-types-the-differential-matrix-never-stored-answer-as-heap)
 - [132. test_iceberg_data_files.py: a delete entry in a data manifest is refused](#132-test_iceberg_data_filespy-a-delete-entry-in-a-data-manifest-is-refused)
 - [133. test_native_skip.py: a type binary-coercible to its opclass's type prunes](#133-test_native_skippy-a-type-binary-coercible-to-its-opclasss-type-prunes)
+- [134. test_unique_conc.py: concurrent inserters of one unique key serialize](#134-test_unique_concpy-concurrent-inserters-of-one-unique-key-serialize)
+- [135. test_update_conc.py: two writers of one row serialize](#135-test_update_concpy-two-writers-of-one-row-serialize)
 
 ## 1. How to read a test in here
 
@@ -7221,3 +7223,46 @@ A partial pytest twin of `test/native_skip.sh` (#1433). It ports only the arms t
 | --- | --- |
 | `test_a_range_on_a_coercible_type_prunes` | a range on inet, cidr, a domain over cidr, text, varchar and a domain over varchar returns heap's rows and removes chunk groups |
 | `test_equality_on_a_coercible_type_prunes` | equality on cidr and on varchar, which also takes the bloom probe, returns heap's row and removes chunk groups |
+
+## 134. test_unique_conc.py: concurrent inserters of one unique key serialize
+
+The pytest twin of `test/unique_conc.sh` (#5, ported for #1341), every check by the shell's name. A columnar row's btree entry is written at once while its data waits for the stripe to flush, so a second inserter of the same key could not see the first's row and would insert a duplicate. The table AM takes a transaction-scoped advisory lock per unique key, hashed with the key type's default hash. So the second inserter waits for the first to commit and gets 23505, asserted by SQLSTATE and the index's name. T1 is held mid-INSERT, its key row indexed and unflushed, by a second row that waits in `cq_block` on a lock a holder session has. Every barrier polls `pg_stat_activity` for a lock wait.
+
+The shapes: one key, with the lock off in T2 as the control that shows the bug; keys equal under the opclass with different bytes (numeric 1.0 and 1.00, and citext by case); different keys, which must not wait; a multi-column key, equal and differing in its second column; a partial index, inside and outside its predicate; NULL keys under NULLS DISTINCT and NULLS NOT DISTINCT; a duplicate within one statement; and a first inserter that aborts. A private cluster runs `pgcolumnar.unique_lock_buckets` at 100003, which is fixed at server start, so unrelated keys do not share a bucket.
+
+### Every test
+
+| test | what it holds |
+| --- | --- |
+| `test_the_bucket_count_is_fixed_at_server_start` | `SET` of the bucket count is refused 55P02, and the cluster runs 100003 |
+| `test_one_key_serializes_and_the_lock_is_what_does_it` | with the lock off in T2 both inserts commit; with it on T2 waits, gets 23505, and one row is left |
+| `test_keys_equal_under_the_opclass_serialize` | numeric 1.0 and 1.00: the same pair, lock off and on |
+| `test_citext_keys_equal_by_case_serialize` | citext 'ABC' and 'abc' serialize and conflict; unrunnable without citext |
+| `test_different_keys_do_not_block_each_other` | T2's different key returns at once, and both rows are kept |
+| `test_a_multi_column_key` | (1,2) twice serializes and conflicts; (5,6) and (5,7) do not wait |
+| `test_a_partial_unique_index` | inside the predicate the keys serialize and conflict; outside it neither waits and both rows are kept |
+| `test_null_keys` | two NULLs never wait under NULLS DISTINCT; under NULLS NOT DISTINCT they serialize and conflict |
+| `test_a_duplicate_within_one_statement` | one INSERT of the same key twice raises 23505 and inserts nothing |
+| `test_an_aborted_first_inserter` | T2 waits behind T1's flushed, uncommitted row; T1 rolls back and T2's insert succeeds |
+
+## 135. test_update_conc.py: two writers of one row serialize
+
+The pytest twin of `test/update_conc.sh` (#5, ported for #1341), every check by the shell's name. A columnar UPDATE marks the old row deleted and appends a new one, so without a lock on the row identity two transactions updating one row each kept their own new version, and the row was duplicated. `columnar_row_lock.c` takes a transaction-scoped advisory lock per row at the delete-vector flush, re-reads the latest committed state, and gives the loser a retryable 40001. A table with a unique index takes the lock during the UPDATE, so the unique check cannot answer 23505 first (#1421).
+
+Each concurrent case runs one interleaving on a heap table, the oracle, and on a columnar one: s1 writes and holds its transaction, s2's statement waits on a lock, which a poll of `pg_stat_activity` confirms, and s1 commits. At READ COMMITTED a heap re-applies s2's update where columnar fails it, so the columnar value is compared with the heap's only at REPEATABLE READ. A private database; each session runs at a 30s `lock_timeout`, as the shell's cluster does.
+
+### Every test
+
+| test | what it holds |
+| --- | --- |
+| `test_update_vs_update_read_committed` | one row is left and the loser gets 40001; the heap composes both updates into one row |
+| `test_update_vs_update_repeatable_read` | count and values match the heap, and both losers get 40001 |
+| `test_update_vs_update_serializable` | the count matches the heap and the loser gets 40001, which held before the row lock too |
+| `test_update_vs_delete_read_committed` | the heap's DELETE removes the updated row; columnar's loser gets 40001 |
+| `test_delete_vs_update_read_committed` | the committed DELETE stands on both, and columnar's loser gets 40001 |
+| `test_a_row_updated_twice_by_one_statement` | `UPDATE ... FROM (VALUES (1), (1))` leaves one row on both |
+| `test_overlapping_multi_row_updates` | 41 overlapping rows: 200 rows remain, and the loser gets 40001 |
+| `test_serial_updates_control` | the same two updates without overlap leave one row with the last writer's value |
+| `test_a_covering_unique_index` | with a unique index the loser still gets 40001, not 23505; the heap composes |
+| `test_a_savepoint_rolled_back` | an UPDATE rolled back to a savepoint leaves the pre-savepoint value, as on the heap |
+| `test_a_statement_failing_inside_a_savepoint` | an UPDATE that fails partway inside a savepoint leaves nothing behind: its buffered mark and row are discarded |

test/pytest/expected_tests.txt

@@ -733,4 +733,8 @@ guard_tests 438
 # listed controls and three parametrized refusals), by collection over the cluster files.
 # cluster_tests 609 -> 617 for #1433's test_native_skip.py, eight tests (six ranges and
 # two equalities, parametrized), by collection over the cluster files on this tree.
-cluster_tests 617
+# cluster_tests 617 -> 627 for #1341's test_unique_conc.py, the twin of unique_conc.sh:
+# ten tests, by collection over the cluster files on this tree.
+# cluster_tests 627 -> 638 for #1341's test_update_conc.py, the twin of update_conc.sh:
+# eleven tests, by collection over the cluster files on this tree.
+cluster_tests 638

test/pytest/test_compare_to_bash.py

@@ -165,6 +165,8 @@
             "timeseries_recipes",
             "ttl_expire",
             "type_matrix",
+            "unique_conc",
+            "update_conc",
             "vacuum_lock_privilege",
             "validity_elision",
             "vm_privilege",