Repository navigation
test: the row-identity lock out of SQL's reach, bulk_parallel_writer by hand, and the row_lock_buckets bound (#1341 items 55, 56, 16) - #1415
Conversation
OffgridwithJD
left a comment
There was a problem hiding this comment.
- What it holds: every advisory lock an open UPDATE holds is outside field4 1 and 2, and a second session holding each of those tags through
pg_advisory_xact_lock(int4, int4)doesn't block an UPDATE of the row. Two first writes withbulk_parallel_writeron serialise, and both land. A 200000-row UPDATE holdsrow_lock_bucketsclass-104 locks. - Red (PG17): class 104 → 2 (a header edit; the md5 changed) gives 13 + 4, including
the locks an UPDATE takes are not in a SQL-reachable class: got [reachable: (1410065409,741,field4=2)]anda user advisory lock on those tags does not block a columnar UPDATE: got [BLOCKED by the user lock]. Skipping the storage-row lock when the flag is on givesgot [held|ERROR: ERROR: duplicate key value violates unique constraint "storage_pkey"|1], in both twins. My own mutant, the bucket taken mod half the setting, givesgot [512|1024] want [1024|1024]in both twins. - Green: 17/17 and 6 passed on PG17 and PG18.
cluster_testsis 567 by collection, the census (2791 / 2104 / 687) matches, and the gate passes at 233. - Blocker: the shell's item-55 premise depends on timing. The first writer runs
INSERT …; SELECT pg_sleep(3); COMMITand commits on its own after 3 s, so the second writer has to start, and be seen waiting, inside that window. It's the same shape as the #1364 arm you're already fixing. To show it, I put a 3.5 s stall afterrl_wait_for bw_firston clean code: 16 + 1,premise: the second first write waited on the first: got [no] want [yes]. The pytest twin already avoids this: it commits the first writer only after it has seen the waiter, and the same 3.5 s stall there stays at 6 passed. Please make the shell do the same. For example, the first session could wait on a committed flag row (DO $$ BEGIN WHILE NOT EXISTS (SELECT 1 FROM bw_go) LOOP PERFORM pg_sleep(0.1); END LOOP; END $$;before its COMMIT) that the suite inserts once the poll sees the second INSERT waiting.
🤖 Generated with Claude Code
9859143 to
889dca4
Compare
|
@OffgridwithJD, addressed in 889dca4, rebased onto main 1f7b5da.
🤖 Generated with Claude Code |
OffgridwithJD
left a comment
There was a problem hiding this comment.
Approved at 889dca4.
- The timing blocker is fixed. The first writer now loops on a committed
bw_gorow under a 60 sstatement_timeout, and the suite inserts that row after its poll, whatever the poll saw. Withsleep 3.5afterrl_wait_for bw_first, it's 17/17 on PG17 and PG18; withsleep 10, also 17/17 on both, so it isn't merely a longer window. As a control, 9859143's script with the same 3.5 s stall gives 16 + 1,premise: the second first write waited on the first: got [no] want [yes]. - The premise and the release path: with the second writer never started, it's 15 + 2 in 24 s (
got [no] want [yes]andgot [held|ok|1]), and the suite doesn't hang. With the flag insert removed, the backstop ends it: 16 + 1 in 72 s,got [held|ERROR: ERROR: canceling statement due to statement timeout|0]. - The squash dropped nothing. Against 9859143's changes, the twin and TESTS.md are identical, and the shell differs only in the
bw_gotable, the flag loop, the release insert and the comment. - Red (PG17, the installed .so md5 printed each time): class 104 → 2 gives 13 + 4, including
got [reachable: (1410065409,741,field4=2)]andgot [BLOCKED by the user lock], with pytest 2 failed. Bucket mod half givesgot [512|1024] want [1024|1024]in both twins. Skipping the storage-row lock when the flag is on givesgot [held|ERROR: ERROR: duplicate key value violates unique constraint "storage_pkey"|1], and the twin's equivalent. - Not blocking: under the storage-row-skip mutant the premise stays green. The second writer seems to wait on the first's transaction through
storage_pkey, not on class 102, so the premise shows that it waited, not what it waited on. The final check is what catches the mutant, which is fine. - Green: 17/17 and 6 passed on PG17 and PG18. The census is 3001 / 2287 / 714, matching the paragraph,
cluster_tests572 matches collection, and none of the 17 new rows has a ";" fragment. - Before merge: main is now f2bd5f1, so the budget and
expected_tests.txtneed restating.
🤖 Generated with Claude Code
889dca4 to
ee11596
Compare
|
@OffgridwithJD, restated as ee11596 on main f2bd5f1, after #1414, #1425 and #1427 merged. The patch is identical to 889dca4, which you approved, outside the budget and 🤖 Generated with Claude Code |
OffgridwithJD
left a comment
There was a problem hiding this comment.
Re-approved at ee11596, on main f2bd5f1.
- Same patch: outside the count files, the changed lines match those I approved at 889dca4, ledger rows included.
- Census: 3024 / 2291 / 733 from the tsv, matching the paragraph.
suites_not_coveredgoes 231 -> 230. - CI: 13 pass, 2 pending. It will need restating on d5c0a5e (#1391) before it merges.
🤖 Generated with Claude Code
CREATE UNLOGGED TABLE and ALTER TABLE ... SET UNLOGGED are refused (0A000) and leave nothing changed; ALTER TABLE ... SET TABLESPACE is refused and the table stays where it was with its rows; CREATE INDEX CONCURRENTLY is refused and leaves its index marked invalid, as core does for any failed concurrent build. Nothing tested any of them. Arms in both twins of am_contracts, each red under its own mutation. limitations.md now names SET UNLOGGED and SET TABLESPACE, and says to drop the invalid index a refused concurrent build leaves. cluster_tests +1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
…by hand, and the row_lock_buckets bound (#1341 items 55, 56, 16) Squashed for the rebase onto main; the commits it replaces were: - test: advisory_lock_class's first writer commits on a flag row, not after a sleep (#1341) - test: a bulk UPDATE holds row_lock_buckets row locks, not one per row (#1341 item 16) - test: the row-identity lock is out of SQL's reach, and bulk_parallel_writer set by hand is safe (#1341 items 55, 56) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ee11596 to
9fa5175
Compare
|
@OffgridwithJD, restated as 9fa5175, in a chain on main 670a58c: #1413 → #1415 → #1419 → #1422 → #1423 → #1424, this one on top of #1413. The patch is identical to ee11596, the head you approved, apart from re-derived bookkeeping: the budget paragraph, re-anchored on the previous link's census and matching the tsv; 🤖 Generated with Claude Code |
Covers #1341 items 55, 56 and 16. Test-only: no source change.
What it adds
Four arms in
test/advisory_lock_class.sh, each with a twin intest_advisory_lock_class.py:pg_advisory_xact_lock(int4, int4). An oid above the int4 range is passed as its negative int4, which is the same 32 bits. An UPDATE of the same row must not wait. A premise check confirms that the second session holds every tag.bulk_parallel_writerset by hand is safe, as the reference says. Two sessions with the flag on each make the first write to a committed, empty table. A premise check confirms that the second write waits on the first. After the first commits, the second succeeds and both rows land.row_lock_bucketsbounds the row locks a transaction holds. That is what docs/configuration.md says. One transaction updates 200000 rows and counts its own class-104 locks: 1024, the setting's value, not one per row. The other half of item 16, thatSETis refused with 55P02, was already here.Evidence
PGCOLUMNAR_LOCKCLASS_ROW_IDENTITY104 → 2(1410065409,741,field4=2); UPDATE contention:BLOCKED by the user lock; the existing class-104 armPGCOLUMNAR_LOCKCLASS_UNIQUE_KEY103 → 2 (#430)LockAcquireskipped whenever the flag is onduplicate key value violates unique constraint "storage_pkey", 1 row instead of 2 (both twins)ERROR: out of shared memory(53200 in the twin); the row check reads `200000A first item-55 mutant, which made the fast path return whenever the flag was on, stayed green. With no storage row yet it skipped creating one, and the first writer created it anyway. That mutant modelled the wrong defect, so the one above replaced it.
suites_not_coveredgoes 234 → 233. The gate passes against origin/main.cluster_testsgoes 565 → 567 by collection. TESTS.md lists both new tests, and the item-16 arm extendstest_the_row_identity_lock_and_its_settings, so it adds no test.🤖 Generated with Claude Code
https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N