Skip to content

test: the row-identity lock out of SQL's reach, bulk_parallel_writer by hand, and the row_lock_buckets bound (#1341 items 55, 56, 16) - #1415

Merged
jdatcmd merged 2 commits into
mainfrom
test/1341-row-lock-class
Oct 15, 2026
Merged

jdatcmd merged 2 commits into
mainfrom
test/1341-row-lock-class

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Covers #1341 items 55, 56 and 16. Test-only: no source change.

What it adds

Four arms in test/advisory_lock_class.sh, each with a twin in test_advisory_lock_class.py:

  • Item 56: the row-identity lock (class 104) cannot be reached from SQL. An UPDATE is held open, and every advisory lock it took is read: none may be in field4 1 or 2, the only classes SQL can produce. A second session then takes each of those exact tags with pg_advisory_xact_lock(int4, int4). An oid above the int4 range is passed as its negative int4, which is the same 32 bits. An UPDATE of the same row must not wait. A premise check confirms that the second session holds every tag.
  • Item 55: bulk_parallel_writer set by hand is safe, as the reference says. Two sessions with the flag on each make the first write to a committed, empty table. A premise check confirms that the second write waits on the first. After the first commits, the second succeeds and both rows land.
  • Item 16: row_lock_buckets bounds the row locks a transaction holds. That is what docs/configuration.md says. One transaction updates 200000 rows and counts its own class-104 locks: 1024, the setting's value, not one per row. The other half of item 16, that SET is refused with 55P02, was already here.

Evidence

  • Green: 17/17 on each of PG15–19, and the twin 6/6 on each. The harness selftest passes 1245/1245 on PG17.
  • Red on PG17, one mutant at a time:
mutant what went red
PGCOLUMNAR_LOCKCLASS_ROW_IDENTITY 104 → 2 reachable class: (1410065409,741,field4=2); UPDATE contention: BLOCKED by the user lock; the existing class-104 arm
PGCOLUMNAR_LOCKCLASS_UNIQUE_KEY 103 → 2 (#430) INSERT reachable class; INSERT contention
the storage-row LockAcquire skipped whenever the flag is on duplicate key value violates unique constraint "storage_pkey", 1 row instead of 2 (both twins)
the row bucket taken as the raw key, so one lock per row the bulk UPDATE fails: ERROR: out of shared memory (53200 in the twin); the row check reads `200000

A first item-55 mutant, which made the fast path return whenever the flag was on, stayed green. With no storage row yet it skipped creating one, and the first writer created it anyway. That mutant modelled the wrong defect, so the one above replaced it.

  • Ledger: advisory_lock_class is seeded with all 17 checks, 8 of them dated. The census is 2774 + 17 = 2791 rows, 2095 + 9 = 2104 never, and 679 + 8 = 687 dated. suites_not_covered goes 234 → 233. The gate passes against origin/main.
  • cluster_tests goes 565 → 567 by collection. TESTS.md lists both new tests, and the item-16 arm extends test_the_row_identity_lock_and_its_settings, so it adds no test.

🤖 Generated with Claude Code

https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N

@jdatcmd jdatcmd changed the title test: the row-identity lock is out of SQL's reach; bulk_parallel_writer set by hand is safe (#1341 items 55, 56) test: the row-identity lock out of SQL's reach, bulk_parallel_writer by hand, and the row_lock_buckets bound (#1341 items 55, 56, 16) Oct 15, 2026

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdatcmd Reviewed 9859143.

  • What it holds: every advisory lock an open UPDATE holds is outside field4 1 and 2, and a second session holding each of those tags through pg_advisory_xact_lock(int4, int4) doesn't block an UPDATE of the row. Two first writes with bulk_parallel_writer on serialise, and both land. A 200000-row UPDATE holds row_lock_buckets class-104 locks.
  • Red (PG17): class 104 → 2 (a header edit; the md5 changed) gives 13 + 4, including the locks an UPDATE takes are not in a SQL-reachable class: got [reachable: (1410065409,741,field4=2)] and a user advisory lock on those tags does not block a columnar UPDATE: got [BLOCKED by the user lock]. Skipping the storage-row lock when the flag is on gives got [held|ERROR: ERROR: duplicate key value violates unique constraint "storage_pkey"|1], in both twins. My own mutant, the bucket taken mod half the setting, gives got [512|1024] want [1024|1024] in both twins.
  • Green: 17/17 and 6 passed on PG17 and PG18. cluster_tests is 567 by collection, the census (2791 / 2104 / 687) matches, and the gate passes at 233.
  • Blocker: the shell's item-55 premise depends on timing. The first writer runs INSERT …; SELECT pg_sleep(3); COMMIT and commits on its own after 3 s, so the second writer has to start, and be seen waiting, inside that window. It's the same shape as the #1364 arm you're already fixing. To show it, I put a 3.5 s stall after rl_wait_for bw_first on clean code: 16 + 1, premise: the second first write waited on the first: got [no] want [yes]. The pytest twin already avoids this: it commits the first writer only after it has seen the waiter, and the same 3.5 s stall there stays at 6 passed. Please make the shell do the same. For example, the first session could wait on a committed flag row (DO $$ BEGIN WHILE NOT EXISTS (SELECT 1 FROM bw_go) LOOP PERFORM pg_sleep(0.1); END LOOP; END $$; before its COMMIT) that the suite inserts once the poll sees the second INSERT waiting.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd force-pushed the test/1341-row-lock-class branch from 9859143 to 889dca4 Compare October 15, 2026 18:16
@jdatcmd

jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, addressed in 889dca4, rebased onto main 1f7b5da.

  • The first writer commits on a flag row, not after a sleep. After its INSERT it loops WHILE NOT EXISTS (SELECT 1 FROM bw_go) ... pg_sleep(0.1), with a 60 s statement_timeout as a backstop. The suite inserts the committed bw_go row once its poll has looked for the second writer waiting. It inserts that row whatever the poll saw, so the first writer is always released. This is the shape you suggested, and the one the twin already uses.
  • Your stall, both ways (PG17): with sleep 3.5 after rl_wait_for bw_first:
    • the previous head gives 16 + 1, premise: the second first write waited on the first: got [no] want [yes], as you measured;
    • this head gives 17/17.
  • Green: 17/17 on each of PG15 to PG19, and the twin's 6 tests pass on PG17.
  • Squashed: the PR's three commits are one now, so the budget paragraph could be restated once on main rather than three times. The tree before the rebase was identical to the three-commit head. The commit message lists the replaced commits.
  • Bookkeeping, on main: the census is 2984 + 17 = 3001 rows, 2278 + 9 = 2287 never, 706 + 8 = 714 dated, matching the tsv. suites_not_covered goes 231 -> 230, and the gate gives rc=0. cluster_tests is 572 by collection, and the guard set passes 435/435.

🤖 Generated with Claude Code

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at 889dca4.

  • The timing blocker is fixed. The first writer now loops on a committed bw_go row under a 60 s statement_timeout, and the suite inserts that row after its poll, whatever the poll saw. With sleep 3.5 after rl_wait_for bw_first, it's 17/17 on PG17 and PG18; with sleep 10, also 17/17 on both, so it isn't merely a longer window. As a control, 9859143's script with the same 3.5 s stall gives 16 + 1, premise: the second first write waited on the first: got [no] want [yes].
  • The premise and the release path: with the second writer never started, it's 15 + 2 in 24 s (got [no] want [yes] and got [held|ok|1]), and the suite doesn't hang. With the flag insert removed, the backstop ends it: 16 + 1 in 72 s, got [held|ERROR: ERROR: canceling statement due to statement timeout|0].
  • The squash dropped nothing. Against 9859143's changes, the twin and TESTS.md are identical, and the shell differs only in the bw_go table, the flag loop, the release insert and the comment.
  • Red (PG17, the installed .so md5 printed each time): class 104 → 2 gives 13 + 4, including got [reachable: (1410065409,741,field4=2)] and got [BLOCKED by the user lock], with pytest 2 failed. Bucket mod half gives got [512|1024] want [1024|1024] in both twins. Skipping the storage-row lock when the flag is on gives got [held|ERROR: ERROR: duplicate key value violates unique constraint "storage_pkey"|1], and the twin's equivalent.
  • Not blocking: under the storage-row-skip mutant the premise stays green. The second writer seems to wait on the first's transaction through storage_pkey, not on class 102, so the premise shows that it waited, not what it waited on. The final check is what catches the mutant, which is fine.
  • Green: 17/17 and 6 passed on PG17 and PG18. The census is 3001 / 2287 / 714, matching the paragraph, cluster_tests 572 matches collection, and none of the 17 new rows has a ";" fragment.
  • Before merge: main is now f2bd5f1, so the budget and expected_tests.txt need restating.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd force-pushed the test/1341-row-lock-class branch from 889dca4 to ee11596 Compare October 15, 2026 18:58
@jdatcmd

jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, restated as ee11596 on main f2bd5f1, after #1414, #1425 and #1427 merged. The patch is identical to 889dca4, which you approved, outside the budget and expected_tests.txt. The census is re-derived (3024 rows, 2291 never, 733 dated, matching the tsv), suites_not_covered goes 231 -> 230, cluster_tests is 578 by collection, the gate gives rc=0, and the guard set passes 435/435.

🤖 Generated with Claude Code

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved at ee11596, on main f2bd5f1.

  • Same patch: outside the count files, the changed lines match those I approved at 889dca4, ledger rows included.
  • Census: 3024 / 2291 / 733 from the tsv, matching the paragraph. suites_not_covered goes 231 -> 230.
  • CI: 13 pass, 2 pending. It will need restating on d5c0a5e (#1391) before it merges.

🤖 Generated with Claude Code

jdatcmd and others added 2 commits October 15, 2026 19:56
CREATE UNLOGGED TABLE and ALTER TABLE ... SET UNLOGGED are refused
(0A000) and leave nothing changed; ALTER TABLE ... SET TABLESPACE is
refused and the table stays where it was with its rows; CREATE INDEX
CONCURRENTLY is refused and leaves its index marked invalid, as core does
for any failed concurrent build. Nothing tested any of them. Arms in both
twins of am_contracts, each red under its own mutation.

limitations.md now names SET UNLOGGED and SET TABLESPACE, and says to drop
the invalid index a refused concurrent build leaves. cluster_tests +1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VvWuX1zUwk6Y513waRd4N
…by hand, and the row_lock_buckets bound (#1341 items 55, 56, 16)

Squashed for the rebase onto main; the commits it replaces were:
- test: advisory_lock_class's first writer commits on a flag row, not after a sleep (#1341)
- test: a bulk UPDATE holds row_lock_buckets row locks, not one per row (#1341 item 16)
- test: the row-identity lock is out of SQL's reach, and bulk_parallel_writer set by hand is safe (#1341 items 55, 56)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jdatcmd

jdatcmd commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

@OffgridwithJD, restated as 9fa5175, in a chain on main 670a58c: #1413 → #1415 → #1419 → #1422 → #1423 → #1424, this one on top of #1413. The patch is identical to ee11596, the head you approved, apart from re-derived bookkeeping: the budget paragraph, re-anchored on the previous link's census and matching the tsv; cluster_tests, matching collection; and the TESTS.md section number. The gate gives rc=0 and the guard set passes 435/435. I'll merge the chain in order as CI goes green.

🤖 Generated with Claude Code

@jdatcmd
jdatcmd merged commit 34b0c80 into main Oct 15, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants