Repository navigation
Backport #509 and #512 to 0.6: replica identity and prune truncating only used children - #729
Merged
Merged
Conversation
PostgreSQL rejects UPDATE and DELETE on a table that is in a logical-replication publication and has no primary key, replica-identity index or FULL identity. On 0.6.17, 19 awa tables have none: awa.admin_dirty_queue_marks, awa.admin_dirty_kind_marks, awa.job_unique_claims and the 16 partitions of awa.queue_terminal_count_deltas. With a FOR ALL TABLES or FOR TABLES IN SCHEMA awa publication, the dirty-mark drain fails on every maintenance tick and unique-claim release fails. The 0.6 line keeps the schema at v40, so this ships as the repeatable schema patch replica_identity instead of migration v047. It sets REPLICA IDENTITY FULL on the two mark tables and on queue_terminal_count_deltas and each of its partitions in every queue-storage schema, and REPLICA IDENTITY USING INDEX idx_awa_jobs_unique on awa.job_unique_claims. Partitions do not inherit the setting from their parent, so the queue-storage installer sets it on the parent and every partition of a schema prepared later. Each ALTER takes ACCESS EXCLUSIVE on its table (and SHARE on idx_awa_jobs_unique) until the patch commits. Each lock wait is bounded by the session's lock_timeout, or 300 ms when none is set; a timeout fails the patch with 55P03 and changes nothing. Tables that already have their identity are skipped, so a re-run takes no lock. The patch is last in SCHEMA_PATCHES so a single-transaction export holds the locks only at its end. A missing mark table keeps the patch pending, so it runs after wait_free_dirty_marks recreates one. Tests cover every relation's identity in awa and custom schemas with and without receipts, the patch's exact lock set, its lock-free replay, its bounded lock wait, and UPDATE and DELETE on every table under a FOR ALL TABLES publication, including partitions of a schema prepared after the patch, with drain, unique-claim release, claim, completion, rotation, rollup and prune running. The release-gate rehearsal now expects the patch and allows exactly its locks.
…s (backport #512) Queue, lease and claim prune truncated every child of the sealed slot, so a prune swapped relfilenodes and wrote WAL and catalog rows for partitions that never held a row, such as the receipt children of a schema without receipts. Prune now runs TRUNCATE only on the children with heap, TOAST, free-space-map or visibility-map pages, or index pages beyond an empty btree, GiST or BRIN index, as main does after #512 and its follow-up #516. The probe runs after the slot's ACCESS EXCLUSIVE locks are granted. The lock statements, the children they name and their order are as in 0.6.17: queue prune still takes SHARE ROW EXCLUSIVE on all eight children and upgrades all eight to ACCESS EXCLUSIVE, and claim prune still locks its slot row only after its four children. A skipped child has no heap page, so it holds no row of any generation for slot reuse, the failed-row carry or the rollup fold to see. The 0.6.17 empty-slot fast path still decides on visible rows, and 0.6 has no idle rotation gate, so main's change to the lease rotation probe does not apply. Tests cover a receipt-mode queue and claim slot with used and unused children, reuse of the slot after the ring wraps, a vacuumed child with expanded indexes, and the BRIN free-space-map baseline per block size.
hardbyte
added
the
full-ci
Claude
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
 |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8fc6f473f6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…eated External runners apply repeatable scripts in name order, so R__replica_identity ran before R__wait_free_dirty_marks created the mark tables and left them without an identity. wait_free_dirty_marks now sets REPLICA IDENTITY FULL on a mark table it creates, and leaves tables that already existed alone.
…ort-509-512-0.6 # Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backports two main changes for the next 0.6 release.
Replica identity (#509)
PostgreSQL rejects
UPDATEandDELETEon a table that is in a logical-replication publication and has no primary key, replica-identity index orFULLidentity. A migrated 0.6 database has 19 such tables, so with awa in aFOR ALL TABLESpublication the dirty-mark drain fails every maintenance tick and unique-claim release fails.The schema stays at v40. A new repeatable patch,
replica_identity, registered last inSCHEMA_PATCHES:REPLICA IDENTITY FULLonawa.admin_dirty_queue_marks,awa.admin_dirty_kind_marks, andqueue_terminal_count_deltaswith each of its partitions in every queue-storage schema;REPLICA IDENTITY USING INDEX idx_awa_jobs_uniqueonawa.job_unique_claims;Unlike the earlier 0.6 patches it takes table locks:
ACCESS EXCLUSIVEon each table it changes (andSHAREonidx_awa_jobs_unique) until it commits. Each lock wait is bounded by the session'slock_timeout, or 300 ms when none is set. A timeout fails with55P03and changes nothing, and a table that already has its identity is skipped.docs/upgrade-0.6.17-to-0.6.18.mddescribes the locks.awa-model/tests/replica_identity_test.rschecks every table in awa and in custom schemas with and without receipts, the exact lock set, a re-run that locks nothing, and the bounded wait. With aFOR ALL TABLESpublication it then runs the mark drain, unique-claim release, claim, complete, rotate, rollup and prune, including on partitions of a schema prepared after the patch.Prune truncates only children that hold data (#512, with #516)
Queue, lease and claim prune take the same locks on every child of the sealed slot, in the same order, as in 0.6.17. After those locks are granted they
TRUNCATEonly the children with heap, TOAST, FSM or VM pages, or index pages beyond an empty btree, GiST or BRIN index. A child that never held a row keeps its relfilenode, so prune no longer writes WAL and catalog rows for empty partitions. Main's lease-rotation idle-gate change is not included, because 0.6 has no idle rotation gate. 0.6.17's empty-slot fast path is unchanged.New tests in
queue_storage_runtime_testcover used and unused children in queue and claim slots with reuse after the ring wraps, a vacuumed child with expanded indexes, and the BRIN block-size baseline.Testing
Run locally on PostgreSQL 18:
cargo test -p awa-model,queue_storage_runtime_test(275 passed, 1 ignored),migration_test,dirty_marks_wait_free_test,lane_ownership_roles_test,insert_many_schema_fallback_test,migrate_patch_cli_test, and the awa-worker prune unit tests.cargo fmtandclippy -D warningsare clean.This and #726 both add a
## [Unreleased]CHANGELOG section, so whichever merges second needs a rebase.https://claude.ai/code/session_01YSKjmBe32Fd2jj7gvxT6ZL
Generated by Claude Code