Repository navigation
docs(upgrade): revoke a running 0.6 fleet's schema CREATE without a pause - #541
Merged
Merged
Conversation
The enqueue and claim head triggers fire BEFORE INSERT, ahead of ON CONFLICT, and read the lane's sequence. A lane a 0.6 runtime created has a sequence that runtime owns, so awa storage prepare-queue run as the migrator failed with 42501 on any such lane, and every runtime cache miss re-synced an existing sequence. Head rows are now inserted only when absent, under the lane lock runtimes already take for these inserts.
…a CREATE without a pause The migration trust check refuses a schema where another role holds CREATE, but 0.6 lane helpers run CREATE SEQUENCE IF NOT EXISTS on every reservation, so revoking the grant before migrating stopped running 0.6 producers and workers with 42501. The documented path was a pause. awa storage prepare-create-revoke, run as the migrator under a running 0.6 fleet, replaces each queue-storage schema's lane helpers with the ones v049 installs (identical to every 0.6 release's helpers apart from creating a sequence only when it is missing), provisions all four priorities of every shard of every queue in use, and grants the roles holding CREATE the privileges they need on the sequences it creates. It applies the migration's own trust rules apart from the CREATE grant, leaves a schema with any other violation unchanged, checks the rules again before committing, and prints the REVOKE statements plus the sequence grants only the sequences' runtime owners can make, which the migration needs to read and set those sequences. The upgrade guide replaces the pause procedure with these steps. The new test installs a v040 schema with the 0.6.9 lane helpers and 0.6 grants, shows revoking CREATE first breaks a 0.6 reservation, and then runs the command, grants and revokes under a 2 s lock_timeout while a 0.6 enqueue transaction stays open, before migrating to the current version. Each part of the fix fails the test when reverted.
Runs an awa-pg 0.6.9 producer and worker as the runtime role while CREATE is revoked, with and without prepare-create-revoke, then migrates to the current version and reports failed inserts and slow inserts with lock waits. Against PostgreSQL 17: revoking without preparing fails every enqueue for the rest of the run (471 errors in 10 s); preparing first gives 0 failures and every job completes, with the prepare, grants and revokes taking 0.2 s and no insert slower than 250 ms until migrate, which blocks enqueues for its whole 5.2 s transaction.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 20 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
|
21 tasks
hardbyte
added
the
full-ci
ChatGPT Codex Connector
Apply the reviewed standalone read correction without changing the CREATE-revoke preparation protocol. Cover priority and queue moves across ready-ring wrap with the same two deterministic regressions.
The 0.6 line now ships what prepare-create-revoke did. On a v40 database, the prepared_lane_sequences schema patch (0.6.10) makes the lane helpers skip CREATE SEQUENCE when a lane's sequence exists, and the definer_lane_provisioning patch (0.6.14) creates a missing lane's sequences through the schema owner's SECURITY DEFINER functions. Running `awa migrate` with a 0.6.14 or later CLI therefore lets runtime roles lose schema CREATE while 0.6 binaries keep running, without a 0.7 command. The merged command would also have installed main's lane helpers, whose provision_lane_sequences calls awa.lane_sequence_problems, on 0.6 databases that lack it. The point-read tombstone fix, its two ring-wrap regressions and the carry-forward prune retry reached main through #629 and e2fbfa6.
The carry-forward test already retries prune while a background session holds the slot. The re-carry test's two prunes and the reader-lock test's prune after the lock is released had the same exposure; route all three through one helper that retries only Blocked for the expected slot.
….6 lane patches The guide asked for a pause because unpatched 0.6 lane helpers check schema CREATE on every reservation. awa migrate from 0.6.14 or later applies prepared_lane_sequences and definer_lane_provisioning to a v40 database, after which no runtime role needs CREATE, so the revoke can happen under traffic. Point at the 0.6 rehearsal that covers it.
hardbyte
changed the title
hardbyte
marked this pull request as ready for review
November 13, 2026 01:08
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 443e6790af
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The 0.6→0.7 upgrade guide told split-role installs to pause 0.6 producers and workers while revoking the runtime role's schema
CREATE. That pause is no longer needed:awa migratefrom 0.6.14 or later applies theprepared_lane_sequencesanddefiner_lane_provisioningschema patches to a v40 database, after which no runtime role needsCREATE. The guide and CHANGELOG now say to run those patches first and then revoke under traffic, and point at the 0.6 rehearsal that covers it.This PR originally added an
awa storage prepare-create-revokecommand to do that preparation from a 0.7 binary. It is removed: the 0.6 patches already do the job, and the command would have installed main's lane helpers, which callawa.lane_sequence_problems, on 0.6 databases that lack it. The point-read tombstone fix this PR also carried reached main through #629.What remains besides the docs: the prune tests that a background session can block (the carry-forward, re-carry and reader-lock tests) share one helper that retries only a
Blockedoutcome for the expected slot, as the maintenance loop would.Validation: workspace build, fmt, clippy on the
awacrate, and theprunetests inqueue_storage_runtime_test(37 passed). Full CI covers the rest on PostgreSQL 17 and 18.