chore: prepare 0.6.17 release

CHANGELOG.md

@@ -2,7 +2,9 @@
 
 Notable changes between releases. Detailed migration notes for storage transitions live in [`docs/upgrade-0.5-to-0.6.md`](docs/upgrade-0.5-to-0.6.md).
 
-## [Unreleased]
+## [0.6.17] — 2026-11-12
+
+Recommended for everyone on 0.6.16 using queue storage. Claims commit asynchronously and reuse the runtime's claimer lease, which raised throughput by 19 to 21% at 4, 16 and 64 workers and cut end-to-end p99 at 800 jobs/s from 31 ms to 24 ms in a three-round comparison against 0.6.16 on PostgreSQL 18 with `synchronous_commit = on`. Promotion and rescue no longer wait behind the leader's other maintenance work, a runtime can opt out of maintenance leadership, and several admin reads and the admin UI are fixed. See the [upgrade guide](docs/upgrade-0.6.16-to-0.6.17.md).
 
 ### Upgrading
 
@@ -17,12 +19,11 @@ Notable changes between releases. Detailed migration notes for storage transitio
 
 - Syncing a periodic job declaration that matches the stored schedule no longer rewrites its `awa.cron_jobs` row or moves `updated_at`, so runtimes that sync on a timer stop generating a row version per schedule per interval.
 - **Claim transactions commit asynchronously (0.6 port of #507).** Claims and the claim-cursor advance that follows them run with `synchronous_commit = off`, so a lane's claim head is no longer locked across a WAL fsync. A claim lost to a server crash leaves the job claimable again, consistent with at-least-once delivery; enqueues, completions and every other transaction stay synchronous. The setting is `SET LOCAL` in those transactions only. A post-claim cursor advance now applies only while the claim it follows is visible on the server it runs against, so a retry that lands on a promoted standby cannot move the cursor past a claim that never reached it. A worker still running a claim lost this way is not fenced by its run lease, because the job's next claim reuses it; see [architecture](docs/architecture.md). No schema or configuration change.
-
 - **Claims reuse the instance's claimer lease (0.6 port of #508).** A queue-storage claim no longer re-reads the queue's claimer target and its own claimer lease on every call; the last read or refreshed lease is reused for up to 250 ms (never within a second of expiry), removing two round trips and two transactions from each claim. On `main` at 4 workers this cut transactions per completed job from 2.35 to 1.39.
 
 ### Fixed
 
-- Queue and claim ring prunes no longer starve under sustained claim load (#721). Prune requested `ACCESS EXCLUSIVE` on the sealed slot's partitions for 10 ms, so with claims, completions and heartbeats always overlapping it never got the lock. That request now waits the prune lock timeout, lets waiting readers through, and retries with twice the wait up to 320 ms (`QueueStorage::prune_lock_wait_ceiling`) before reporting `blocked`. Queue prune also ran its safety checks under that lock, so when it did get it every reader of those partitions waited for the checks, which take seconds on a large slot. It now takes `SHARE ROW EXCLUSIVE` first, which blocks writes to the sealed slot and no reads, runs the checks, the failed-row carry and the rollup scan under it, and requests `ACCESS EXCLUSIVE` only for the `TRUNCATE`; after the upgrade it re-checks active leases and pending ready rows, which is what catches a lease committed while it waited. Queue prune requests the segment partition first because claims lock it first; a statement that holds a later partition while it waits for an earlier one still blocks that attempt until its timeout. Claim prune still runs its open-claim check under `ACCESS EXCLUSIVE`, because heartbeat, retry, fail and snooze read `lease_claims` and then update it in one transaction and would deadlock with a lock upgrade. A deadlock that PostgreSQL resolves by cancelling a prune is now counted as `blocked` rather than `error`. Readers of a sealed slot can now wait up to 320 ms behind a contended prune instead of 10 ms. The startup warning compares the 320 ms ceiling, not the base timeout, with `deadlock_timeout`, and eight times the ceiling with `statement_timeout`, because the lock timeout applies to each of the eight partitions one lock statement takes.
+- Queue prune no longer runs its safety checks under `ACCESS EXCLUSIVE`, and neither ring prune holds a lock-order cycle with rescue, heartbeats or claims while it waits (#721, partial). Queue prune takes `SHARE ROW EXCLUSIVE` on the sealed slot first, which blocks writes to the slot and no reads, runs the checks, the failed-row carry and the rollup scan under it, and requests `ACCESS EXCLUSIVE` only for the `TRUNCATE`; after the upgrade it re-checks active leases and pending ready rows. On 0.6.16 a prune that got the lock made every reader of those partitions wait for the checks, which take seconds on a large slot. For both rings the `ACCESS EXCLUSIVE` request now waits the prune lock timeout, lets waiting readers through, and retries with twice the wait up to 320 ms (`QueueStorage::prune_lock_wait_ceiling`) before reporting `blocked`, so a reader of a sealed slot can wait up to 320 ms behind a contended prune instead of 10 ms. Queue prune requests the segment partition first because claims lock it first, and claim prune locks its `claim_ring_slots` row only after the slot's partitions because rescue takes them in that order. A deadlock that PostgreSQL resolves by cancelling a prune is counted as `blocked` rather than `error`. The startup warning compares the 320 ms ceiling with `deadlock_timeout`, and eight times the ceiling with `statement_timeout`. This does not end the starvation the issue describes: with a large backlog of abandoned claims prune is still blocked in most attempts, as on 0.6.16, because claims, completions and heartbeats still lock sealed slots.
 - The embedded admin UI now works when its axum router is nested under a path such as `/awa` (#502). Assets, API requests, navigation and direct deep links automatically use the mount path, including outer nested routers, without rebuilding the frontend or changing existing router calls.
 - `admin::list_jobs` and `admin::job_kind_overviews` no longer omit a failed job, or leave it out of its kind's count, while queue prune moves its row out of the slot being pruned. Both read several queries from one snapshot and release partition locks between them, so the queries after a prune committed saw neither the pruned slot's rows nor the moved ones. These reads and the point reads such as `admin::get_job` now compare the relfilenode of every ring partition before their snapshot and after it ends, and read again in a new snapshot when prune or the terminal-count rollup truncated one in between, where 0.6.16 read a single job again only when it found nothing. After three reads that each overlapped a truncation, the fourth locks the ring partitions before taking its snapshot and holds them until it commits, so a read never returns a result a truncation could have changed; prune skips its pass if it cannot get its locks within its lock timeout. The comparison adds two catalog lookups to each read and takes no partition locks, so prune is blocked no more often than before.
 - Claiming from a ready slot that holds tombstones no longer scans the slot's whole tombstone partition on every claim (#713, reported and fixed by @Wallace99). Each claim checks its candidate rows against `ready_tombstones`, and for a small batch PostgreSQL could plan that check as a hashed subplan over the partition instead of one index probe per row: a 32-row claim read 10,000 tombstone rows in the regression test. The check now carries `OFFSET 0`, which keeps it correlated. Tombstones are left by cancelled, retried and moved jobs, so the cost grew with churn in the slot. Applied by the `correlated_claim_tombstones` schema patch; no configuration or protocol change.

Cargo.toml

@@ -14,7 +14,7 @@ members = [
 exclude = ["awa-python", "spike"]
 
 [workspace.package]
-version = "0.6.16"
+version = "0.6.17"
 edition = "2021"
 license = "MIT OR Apache-2.0"
 description = "Postgres-native background job queue — transactional enqueue, heartbeat crash recovery, SKIP LOCKED dispatch"
@@ -25,11 +25,11 @@ categories = ["database", "asynchronous", "web-programming"]
 
 [workspace.dependencies]
 # Internal crates
-awa-model = { path = "awa-model", version = "0.6.16" }
-awa-macros = { path = "awa-macros", version = "0.6.16" }
-awa-metrics = { path = "awa-metrics", version = "0.6.16" }
-awa-worker = { path = "awa-worker", version = "0.6.16" }
-awa = { path = "awa", version = "0.6.16" }
+awa-model = { path = "awa-model", version = "0.6.17" }
+awa-macros = { path = "awa-macros", version = "0.6.17" }
+awa-metrics = { path = "awa-metrics", version = "0.6.17" }
+awa-worker = { path = "awa-worker", version = "0.6.17" }
+awa = { path = "awa", version = "0.6.17" }
 
 # Database
 sea-orm = { version = "=2.0.0-rc.38", default-features = false, features = ["sqlx-postgres", "runtime-tokio-rustls", "with-chrono", "with-json"] }

README.md

@@ -391,6 +391,7 @@ All coordination through Postgres. The Rust runtime owns dispatch, leases, heart
 | [0.6.9 → 0.6.10 upgrade](docs/upgrade-0.6.9-to-0.6.10.md) | Rolling patch upgrade, rollback, and optional runtime privilege tightening |
 | [0.6.14 → 0.6.15 upgrade](docs/upgrade-0.6.14-to-0.6.15.md) | Rolling patch upgrade: bounded worker finalization, the claim batch index and the batch compatibility insert |
 | [0.6.15 → 0.6.16 upgrade](docs/upgrade-0.6.15-to-0.6.16.md) | Rolling patch upgrade: bounded rescue drains of abandoned claims, batch compatibility inserts that give way to per-row producers, new-lane provisioning order, the operator prune halt and its grants, and claim batch index fixes |
+| [0.6.16 → 0.6.17 upgrade](docs/upgrade-0.6.16-to-0.6.17.md) | Rolling patch upgrade: asynchronous claim commits, the maintenance eligibility table, the claim tombstone patch, the changed `/api/stats` and `state_counts` shape, and prune lock waits |
 | [Configuration reference](docs/configuration.md) | `QueueConfig`, `ClientBuilder`, Python `start()`, env vars |
 | [HTTP workers and callbacks](docs/http-callbacks.md) | Serverless dispatch, callback receiver endpoints, signature verification |
 | [Lifecycle hooks](docs/lifecycle-hooks.md) | React to job start/finish/retry/callback events for metrics, alerting, cleanup |

awa-cli/Cargo.toml

@@ -14,7 +14,7 @@ path = "src/main.rs"
 [dependencies]
 awa-model.workspace = true
 awa-worker.workspace = true
-awa-ui = { path = "../awa-ui", version = "0.6.16" }
+awa-ui = { path = "../awa-ui", version = "0.6.17" }
 axum.workspace = true
 sqlx.workspace = true
 tokio.workspace = true
@@ -28,5 +28,5 @@ hex.workspace = true
 uuid.workspace = true
 
 [dev-dependencies]
-awa-testing = { path = "../awa-testing", version = "0.6.16" }
+awa-testing = { path = "../awa-testing", version = "0.6.17" }
 assert_cmd = "2"

awa-cli/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "awa-cli"
-version = "0.6.16"
+version = "0.6.17"
 description = "CLI for the Awa Postgres-native job queue (migrations, admin, serve)"
 readme = "README.md"
 requires-python = ">=3.10"

awa-python/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "awa-python"
-version = "0.6.16"
+version = "0.6.17"
 edition = "2021"
 
 [lib]
@@ -12,8 +12,8 @@ default = ["cel"]
 cel = ["awa-model/cel"]
 
 [dependencies]
-awa-model = { path = "../awa-model", version = "0.6.16" }
-awa-worker = { path = "../awa-worker", version = "0.6.16", features = ["__python-bridge"] }
+awa-model = { path = "../awa-model", version = "0.6.17" }
+awa-worker = { path = "../awa-worker", version = "0.6.17", features = ["__python-bridge"] }
 pyo3 = { version = "0.29", features = ["macros", "abi3-py310", "chrono"] }
 pyo3-async-runtimes = { version = "0.29", features = ["tokio-runtime"] }
 sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "postgres", "chrono", "json"] }

awa-python/pyproject.toml

@@ -5,7 +5,7 @@ build-backend = "maturin"
 [project]
 name = "awa-pg"
 requires-python = ">=3.10"
-version = "0.6.16"
+version = "0.6.17"
 description = "Postgres-native background job queue — Python SDK with async/sync workers, transactional enqueue, and progress tracking. Install awa-pg[ui] to add the bundled web dashboard."
 readme = "README.md"
 license = {text = "MIT OR Apache-2.0"}
@@ -34,7 +34,7 @@ classifiers = [
 #
 # Kept opt-in so the default `awa-pg` install stays small — workers and
 # producers don't need the ~10 MB UI bundle.
-ui = ["awa-cli==0.6.16"]
+ui = ["awa-cli==0.6.17"]
 
 [project.urls]
 Homepage = "https://github.com/hardbyte/awa"

awa-seaorm/Cargo.toml

@@ -14,6 +14,6 @@ serde_json.workspace = true
 sqlx.workspace = true
 
 [dev-dependencies]
-awa-testing = { path = "../awa-testing", version = "0.6.16" }
+awa-testing = { path = "../awa-testing", version = "0.6.17" }
 serde = { workspace = true, features = ["derive"] }
 tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }

awa/Cargo.toml

@@ -20,7 +20,7 @@ awa-macros.workspace = true
 awa-worker.workspace = true
 
 [dev-dependencies]
-awa-testing = { path = "../awa-testing", version = "0.6.16" }
+awa-testing = { path = "../awa-testing", version = "0.6.17" }
 sqlx.workspace = true
 serde.workspace = true
 serde_json.workspace = true

docs/test-plan.md

@@ -15,6 +15,7 @@ Tests run against real Postgres 15+ (not managed services) using a dedicated tes
 | T5 | Deadline rescue (hung job) | ✓ |  |
 | T5a | Poison jobs (handler panic, crash loop, deadline loop) stop at `max_attempts` on both engines | ✓ |  |
 | T5b | Rescue drains a backlog of abandoned claims in bounded passes without rescuing live claims, and the queue and claim rings then rotate past the pinned slots | ✓ |  |
+| T5c | Ring prune wins against overlapping short readers, sees a lease committed while it waits to truncate, and does not deadlock with rescue or a read-then-write transaction on the claim ring | ✓ |  |
 | T6 | Transactional atomicity | ✓ |  |
 | T7 | Transactional atomicity (Python) |  | ✓ |
 | T8 | Uniqueness under contention | ✓ | ✓ |
@@ -247,6 +248,7 @@ Concurrent lifecycle benchmark (1 queue × 128 workers, 20K jobs):
 | TLA12 | AwaStorageLockOrder | Postgres lock ordering across enqueue, new-lane provisioning, claim, complete, cancel, rescue, rotate, and prune |
 | TLA13 | AwaStorageTransition | Storage-transition prepare, mixed-entry, finalize, and abort gates |
 | TLA14 | AwaDeadTupleContract | Hot-table reclaim-kind and partition-truncate contract |
+| TLA15 | AwaQueuePruneTwoPhase | Queue prune's writer lock, exclusive upgrade and recheck against a claim that commits during the upgrade |
 
 The formal suite includes passing configs and expected-counterexample configs. The expected-counterexample configs keep historical bugs executable: old dispatch claim, old view trigger, naive segment race, old storage-transition gate, shard-ignorant prune, and deliberate lock-order cycles. Trace configs use a `TraceIncomplete` invariant as a positive witness: a valid trace violates that invariant after TLC consumes every event.