Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Notable changes between releases. Detailed migration notes for storage transitio

## [0.6.12] — Unreleased

Patch release: batch completion and bulk "move failed to DLQ" on queue storage no longer fail once a batch grows past PostgreSQL's bind-parameter limit. No migration and no public API change.
Patch release: batch completion and bulk "move failed to DLQ" on queue storage no longer fail once a batch grows past PostgreSQL's bind-parameter limit, draining a queue cancels its backlog in bulk, and cancelling or moving an available job no longer acts on one a worker has already claimed. No migration and no public API change.

- **Completing a large batch and moving many failed jobs to the DLQ no longer
fail on queue storage** (backport of the writer approach in
Expand All @@ -21,6 +21,31 @@ Patch release: batch completion and bulk "move failed to DLQ" on queue storage n
`AWA_COMPLETION_BATCH_SIZE` is raised above the default of 512. Both writers
now send one array per column through a fixed-text
`INSERT ... SELECT FROM unnest(...)`.
- **`admin::drain_queue` on queue storage cancels a backlog in bulk and
drains every stripe of a striped queue.** It found the queue's jobs through
a lookup that read every retained ready row, then cancelled them one at a
time, several statements each, inside one transaction, so draining a large
backlog took minutes and held its row locks throughout. Scheduled,
retryable and available jobs are now cancelled 1,000 at a time, each chunk
one short transaction, with available rows read from each lane's claim
cursor; waiting jobs are still cancelled one by one and running jobs are
left alone. Jobs enqueued after the drain starts are left for a later
call, so draining a queue that keeps receiving work still returns, and
the drain moves each lane's claim cursor past the rows it cancelled, so
workers reach the jobs behind them on their next poll. It also matched
only the logical queue name, so draining a striped queue cancelled
nothing; it now covers every stripe. `awa queue drain` and the Python
`drain_queue` use this path.
- **Cancelling, moving or reprioritising an available job on queue storage
no longer acts on a job a worker has already claimed.** A claim moves its
lane's claim cursor past the rows it took only after it commits, so
briefly, or until the next claim if that advance failed, a claimed row
still looked available. A cancel in that window wrote a cancelled terminal
row while the attempt kept running, and a move re-enqueued a job that was
already executing. A claim still in progress was missed the same way.
These paths now wait for any claim in progress on the job's lane and skip
rows a claim has recorded, the same evidence claims use to skip them, so a
cancel reaches the running attempt instead.

## [0.6.11] — 2026-11-04

Expand Down
18 changes: 4 additions & 14 deletions awa-model/src/admin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1629,22 +1629,12 @@ where
}

/// Drain a queue: cancel all non-running, non-terminal jobs.
///
/// On queue storage this also drains the queue's stripes: queues named
/// `<queue>#<digits>`. Jobs enqueued after the drain starts are left alone.
pub async fn drain_queue(pool: &PgPool, queue: &str) -> Result<u64, AwaError> {
if let Some(store) = active_queue_storage(pool).await? {
let sql = format!(
"{} \
SELECT job_id \
FROM current_jobs \
WHERE queue = $1 \
AND state IN ('available', 'scheduled', 'retryable', 'waiting_external') \
ORDER BY job_id ASC",
queue_storage_current_jobs_cte(store.schema())
);
let ids: Vec<i64> = sqlx::query_scalar(&sql).bind(queue).fetch_all(pool).await?;
return store
.cancel_jobs_by_ids(pool, &ids)
.await
.map(|rows| rows.len() as u64);
return store.drain_queue(pool, queue).await;
}

let result = sqlx::query(
Expand Down
Loading
Loading