Skip to content

background_tasks: rapid confirm clicks on single Retry queue duplicate retries (and 500 on SQLite) #467

Closed

Description

@antosubash

Found during QA of #463. This is pre-existing: service.retry is the same on main.

Problem

On /admin/background-tasks/{id}, clicking the Retry dialog's confirm button 5 times quickly sends 5 POST /api/background_tasks/admin/executions/{id}/retry requests.

  • 2 of them returned 200, and each created another pending child row. The execution already had one retry, so it ended up with three.
  • 3 returned 500, with sqlite3.OperationalError: database is locked on the INSERT. background_tasks.signals also logged on_task_publish failed.

Expected

One confirm should produce one retry. The button should be disabled while the request is in flight.

Notes

  • Retrying an execution that has already been retried is allowed on purpose. The bulk_retry_conditions docstring says the single-row dialog "makes the person decide". So the fix belongs in the dialog: disable the confirm button while submitting. A 409 from the API would not be the right fix.
  • The 500s only happen on SQLite. With concurrent requests, the publish signal's synchronous write (sync_session) runs on the event loop while another request holds SQLite's single write lock. Postgres is not affected. It is worth confirming whether the dev-mode signal write should go through asyncio.to_thread.

Repro

Run the app on SQLite with no worker. Seed a failed execution, open its detail page, press Retry, then click confirm 5 times quickly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions