Add BackgroundTasks module with Celery + Redis queue and admin UI

[antosubash]

Summary

Introduces a complete BackgroundTasks module that integrates Celery + Redis for asynchronous task processing with a full-featured admin UI for monitoring and retrying failed/stuck tasks.

Key Changes

Core Infrastructure

• Celery + Redis Integration: New celery_app.py with automatic task discovery from installed modules via entry points
• Sync Signal Handlers: signals.py and _signal_support.py keep a persistent TaskExecution table in sync with Celery task lifecycle (publish → prerun → success/failure/retry/revoked)
• Sync Database Layer: sync_db.py maintains a separate synchronous SQLAlchemy engine for signal handlers to avoid deadlocks in async contexts
• Task Execution Model: models.py defines the TaskExecution table with full lifecycle tracking (status, timestamps, args/kwargs, results, tracebacks, retry chains)

Background Jobs

• Internal Tasks: tasks.py implements two scheduled tasks:
  • sweep_stuck_tasks: Flips stale running tasks to stuck status
  • purge_old_executions: Deletes terminal task records older than retention period
• Worker Service: scripts/run_worker.py entry point for Celery worker and beat scheduler

Admin API & Views

• REST Endpoints: endpoints/api_admin.py provides list, detail, and retry operations with permission checks
• Inertia Views: endpoints/views.py renders server-side paginated task listings
• Service Layer: service.py implements business logic for listing, filtering, and retrying tasks with event emission
• Frontend Pages: React components (Index.tsx, Detail.tsx) with search, filtering, pagination, and retry dialogs

Configuration & Contracts

• Settings: settings.py loads Celery broker/result backend URLs and tuning parameters from environment
• Constants: constants.py centralizes all magic strings (table names, permissions, routes, task statuses)
• Public Contracts: contracts/ defines service interface, schemas, and events for other modules to depend on
• Localization: locales/en.json and i18n key generation for UI strings

Testing & Deployment

• Unit Tests: test_signals.py validates signal handlers with temporary SQLite; test_bg_service.py tests service logic; test_admin_api.py end-to-end API tests
• Docker: New worker.Dockerfile for lean Celery worker/beat images; updated docker-compose.yml with Redis service and worker container
• Database Migration: Alembic migration creates background_tasks_task_execution table with indexes

Notable Implementation Details

• Upsert-by-celery-id Pattern: Signal handlers use a read-then-insert-or-update pattern (no advisory locks needed since Celery serializes per-task signals within a worker)
• Sync Signals in Async Context: Web process signals fire synchronously; sync engine avoids event-loop deadlocks while sharing the same database as async app
• Retry Chain Tracking: Retried tasks link back to original via retried_from_id, preserving execution history
• Terminal Status Transitions: Only certain status combinations are valid (e.g., running → success/failed/stuck/revoked); invalid transitions are logged but don't crash
• Heartbeat Mechanism: running tasks update heartbeat_at on prerun; stale heartbeats trigger stuck status for manual retry
• Permission-Based Access: Admin UI and API require background_tasks.view (read) and background_tasks.manage (retry) permissions

https://claude.ai/code/session_013VMtYvyhYDHEDHGKWAQqPf