Task Queue Service (Go + Asynq + Redis + Postgres)

Horizontally scalable HTTP task queue service with:

• Task CRUD API (/tasks)
• Worker processing through Asynq/Redis
• Persistent task state in Postgres
• Retry with exponential backoff + jitter
• Structured JSON logs (slog)
• Prometheus metrics (/metrics)
• Health endpoints (/health/live, /health/ready)

Architecture

1. API receives tasks and persists them in Postgres.
2. API enqueues task IDs into Redis/Asynq.
3. Workers consume tasks, process concurrently, and update Postgres state.
4. Retries are managed by Asynq with custom delay function.

Run with Docker Compose

docker compose up --build

API: http://localhost:8080

Example API calls

Create task:

curl -X POST http://localhost:8080/tasks \
  -H 'content-type: application/json' \
  -d '{"type":"email.send","payload":{"to":"user@example.com"},"max_attempts":5}'

Get task:

curl http://localhost:8080/tasks/<task-id>

List tasks:

curl 'http://localhost:8080/tasks?state=queued&limit=20&offset=0'

Patch task:

curl -X PATCH http://localhost:8080/tasks/<task-id> \
  -H 'content-type: application/json' \
  -d '{"max_attempts":10}'

Delete task:

curl -X DELETE http://localhost:8080/tasks/<task-id>

Local run

1. Copy .env.example values into your shell env.
2. Ensure Postgres + Redis are running.
3. Run migrations from migrations/000001_init.sql.
4. Start API and worker:

go run ./cmd/api
go run ./cmd/worker

Testing

go test ./...