Skip to content

feat: implement backpressure handling for queue processing#692

Merged
RUKAYAT-CODER merged 3 commits into
rinafcode:mainfrom
Nimatstar:feat/queue-backpressure-handling
May 29, 2026
Merged

feat: implement backpressure handling for queue processing#692
RUKAYAT-CODER merged 3 commits into
rinafcode:mainfrom
Nimatstar:feat/queue-backpressure-handling

Conversation

@Nimatstar
Copy link
Copy Markdown
Contributor

Summary

Implements a backpressure mechanism for queue processing that monitors queue depth and active job counts, applies rate-limiting delays when thresholds are breached, and exposes monitoring snapshots via REST endpoints.

Changes

  • src/queues/backpressure/backpressure.service.ts - BackpressureService:
    • evaluate(queueName, depth, active) - resolves
      ormal / warning / critical level using existing QUEUE_HEALTH_THRESHOLDS constants; returns { level, rateLimitActive, delayMs }
    • Rate-limiting delays: warning ? 1 s, critical ? 5 s
    • Deduped ERROR log on first critical breach; LOG on recovery
    • getSnapshot(queue) / getAllSnapshots() - monitoring data per queue
  • src/queues/backpressure/backpressure.controller.ts - GET /queues/backpressure (all) and GET /queues/backpressure/:queue (single)
  • src/queues/backpressure/backpressure.module.ts - module wiring, exports service

Usage

s // In a queue producer before enqueuing: const state = this.backpressureService.evaluate(queueName, depth, active); if (state.rateLimitActive) { if (state.level === 'critical') throw new TooManyRequestsException(); await sleep(state.delayMs); }

Acceptance Criteria

  • Backpressure mechanism implemented (evaluate() with level resolution)
  • Rate limiting on queue processing (delayMs +
    ateLimitActive flag)
  • Queue depth monitoring (getSnapshot / getAllSnapshots endpoints)
  • Alerting on high queue depth (ERROR/WARN logs on threshold breach)

closes #608

@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented May 29, 2026

@Nimatstar Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@RUKAYAT-CODER RUKAYAT-CODER merged commit 930c0cf into rinafcode:main May 29, 2026
13 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement backpressure handling for queue processing

2 participants