|
| 1 | +--- |
| 2 | +layout: post.html |
| 3 | +permalink: /blog/autoscaling-nodejs-services-on-queue-depth/ |
| 4 | +templateEngineOverride: md |
| 5 | +title: "Autoscale on queue depth, not CPU" |
| 6 | +summary: "CPU tells you a process is busy, not whether anyone is waiting. An @imqueue service's backlog is a Redis list, so KEDA can scale on it with no exporter, no Prometheus and no code change — and the metrics server covers the case where you need more." |
| 7 | +description: "Autoscale Node.js microservices on queue depth with @imqueue: a KEDA ScaledObject reading LLEN directly, the built-in /metrics endpoint with a Kubernetes HPA, and the four properties of the number that decide whether the loop behaves." |
| 8 | +keywords: "autoscale on queue depth, keda redis scaler nodejs, scale microservices queue length, kubernetes hpa queue length, imqueue metrics server, queue_length prometheus, scale nodejs workers backlog, keda scaledobject redis list" |
| 9 | +date: 2026-08-22 |
| 10 | +author: mykhailo-stadnyk |
| 11 | +illustration: queue-depth-scaling |
| 12 | +topics: [queue, architecture, performance, resilience] |
| 13 | +ogType: article |
| 14 | +--- |
| 15 | + |
| 16 | +**CPU is a lagging proxy for "is this service behind?"** It goes up when a |
| 17 | +process is working and stays up whether that work is useful or a retry storm |
| 18 | +eating itself. It says nothing about whether anyone is waiting. Most HTTP |
| 19 | +services scale on it anyway, for a structural reason rather than a lazy one: |
| 20 | +there is no queue in front of an HTTP handler, so there is nothing else to |
| 21 | +measure. Requests either get a worker or they get a connection error, and the |
| 22 | +backlog exists only as latency after the fact. |
| 23 | + |
| 24 | +A queue-based service has the number directly. With |
| 25 | +[`@imqueue`](/get-started/), work waiting for a service is a Redis list, and its |
| 26 | +length is the signal an autoscaler actually wants: **how much work is waiting**, |
| 27 | +independent of how hard anything is currently working on it. |
| 28 | + |
| 29 | +> **TL;DR** — The backlog is `LLEN imq:<ServiceName>`. KEDA's built-in `redis` |
| 30 | +> scaler reads that directly: no exporter, no Prometheus, no code change. If you |
| 31 | +> already run Prometheus, or you run a clustered broker fleet, turn on the |
| 32 | +> service's own `/metrics` endpoint instead and point an HPA at it. Both read the |
| 33 | +> same number, so the same four caveats apply to both — and the one that bites is |
| 34 | +> that the number is **0 while the queue's writer is disconnected**. |
| 35 | +
|
| 36 | +## The number |
| 37 | + |
| 38 | +Every `IMQService` reads from one Redis list. The key is the queue's prefix and |
| 39 | +name joined by a colon; the prefix defaults to `imq`, and the name defaults to |
| 40 | +the service class name. A service class called `UserService` therefore drains: |
| 41 | + |
| 42 | +~~~bash |
| 43 | +$ redis-cli LLEN imq:UserService |
| 44 | +17 |
| 45 | +~~~ |
| 46 | + |
| 47 | +That is the whole contract. Everything below is a way of getting that number to |
| 48 | +a scaling loop. |
| 49 | + |
| 50 | +## Rung 1: KEDA, reading the list |
| 51 | + |
| 52 | +KEDA ships a `redis` scaler that does exactly this, so the cheapest working setup |
| 53 | +involves nothing new inside your service at all: |
| 54 | + |
| 55 | +~~~yaml |
| 56 | +apiVersion: keda.sh/v1alpha1 |
| 57 | +kind: ScaledObject |
| 58 | +metadata: |
| 59 | + name: user-service |
| 60 | +spec: |
| 61 | + scaleTargetRef: |
| 62 | + name: user-service |
| 63 | + minReplicaCount: 1 # never 0 — see "0 means two things", below |
| 64 | + maxReplicaCount: 20 |
| 65 | + triggers: |
| 66 | + - type: redis |
| 67 | + metadata: |
| 68 | + address: redis:6379 |
| 69 | + listName: imq:UserService |
| 70 | + listLength: "20" # scale out while more than 20 are waiting |
| 71 | + authenticationRef: |
| 72 | + name: redis-auth # if your broker has a password, and it should |
| 73 | +~~~ |
| 74 | + |
| 75 | +No exporter to deploy, no Prometheus, no adapter, no `metricsServer` option, and |
| 76 | +no change to the service. If your fleet is one Redis and you are already running |
| 77 | +KEDA, stop here. |
| 78 | + |
| 79 | +## Rung 2: the service's own `/metrics`, and an HPA |
| 80 | + |
| 81 | +Rung 1 depends on the scaler being able to reach Redis and knowing the key. When |
| 82 | +you would rather the *service* publish its backlog — because Prometheus is |
| 83 | +already the system of record, because you want the figure on a dashboard next to |
| 84 | +everything else, or because of the clustered case in rung 3 — every `IMQService` |
| 85 | +can serve it, one option away: |
| 86 | + |
| 87 | +~~~typescript |
| 88 | +import { IMQService, expose } from '@imqueue/rpc'; |
| 89 | + |
| 90 | +export class UserService extends IMQService { |
| 91 | + // ...exposed methods |
| 92 | +} |
| 93 | + |
| 94 | +const service = new UserService({ |
| 95 | + metricsServer: { |
| 96 | + enabled: true, // off by default |
| 97 | + port: 9090, // the default |
| 98 | + }, |
| 99 | +}); |
| 100 | + |
| 101 | +await service.start(); // the listener comes up with it |
| 102 | +~~~ |
| 103 | + |
| 104 | +It answers exactly one route, in Prometheus exposition format, and 404s for |
| 105 | +anything else: |
| 106 | + |
| 107 | +~~~bash |
| 108 | +$ curl -s localhost:9090/metrics |
| 109 | +queue_length{} 17 |
| 110 | +~~~ |
| 111 | + |
| 112 | +Point Prometheus at it, expose `queue_length` as an external metric through |
| 113 | +prometheus-adapter, and a `HorizontalPodAutoscaler` reads it like any other: |
| 114 | + |
| 115 | +~~~yaml |
| 116 | +metrics: |
| 117 | + - type: External |
| 118 | + external: |
| 119 | + metric: |
| 120 | + name: queue_length |
| 121 | + target: |
| 122 | + type: Value |
| 123 | + value: "20" # scale out while more than 20 wait |
| 124 | +~~~ |
| 125 | + |
| 126 | +The service *is* the exporter — there is deliberately no separate |
| 127 | +`imqueue-exporter` to install, and no @imqueue-specific KEDA scaler either. The |
| 128 | +built-in `redis` scaler already covers rung 1 and prometheus-adapter already |
| 129 | +covers rung 2; a published component in between would be a thing to maintain |
| 130 | +that buys nothing the built-ins do not. |
| 131 | + |
| 132 | +## Rung 3: a clustered broker fleet, where rung 1 is wrong |
| 133 | + |
| 134 | +If your brokers are a |
| 135 | +[fleet rather than one Redis](/blog/horizontally-scalable-redis-broker/), the |
| 136 | +backlog is spread across all of them. `ClusteredRedisQueue.queueLength()` sums |
| 137 | +the figure across every broker; a KEDA `redis` trigger pointed at one address |
| 138 | +sees only that broker's share and scales on a fraction of the real backlog. |
| 139 | + |
| 140 | +This is a correctness limit, not an upsell. On a clustered fleet, use rung 2 — |
| 141 | +the service already sums for you — or give KEDA one trigger per broker and accept |
| 142 | +that you are approximating. Rung 1 is the right answer for a single broker and |
| 143 | +the wrong one for a fleet. |
| 144 | + |
| 145 | +## Four properties of the number |
| 146 | + |
| 147 | +Both rungs read the same value, so these apply to both, and they are the |
| 148 | +difference between a scaling loop that behaves and one that oscillates. |
| 149 | + |
| 150 | +**It counts messages waiting in the queue's main list.** Delayed messages that |
| 151 | +are not yet due, and messages already leased to a worker under `safeDelivery`, |
| 152 | +are not included. So it is a *backlog* gauge, not the amount of outstanding work |
| 153 | +— which is usually what you want for scaling, and definitely not what you want |
| 154 | +for a "how much is in flight" dashboard. |
| 155 | + |
| 156 | +**0 means two things, and one of them is an outage.** The figure is `0` while the |
| 157 | +queue's writer is disconnected, which makes a broker outage indistinguishable |
| 158 | +from an empty queue. An autoscaler that trusts it will scale the service to |
| 159 | +nothing at exactly the moment Redis comes back and the backlog is largest. |
| 160 | +**Keep `minReplicas`/`minReplicaCount` above zero.** This is the caveat that |
| 161 | +turns a Redis blip into an incident, and it is the reason scale-to-zero is a bad |
| 162 | +fit here. |
| 163 | + |
| 164 | +**Every replica reports the same figure.** They all read the same queue, so the |
| 165 | +number is a property of the queue and not of a pod. That is why an `External` |
| 166 | +target on the value fits, and why an averaged per-pod target does not — averaging |
| 167 | +a constant across N pods gives you a target that moves as you scale, which is a |
| 168 | +feedback loop with the sign you did not want. |
| 169 | + |
| 170 | +**Under `multiProcess`, the port collides.** Every process that starts the |
| 171 | +service binds the metrics port, so the primary plus N workers all try and N of |
| 172 | +the N+1 fail. Either leave `metricsServer` off in that configuration, or expect |
| 173 | +the noise. (Rung 1 is unaffected, since nothing in the service binds anything.) |
| 174 | + |
| 175 | +One shutdown detail if you use rung 2: the signal handlers close the listener on |
| 176 | +`SIGINT`/`SIGTERM`, but |
| 177 | +[`destroy()`](/api/rpc/latest/rpc.imqservice.destroy/) does not — close |
| 178 | +`service.metricsServer` yourself there, or the open listener keeps the process |
| 179 | +alive after everything else has stopped. |
| 180 | + |
| 181 | +## What this does not give you |
| 182 | + |
| 183 | +Queue depth tells you work is waiting. It does not tell you *why*, and scaling is |
| 184 | +the wrong response to some of the reasons. A backlog caused by a slow downstream |
| 185 | +dependency grows just the same as one caused by genuine load, and adding replicas |
| 186 | +to the first makes it worse — you have just increased the pressure on the thing |
| 187 | +that was already the bottleneck. Scaling on backlog is a better default than |
| 188 | +scaling on CPU; it is not a substitute for knowing where the time goes, which is |
| 189 | +what [tracing](/blog/distributed-tracing-nodejs-message-queue/) is for. |
| 190 | + |
| 191 | +It also does not decide the *broker* layer's capacity. That scales on its own |
| 192 | +axis and by a different mechanism — see |
| 193 | +[auto-scaling the Redis broker](/blog/horizontally-scalable-redis-broker/). |
| 194 | + |
| 195 | +## Reference |
| 196 | + |
| 197 | +[`IMQServiceOptions.metricsServer`](/api/rpc/latest/rpc.imqserviceoptions.metricsserver/) · |
| 198 | +[`IMQMetricsServerOptions`](/api/rpc/latest/rpc.imqmetricsserveroptions/) · |
| 199 | +[`IMQMetricsServerOptions.enabled`](/api/rpc/latest/rpc.imqmetricsserveroptions.enabled/) · |
| 200 | +[`IMQMetricsServerOptions.port`](/api/rpc/latest/rpc.imqmetricsserveroptions.port/) · |
| 201 | +[`IMessageQueue.queueLength()`](/api/core/latest/core.imessagequeue.queuelength/) · |
| 202 | +[`ClusteredRedisQueue.queueLength()`](/api/core/latest/core.clusteredredisqueue.queuelength/) · |
| 203 | +[`IMQOptions.prefix`](/api/core/latest/core.imqoptions.prefix/) · |
| 204 | +[`IMQServiceOptions.multiProcess`](/api/rpc/latest/rpc.imqserviceoptions.multiprocess/) · |
| 205 | +[FAQ: how do I auto-scale @imqueue services?](/api/faq/#how-do-i-auto-scale-imqueue-services) |
0 commit comments