You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(webapp): ignore stale ClickHouse gauges in the live queue blocks
Gauges are only emitted while a queue is active, so a drained queue's newest
bucket holds its last non-zero reading. Trust the CH gauge only when its newest
bucket is recent; past that, fall back to the loader's Redis/PG value instead of
lingering on a stale count.
Copy file name to clipboardExpand all lines: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues_.$queueParam/route.tsx
+16-4Lines changed: 16 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -906,6 +906,10 @@ function KeyDrilldown({
906
906
// signals the queue is stuck, not just busy.
907
907
constOLDEST_WAIT_WARNING_MS=5*60_000;
908
908
909
+
// How recent the newest ClickHouse gauge bucket must be to drive the live blocks. Above the 10s
910
+
// bucket + pipeline lag; past it we treat the queue as idle and fall back to the loader value.
911
+
constLIVE_GAUGE_FRESH_MS=90_000;
912
+
909
913
functionQueueStats({
910
914
queue,
911
915
environmentConcurrencyLimit,
@@ -951,11 +955,19 @@ function QueueStats({
951
955
refreshIntervalMs: 15_000,
952
956
}
953
957
);
958
+
// Gauges are only emitted while the queue is active, so a drained queue's newest bucket is a past
959
+
// one holding its last non-zero reading. Trust the CH gauge only when its newest bucket is recent
960
+
// (covers the 10s bucket + pipeline lag); once it ages out we fall back to the loader's live
961
+
// Redis/PG value instead of lingering on a stale count.
0 commit comments