Skip to content

Commit 8c517fb

Browse files
committed
feat(webapp): dedicated ClickHouse service for queue metrics
Queue metrics can now run on their own ClickHouse service via QUEUE_METRICS_CLICKHOUSE_URL: the ingestion consumer inserts through it and every queue-metrics read goes through it, so metrics traffic never competes with runs-list or trace reads. A table names the pool its reads run on, so the dashboards, the Query page and the health report route by table instead of each caller picking a client. Unset, the wiring is unchanged: inserts on CLICKHOUSE_URL, reads on the query pool. Also drops the queue-metrics seed simulator, splits the release notes so percent-based queue limits and the reject-above-the-environment-limit change get their own entries, and replaces a literal NUL byte in a cache key with an escape, which had been making the health report's data layer diff as a binary file.
1 parent 47bf3aa commit 8c517fb

19 files changed

Lines changed: 144 additions & 1004 deletions
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"trigger.dev": patch
3+
---
4+
5+
`trigger mcp` now always starts the MCP server, and the interactive install wizard has moved behind `trigger mcp --install`. Previously the wizard opened whenever stdout was a terminal, so any MCP host that spawns the command over a pseudo-terminal waited on a server that never started and eventually timed out.

.changeset/report-health.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33
"trigger.dev": patch
44
---
55

6-
Add the `get_report` MCP tool, a `trigger report` CLI command, and `GET /api/v1/reports/:key`, starting with the `health` report: a deterministic verdict on whether work is flowing, whether the runs that start are healthy, and whether telemetry is fresh — rendered as text with unicode sparklines (coloured in a terminal). Also adds a `report` MCP prompt, surfaced as a slash command in hosts that support prompts.
6+
Ask whether an environment is healthy and get an answer instead of a wall of charts. `trigger report health` returns a verdict on three questions: is work flowing, are the runs that start succeeding, and is the telemetry fresh enough to trust either answer. When something looks wrong it names the most likely cause and a next action.
77

8-
Also: `trigger mcp` now always starts the server — the install wizard is gated behind `trigger mcp --install`. A TTY previously launched the wizard, so MCP hosts spawning the server over a PTY timed out.
8+
```bash
9+
npx trigger.dev@latest report health --env prod --period 24h
10+
```
11+
12+
The verdict is computed server side, so the CLI, the new `get_report` MCP tool, and `GET /api/v1/reports/health` all return the same text with the same sparklines. In MCP hosts that support prompts, `report` is also available as a slash command.

.server-changes/agent-detail-metrics-layout.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@ area: webapp
33
type: improvement
44
---
55

6-
The agent detail page now matches the shared metrics-page layout: the time filter and pagination sit in a filter row at the top, the activity charts form a tile row beneath it, and the tabs and table flow below in a single page scroll, with the agent config panel as a resizable sidebar on the right.
6+
The agent detail page now uses the same layout as the other metrics pages: filters stay pinned at the top, the activity charts sit in a row beneath them, and the tabs and table scroll with the page, with the agent config in a resizable panel on the right.

.server-changes/paginate-concurrency-keys-table.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@ area: webapp
33
type: improvement
44
---
55

6-
The concurrency keys table on a queue's page is now paginated, so queues with thousands of keys can page through all of them instead of only showing the top 50.
6+
A queue's concurrency keys are now paged and searchable, so a queue with thousands of keys shows all of them instead of only the busiest 50.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: feature
4+
---
5+
6+
A queue's concurrency limit can now be set as a percentage of your environment limit, from the dashboard or the API. Percentage limits track the environment limit, so raising or lowering it re-divides capacity without you editing every queue.

.server-changes/queue-metrics-dashboard.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@ area: webapp
33
type: feature
44
---
55

6-
New Queue metrics & health dashboard (per-org opt-in): per-queue depth, throughput, concurrency, throttling and scheduling-delay charts, a per-queue detail view, and live queue stats. Queue concurrency limits set above the environment limit are now rejected instead of being silently capped.
6+
See how every queue in an environment is doing: backlog, throughput, concurrency against its limit, when it was throttled, and how long runs waited before starting. Each queue also gets its own page, which breaks the same numbers down per concurrency key so you can tell whether one key is starving the rest.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: breaking
4+
---
5+
6+
Setting a queue concurrency limit higher than your environment limit is now rejected with an error instead of being silently reduced to the environment limit, so a queue never looks like it has more capacity than it can get. Limits already saved are unchanged.

apps/webapp/app/env.server.ts

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1952,6 +1952,22 @@ const EnvironmentSchema = z
19521952
.enum(["log", "error", "warn", "info", "debug"])
19531953
.default("info"),
19541954
RUNS_LIST_CLICKHOUSE_COMPRESSION_REQUEST: z.string().default("1"),
1955+
/**
1956+
* Dedicated ClickHouse service for queue metrics: the ingestion consumer's inserts and every
1957+
* queue-metrics read (dashboards, queue pages, run inspector, health report) go through it, so
1958+
* metrics traffic never competes with runs-list or trace reads. Unset keeps the previous
1959+
* wiring: inserts on CLICKHOUSE_URL, reads on the query pool.
1960+
*/
1961+
QUEUE_METRICS_CLICKHOUSE_URL: z.string().optional(),
1962+
/** Reader split so the consumer's inserts can never land on a read endpoint. Defaults to QUEUE_METRICS_CLICKHOUSE_URL. */
1963+
QUEUE_METRICS_CLICKHOUSE_READER_URL: z.string().optional(),
1964+
QUEUE_METRICS_CLICKHOUSE_KEEP_ALIVE_ENABLED: z.string().default("1"),
1965+
QUEUE_METRICS_CLICKHOUSE_KEEP_ALIVE_IDLE_SOCKET_TTL_MS: z.coerce.number().int().optional(),
1966+
QUEUE_METRICS_CLICKHOUSE_MAX_OPEN_CONNECTIONS: z.coerce.number().int().default(10),
1967+
QUEUE_METRICS_CLICKHOUSE_LOG_LEVEL: z
1968+
.enum(["log", "error", "warn", "info", "debug"])
1969+
.default("info"),
1970+
QUEUE_METRICS_CLICKHOUSE_COMPRESSION_REQUEST: z.string().default("1"),
19551971
EVENTS_CLICKHOUSE_BATCH_SIZE: z.coerce.number().int().default(1000),
19561972
EVENTS_CLICKHOUSE_FLUSH_INTERVAL_MS: z.coerce.number().int().default(1000),
19571973
METRICS_CLICKHOUSE_BATCH_SIZE: z.coerce.number().int().default(10000),

apps/webapp/app/presenters/v3/QueueMetricsPresenter.server.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ export class QueueMetricsPresenter {
7272
try {
7373
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
7474
environment.organizationId,
75-
"query"
75+
"queueMetrics"
7676
);
7777

7878
// End bound snaps up to the bucket grid so repeated loads within a bucket produce

apps/webapp/app/presenters/v3/RunQueueMetricsPresenter.server.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,7 @@ async function delaySummary(input: {
145145
try {
146146
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
147147
input.organizationId,
148-
"query"
148+
"queueMetrics"
149149
);
150150

151151
const endMs = Math.ceil(Date.now() / DELAY_GRID_MS) * DELAY_GRID_MS;

0 commit comments

Comments
 (0)