Skip to content

Commit 386c365

Browse files
authored
[HDX-4769] feat(cli): terminal charting — dashboards TUI page and hdx chart command (#2629)
Linear: [HDX-4769](https://linear.app/clickhouse/issue/HDX-4769/implement-hdx-chart-render-dashboard-tiles-and-ad-hoc-charts-in-the) ## Why When troubleshooting from the terminal — especially agent-driven investigations — there was no way to *see* the data. The CLI could search and tail events, but visualizing a metric ("did errors spike at deploy time?", "what does p95 latency look like?") required switching to the web UI. This PR brings dashboard tiles and ad-hoc charts into the terminal so an investigation loop (chart → spot the anomaly → narrow with `--where` / `hdx query` / `hdx stream`) can happen entirely in the CLI. ## What ### `hdx chart` — terminal charts for CLI/agent troubleshooting Three modes on one command, all flowing through the **same `renderChartConfig` SQL pipeline the web dashboards use** (via `queryChartConfig` inherited from common-utils), so SQL and results match the web UI exactly — including metric-source query splitting/joining: ```bash # 1. Saved dashboard tiles (name or ID) hdx chart -d "Service Health" -t "P95 Latency" --since 24h # 2. Ad-hoc builder charts hdx chart -s Logs --where 'SeverityText:error' --group-by ServiceName --since 3h hdx chart -s Traces --agg quantile --level 0.95 --value Duration hdx chart -s Metrics --metric-type sum --metric-name otelcol_exporter_sent_spans # 3. Ad-hoc raw SQL (supports $__timeFilter / $__timeInterval macros) hdx chart -s Logs --sql 'SELECT $__timeInterval(TimestampTime) AS ts, count() FROM default.otel_logs WHERE $__timeFilter(TimestampTime) GROUP BY ts' ``` - Display types: line, stacked bar, number, table, bar, pie, markdown (heatmap/search/patterns/PromQL show a placeholder) - Time ranges: `--since 1h`, or `--from`/`--to` accepting ISO 8601, date-only, and relative (`now-24h`) — same parser as the TUI time-range editor - Agent-friendly: ANSI colors auto-stripped when stdout is not a TTY (`--color auto|always|never`), `--json` emits raw rows + column metadata, actionable errors with exit 1, help text written as agent documentation ### TUI Dashboards page (`d` key) Dashboard picker → tiles in grid order with j/k navigation, Enter for fullscreen, `t` time-range editor, `r` refresh. Only tiles scrolled into view are mounted/queried (mirrors the web's viewport-gated fetching). ### Architecture Chart renderers are pure functions (`data + dimensions → ANSI string`) shared by the Ink TUI and the stdout command. Ports from the web app carry `@source` comments and follow the existing CLI web-alignment convention: | CLI | Web | |---|---| | `shared/tileConfig.ts` | `DBDashboardPage` Tile config resolution + `ChartUtils` transforms | | `shared/tileRender.ts` | `renderChartContent` dispatch | | `shared/chartData.ts` | `formatResponseForTimeChart` / categorical shaping | | `shared/formatNumber.ts` | `formatNumber` (numbro) + number-format resolution | ### Fixes along the way - Silenced the `@clickhouse/client` internal logger (`ClickHouseLogLevel.OFF`): Ink's `patchConsole` re-enabled console output that `silenceLogs` had suppressed, spewing abort stack traces over the TUI whenever an in-flight tile query was legitimately cancelled - Memoized ClickHouse client/metadata in `App` so re-renders don't churn data-fetch effects - Aligned the pre-commit knip invocation with CI (exclude namespace-only findings CI doesn't count) — the hook failed on a clean checkout of `main` ## Testing - 40 unit tests (`tileConfig` resolution incl. metric/raw-SQL/trace sampling paths, granularity parity with the web's 80-bucket cap, ad-hoc flag → config building, ANSI stripping) - Live-verified against a real deployment: builder/metric/raw-SQL/quantile charts (incl. auto duration formatting), all display types, dashboard + tile lookup by name and ID, `--json`, all three color modes, error paths, and the TUI page via pty - `tsc --noEmit`, `yarn ci:lint`, changeset included New deps (CLI devDependencies, bundled): `asciichart` (zero-dep line plots), `numbro` (number formatting parity with the web). ## Demo ### Dashboard <img width="1583" height="735" alt="image" src="https://github.com/user-attachments/assets/7c30ff91-bbb8-4ddd-870c-65ac065720d2" /> ### chart command <img width="903" height="322" alt="image" src="https://github.com/user-attachments/assets/0739a920-1dab-443e-b39a-b60b81af46b8" />
1 parent 1705b37 commit 386c365

36 files changed

Lines changed: 5257 additions & 49 deletions

‎.changeset/nice-pears-shake.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
'@hyperdx/cli': minor
3+
---
4+
5+
Add terminal charting to the CLI, built on the same renderChartConfig SQL
6+
pipeline as the web dashboards:
7+
8+
- Interactive Dashboards page in the TUI (`d` key) with tile navigation,
9+
fullscreen tiles, time-range editing, and refresh
10+
- `hdx chart` command for troubleshooting from the terminal (including by
11+
AI agents): render saved dashboard tiles (`-d/-t`), ad-hoc builder charts
12+
(`-s <source>` with `--agg/--value/--where/--group-by/--series`), or ad-hoc
13+
raw SQL (`--sql` with `$__timeFilter`/`$__timeInterval` macros)
14+
- Supported chart types: line, stacked bar, number, table, bar, pie, and
15+
markdown; flexible time ranges (`--since 1h`, `--from now-24h`, ISO dates);
16+
`--json` output for structured consumption; ANSI colors auto-stripped when
17+
piping (`--color auto|always|never`)

‎packages/cli/AGENTS.md‎

Lines changed: 86 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,10 @@ hdx tui -s <url> # Interactive TUI (main command)
1717
hdx stream -s <url> --source "Logs" # Non-interactive streaming to stdout
1818
hdx sources -s <url> # List available sources
1919
hdx connections # List ClickHouse connections (id, name, host)
20+
hdx dashboards # List dashboards with tile summaries
21+
hdx chart -d <dashboard> [-t tile] # Render dashboard tiles as ANSI charts
22+
hdx chart -s <source> [--agg ...] # Ad-hoc chart over a source (builder mode)
23+
hdx chart --sql <query> -s <source> # Ad-hoc chart from raw SQL
2024
hdx auth login -s <url> # Sign in (interactive or -e/-p flags)
2125
hdx auth status # Show auth status (reads saved session)
2226
hdx auth logout # Clear saved session
@@ -48,12 +52,27 @@ src/
4852
│ # buildFullRowSql (SELECT * for row detail)
4953
├── components/
5054
│ ├── AlertsPage.tsx # Alerts overview page — list + detail with recent history (Shift+A)
55+
│ ├── DashboardPage.tsx # Dashboards page — picker + tile chart view (d)
56+
│ ├── Tile/ # Dashboard tile charting (mirror of web Tile component)
57+
│ │ ├── TileChart.tsx # Ink wrapper: query + render per displayType
58+
│ │ └── useTileData.ts # React hook around shared/tileQuery fetchTileData
5159
│ ├── EventViewer.tsx # Main TUI view — table, search, detail panel with tabs
5260
│ ├── TraceWaterfall.tsx # Trace waterfall chart with j/k navigation + event details
5361
│ ├── RowOverview.tsx # Structured overview (top-level attrs, event attrs, resource attrs)
5462
│ ├── ColumnValues.tsx # Shared key-value renderer (used by Column Values tab + Event Details)
5563
│ ├── LoginForm.tsx # Email/password login form (used inside TUI App)
5664
│ └── SourcePicker.tsx # Arrow-key source selector
65+
├── shared/
66+
│ ├── tileConfig.ts # Tile → queryable chart config resolution + per-displayType
67+
│ │ # transforms (ports of web DBDashboardPage / ChartUtils)
68+
│ ├── tileQuery.ts # fetchTileData — executes via queryChartConfig (renderChartConfig)
69+
│ ├── tileRender.ts # displayType → ANSI string dispatch (mirror of renderChartContent)
70+
│ ├── chartData.ts # Response shaping (formatResponseForTimeChart etc. ports)
71+
│ └── formatNumber.ts # numbro-based formatNumber + number-format resolution ports
72+
├── termchart/ # Self-contained ANSI chart renderers (only dep: chalk) —
73+
│ # line, stacked bar, categorical, number, table, markdown,
74+
│ # peak-preserving resampling + nice y-axis ticks. No HyperDX
75+
│ # imports; candidate for open-sourcing. See termchart/README.md
5776
└── utils/
5877
├── config.ts # Session persistence (~/.config/hyperdx/cli/session.json)
5978
├── editor.ts # $EDITOR integration for time range and select clause editing
@@ -112,6 +131,56 @@ Port of the web frontend's `DBRowOverviewPanel`. Three sections:
112131
`source.resourceAttributesExpression`, rendered as chips with
113132
`backgroundColor="#3a3a3a"` and cyan key / white value
114133

134+
### DashboardPage + Tile charting (`components/DashboardPage.tsx`, `components/Tile/`)
135+
136+
Dashboard tiles are queried and rendered with the **same SQL pipeline as the
137+
web dashboard**:
138+
139+
1. `shared/tileConfig.ts` resolves a tile's `SavedChartConfig` against its
140+
source (port of the web Tile's `queriedConfig` effect in
141+
`DBDashboardPage.tsx`) and applies the per-displayType config transform
142+
(`convertToTimeChartConfig` / `convertToNumberChartConfig` /
143+
`convertToTableChartConfig` / `convertToCategoricalChartConfig`).
144+
2. `shared/tileQuery.ts` executes it via
145+
`clickhouseClient.queryChartConfig()` from common-utils — internally
146+
`setChartSelectsAlias → splitChartConfigs → renderChartConfig`, identical
147+
to the web's `useQueriedChartConfig` core (minus chunking / MV
148+
optimization / PromQL).
149+
3. `shared/chartData.ts` shapes the response (ports of
150+
`formatResponseForTimeChart`, `formatResponseForCategoricalChart`, etc.).
151+
4. `termchart/` renders pure ANSI strings, consumed by both the Ink
152+
`TileChart` component and the non-interactive `hdx chart` command. It is
153+
a self-contained module (only dependency: chalk) — number formatting is
154+
injected via callbacks, and it must never import from the rest of the
155+
CLI.
156+
157+
Supported display types: line, stacked_bar, number, table, bar, pie,
158+
markdown. Heatmap / search / event patterns / PromQL show a placeholder.
159+
The web's previous-period comparison overlay is not rendered; tiles with
160+
`compareToPreviousPeriod` set show a dim callout instead.
161+
Time-series charts stretch to the full terminal width via peak-preserving
162+
resampling (`resampleSeries`): bucket values are placed exactly at their
163+
nearest column (upscale) or each column keeps its bucket range's
164+
max-magnitude value (downscale) — plain linear interpolation would sample
165+
*between* buckets and attenuate narrow spikes. Stacked bars map columns to
166+
buckets nearest-neighbor (upscale) or by max-total bucket (downscale).
167+
Auto granularity is capped at 80 buckets like the web (`maxTimeBuckets`).
168+
Y-axes use "nice" tick domains (`niceTicks`, steps of 1/2/2.5/5×10ⁿ,
169+
zero-pinned, max rounded up) with sparse tick-row labels — mirroring the
170+
web's recharts `domain={[0, 'auto']}` axis rather than labeling every row
171+
with raw range fractions.
172+
173+
The `hdx chart` command (designed for agent-driven troubleshooting) reuses
174+
this same pipeline in three modes: dashboard tiles (`-d`), ad-hoc builder
175+
charts (`-s <source>` + `--agg/--value/--where/--group-by/--series`, built
176+
by `shared/adhocChart.ts`), and ad-hoc raw SQL (`--sql` with
177+
`$__timeFilter` / `$__timeInterval` macros). ANSI colors are stripped
178+
automatically when stdout is not a TTY (`--color auto|always|never`), and
179+
`--json` emits raw rows + column metadata.
180+
181+
Do NOT change resolution/shaping rules without checking the web components
182+
first (see the alignment table below).
183+
115184
### ColumnValues (`components/ColumnValues.tsx`)
116185

117186
Shared component for rendering key-value pairs from a row data object. Used by:
@@ -127,13 +196,17 @@ This package mirrors several web frontend components. **Always check the
127196
corresponding web component before making changes** to ensure behavior stays
128197
consistent:
129198

130-
| CLI Component | Web Component | Notes |
131-
| ---------------- | ----------------------- | -------------------------------- |
132-
| `TraceWaterfall` | `DBTraceWaterfallChart` | Tree builder is a direct port |
133-
| `RowOverview` | `DBRowOverviewPanel` | Same sections and field list |
134-
| Trace tab logic | `DBTracePanel` | Source resolution (trace/log) |
135-
| Detail panel | `DBRowSidePanel` | Tab structure, highlight hint |
136-
| Event query | `DBTraceWaterfallChart` | `getConfig()` → `buildTrace*Sql` |
199+
| CLI Component | Web Component | Notes |
200+
| ---------------------- | --------------------------------- | ---------------------------------- |
201+
| `TraceWaterfall` | `DBTraceWaterfallChart` | Tree builder is a direct port |
202+
| `RowOverview` | `DBRowOverviewPanel` | Same sections and field list |
203+
| Trace tab logic | `DBTracePanel` | Source resolution (trace/log) |
204+
| Detail panel | `DBRowSidePanel` | Tab structure, highlight hint |
205+
| Event query | `DBTraceWaterfallChart` | `getConfig()` → `buildTrace*Sql` |
206+
| `shared/tileConfig` | `DBDashboardPage` + `ChartUtils` | Tile config resolution/transforms |
207+
| `shared/tileRender` | `DBDashboardPage` | `renderChartContent` dispatch |
208+
| `shared/chartData` | `ChartUtils` | Response shaping ports |
209+
| `shared/formatNumber` | `utils.ts` + `source.ts` | formatNumber + format resolution |
137210

138211
Key expression mappings from the web frontend's `getConfig()`:
139212

@@ -163,12 +236,18 @@ Key expression mappings from the web frontend's `getConfig()`:
163236
| `o` | Open trace in browser (detail panel) |
164237
| `w` | Toggle line wrap |
165238
| `A` (Shift+A) | Open alerts page |
239+
| `d` | Open dashboards page |
166240
| `?` | Toggle help screen |
167241
| `q` | Quit |
168242

169243
In the **Trace tab**, `j`/`k` navigate spans/logs in the waterfall instead of
170244
the main table.
171245

246+
In the **Dashboards page**: `j`/`k` select a tile, `Enter`/`l` opens the tile
247+
fullscreen, `t` edits the time range in `$EDITOR`, `r` refetches all tiles,
248+
`Esc`/`h` goes back (fullscreen → tile list → dashboard picker → previous
249+
screen).
250+
172251
## Development
173252

174253
```bash

‎packages/cli/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@
5454
"ink-spinner": "^5.0.0",
5555
"ink-text-input": "^6.0.0",
5656
"jest": "^30.2.0",
57+
"numbro": "^2.5.0",
5758
"react": "^19.0.0",
5859
"react-devtools-core": "^7.0.1",
5960
"sqlstring": "^2.3.3",

‎packages/cli/src/App.tsx‎

Lines changed: 51 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import React, { useState, useEffect, useCallback } from 'react';
1+
import React, { useState, useEffect, useCallback, useMemo } from 'react';
22
import { Box, Text } from 'ink';
33
import Spinner from 'ink-spinner';
44

@@ -10,12 +10,19 @@ import {
1010
type SavedSearchResponse,
1111
} from '@/api/client';
1212
import AlertsPage from '@/components/AlertsPage';
13+
import DashboardPage from '@/components/DashboardPage';
1314
import ErrorDisplay from '@/components/ErrorDisplay';
1415
import LoginForm from '@/components/LoginForm';
1516
import SourcePicker from '@/components/SourcePicker';
1617
import EventViewer from '@/components/EventViewer';
1718

18-
type Screen = 'loading' | 'login' | 'pick-source' | 'events' | 'alerts';
19+
type Screen =
20+
| 'loading'
21+
| 'login'
22+
| 'pick-source'
23+
| 'events'
24+
| 'alerts'
25+
| 'dashboards';
1926

2027
interface AppProps {
2128
appUrl: string;
@@ -33,13 +40,24 @@ export default function App({ appUrl, query, sourceName, follow }: AppProps) {
3340
const [currentAppUrl, setCurrentAppUrl] = useState(appUrl);
3441
const [sessionExpired, setSessionExpired] = useState(false);
3542
const [eventSources, setLogSources] = useState<SourceResponse[]>([]);
43+
// All sources (incl. metric/session) — dashboard tiles may reference any kind
44+
const [allSources, setAllSources] = useState<SourceResponse[]>([]);
3645
const [savedSearches, setSavedSearches] = useState<SavedSearchResponse[]>([]);
3746
const [selectedSource, setSelectedSource] = useState<SourceResponse | null>(
3847
null,
3948
);
4049
const [activeQuery, setActiveQuery] = useState(query ?? '');
4150
const [error, setError] = useState<string | null>(null);
4251

52+
// Stable ClickHouse client + metadata instances — recreating them on
53+
// every render would re-trigger data-fetch effects downstream (each
54+
// effect aborts its in-flight query when its deps change).
55+
const clickhouseClient = useMemo(
56+
() => client.createClickHouseClient(),
57+
[client],
58+
);
59+
const metadata = useMemo(() => client.createMetadata(), [client]);
60+
4361
// Check existing session on mount
4462
useEffect(() => {
4563
(async () => {
@@ -60,6 +78,8 @@ export default function App({ appUrl, query, sourceName, follow }: AppProps) {
6078
apiClient.getSavedSearches().catch(() => [] as SavedSearchResponse[]),
6179
]);
6280

81+
setAllSources(sources);
82+
6383
const queryableSources = sources.filter(
6484
s => s.kind === SourceKind.Log || s.kind === SourceKind.Trace,
6585
);
@@ -146,17 +166,26 @@ export default function App({ appUrl, query, sourceName, follow }: AppProps) {
146166
[eventSources],
147167
);
148168

149-
// Track the screen before alerts so we can return to it
150-
const [preAlertsScreen, setPreAlertsScreen] = useState<Screen>('events');
169+
// Track the screen before alerts/dashboards so we can return to it
170+
const [preOverlayScreen, setPreOverlayScreen] = useState<Screen>('events');
151171

152172
const handleOpenAlerts = useCallback(() => {
153-
setPreAlertsScreen(screen);
173+
setPreOverlayScreen(screen);
154174
setScreen('alerts');
155175
}, [screen]);
156176

157177
const handleCloseAlerts = useCallback(() => {
158-
setScreen(preAlertsScreen);
159-
}, [preAlertsScreen]);
178+
setScreen(preOverlayScreen);
179+
}, [preOverlayScreen]);
180+
181+
const handleOpenDashboards = useCallback(() => {
182+
setPreOverlayScreen(screen);
183+
setScreen('dashboards');
184+
}, [screen]);
185+
186+
const handleCloseDashboards = useCallback(() => {
187+
setScreen(preOverlayScreen);
188+
}, [preOverlayScreen]);
160189

161190
if (error) {
162191
return (
@@ -202,25 +231,38 @@ export default function App({ appUrl, query, sourceName, follow }: AppProps) {
202231
sources={eventSources}
203232
onSelect={handleSourceSelect}
204233
onOpenAlerts={handleOpenAlerts}
234+
onOpenDashboards={handleOpenDashboards}
205235
/>
206236
</Box>
207237
);
208238

209239
case 'alerts':
210240
return <AlertsPage client={client} onClose={handleCloseAlerts} />;
211241

242+
case 'dashboards':
243+
return (
244+
<DashboardPage
245+
client={client}
246+
clickhouseClient={clickhouseClient}
247+
metadata={metadata}
248+
sources={allSources}
249+
onClose={handleCloseDashboards}
250+
/>
251+
);
252+
212253
case 'events':
213254
if (!selectedSource) return null;
214255
return (
215256
<EventViewer
216-
clickhouseClient={client.createClickHouseClient()}
217-
metadata={client.createMetadata()}
257+
clickhouseClient={clickhouseClient}
258+
metadata={metadata}
218259
appUrl={currentAppUrl}
219260
source={selectedSource}
220261
sources={eventSources}
221262
savedSearches={savedSearches}
222263
onSavedSearchSelect={handleSavedSearchSelect}
223264
onOpenAlerts={handleOpenAlerts}
265+
onOpenDashboards={handleOpenDashboards}
224266
initialQuery={activeQuery}
225267
follow={follow}
226268
/>

0 commit comments

Comments
 (0)