feat(#307): render ClickHouse FORMAT PNG results in Workbench and Dashboard#319
feat(#307): render ClickHouse FORMAT PNG results in Workbench and Dashboard#319BorisTyshkevich wants to merge 9 commits into
Conversation
runQuery gains a binary raw-mode branch (arrayBuffer, normalized image/png content type) for isBinaryFormat() formats; StreamResult carries a validated ImageResultPayload; executeRead validates PNG signature/IHDR/limits via the new pure src/core/png.ts before any bytes reach the UI. PNG added to formatFileMeta. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
Workbench: image results render as a centered, scrollable <img> behind a new injected createObjectUrl/revokeObjectUrl env seam, with a locked 'Image (PNG)' toolbar tab, Copy disabled, and exact-bytes Download PNG. Object URLs are cached per payload (WeakMap) and revoked on rerun replacement, tab close, and format-error overwrite; history records rows:null for image results. Dashboard: new schema-validated 'image' panel type (fit/background/alt) regenerated through the schema pipeline; panelExecution image arm requires authored FORMAT PNG and owns format='PNG'; the viewer-session blanket FORMAT rejection now exempts only image panels; tiles render the image in the full body with object-fit modes + checkerboard background, repaint on image identity (resize never re-executes), and revoke the previous URL via the shared destroy slot in both engines. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
examples/image-panels.json ships four FORMAT PNG demos (spiral, RGB gradient, RGBA checkerboard glow, original ray-traced sphere credited to ClickHouse's RayTracer demos) — all validated live against ClickHouse 26.6. New e2e fixture proves the Workbench Image view and Dashboard tile aspect-preserving fit with real pixel sampling in chromium+webkit. README notes the 26.6+ requirement. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
…acks, gate tests + CHANGELOG The schema-lineage view now revokes a displayed image result's blob URL before replacing tab.result (new revokeResultImage hook, mirroring the workbench-session pattern). The Panel-drawer/readonly image arms thread the active tab name into alt with a 'PNG query result' final fallback. Dashboard blanket-gate rejection now explicitly tested for line and table panel types with authored FORMAT PNG. CHANGELOG [Unreleased] documents the feature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
…capped binary formats Starring a panel-role query with no dashboard document now creates the document (shared createEmptyDashboardDocument helper) instead of silently doing nothing, and Import queries runs the new additive-only syncFavoriteTileMembership so imported favorited panel-role queries get tiles — the documented kpi-panel.json/image-panels.json import flows show tiles again (pre-existing Phase-8 regression exposed by the #307 demos). runQuery no longer sends max_result_rows/result_overflow_mode for binary formats: the source-row cap was truncating large PNGs to their first rows; PNG dimension/byte limits remain the guards. Both verified live against ClickHouse 26.6: image-panels import yields 4 rendering tiles, and the 1024x1024 spiral repro renders full-size. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
…eries A panel-role tile whose query ends in FORMAT PNG but has no explicit panel cfg is now treated as an Image panel (shouldInferImagePanel, pure core helper) instead of hitting the blanket 'remove the explicit FORMAT clause' rejection — starring a PNG query just works. Explicitly-typed non-image panels with FORMAT PNG stay rejected, now with a guided message naming the Image panel type. Owner decision at the merge gate; verified live (save -> star -> dashboard tile renders the PNG). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
…-auth> tag build/local.py only ever emitted the default bearer auth for OAuth connections from ~/.clickhouse-client/config.xml, so the ch-jwt-verify path (stock/OSS ClickHouse behind a Basic-password JWT verifier) could not be exercised locally — the browser always sent Authorization: Bearer, which such a server rejects. A <ch-auth>basic</ch-auth> tag on the connection now mirrors install.sh --ch-auth basic into the generated config.json IdP entry. Docs + README updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
BorisTyshkevich
left a comment
There was a problem hiding this comment.
The core binary-result and Image-panel design is appropriate, and CI is green, but the following issues should be fixed before merge:
-
[P1] Enforce
MAX_PNG_BYTESwhile reading the response.src/net/ch-client.tscurrently callsresponse.arrayBuffer()beforevalidatePng()applies the 64 MiB limit. An oversized response can therefore allocate far beyond the advertised cap and freeze or terminate the tab. Read the body incrementally, reject an oversizedContent-Lengthimmediately when present, cancel the reader after the cap is crossed, and only assemble an in-bounds finalUint8Array. -
[P2] Complete Dashboard renderer disposal. Image cleanup uses the shared renderer
destroyslot, but removed tiles do not call it, and a same-windowrenderDashboard()reload disposes only the old resize listener. The previous signal effect, viewer session, tile renderers, charts, and blob URLs remain undisposed. Add a route-level disposer that disposes the effect/session, destroys every retained renderer, removes the listener, and calls renderer cleanup before pruning removed tile IDs. This includes the behavior currently filed as #318. -
[P2] Revoke image URLs before script export replaces
tab.result. Normal run, schema graph, format-error, and tab-close paths dispose image results, butExportService.exportScript()directly assignsscriptExportResult. Add a result-disposal hook toExportServiceDepsand invoke it before this replacement, with a regression test starting from a rendered image result. -
[P2] Increment Dashboard revision when query import adds tiles.
planImportQueries()can now mutate an existing Dashboard throughsyncFavoriteTileMembership(), but the candidate preserves the previous revision. A successful persisted Dashboard mutation must increment revision exactly once. Keep a newly created Dashboard at revision 1; increment an existing Dashboard only when synchronization actually changes it; preserve revision for a no-op import. -
[P2] Do not accept a header-only/truncated PNG as valid.
validatePng()currently accepts a 29-byte signature+IHDR prefix with no IHDR CRC, image data, or IEND. RequireIHDRlength exactly 13, ensure the complete IHDR data and CRC are in bounds, and require a structurally complete terminalIEND. Also handle<img>decode failure by showing a result error and revoking the URL. -
[P2] Preserve useful ClickHouse errors returned with HTTP 2xx. Every successful binary response is treated as PNG bytes. When validation fails, the user receives only a structural PNG error, even if the body is ClickHouse exception text. Inspect a bounded textual prefix after validation failure and use
parseExceptionText()when it resembles a ClickHouse error; otherwise retain the PNG validation message.
These are merge-blocking from this review. The transport allocation issue is the highest priority; the object-URL lifecycle and Dashboard revision invariant should also be resolved in this PR.
…ull dashboard disposal P1: runQuery's binary branch enforces MAX_PNG_BYTES during transport — oversized Content-Length rejects without reading, chunked reads cancel the reader the moment the cap is crossed, and only an in-bounds payload is assembled (single allocation). P2s: validatePng requires IHDR length exactly 13 with in-bounds CRC and a structurally complete terminal IEND; a 2xx body that fails PNG validation but resembles a ClickHouse exception surfaces via parseExceptionText instead of a structural error; renderDashboard gains a route-level disposer (effect, viewer session, every retained tile renderer, resize listener) and removed tiles destroy their renderer before pruning (implements #318, closes with the PR); exportScript revokes a displayed image result's URL via a new ExportHooks.revokeResultImage before replacing tab.result; <img> decode failures show a result/tile error, revoke the URL, and clear the cache so re-renders mint fresh URLs; query-import tile sync bumps an existing Dashboard's revision exactly once when it actually changes (new doc stays revision 1, no-op imports keep the reference). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB
|
All six review items addressed in f6c1f32:
Gate after all fixes: 147 files / 3800 unit tests green with per-file coverage held, |
What & why
Closes #307. Closes #318 (the review-pass dashboard disposer implements it).
ClickHouse 26.6's
FORMAT PNGoutput now renders as a real image in both the Workbench and Dashboard, over a genuinely binary transport — the old raw path decoded every non-streaming response withResponse.text(), which corrupts PNG bytes.src/net/ch-client.ts+src/core/format.ts):isBinaryFormat()formats are read asarrayBuffer()intoRunQueryResult.binary(Uint8Array, content type normalized toimage/png, never base64). Text raw formats, streaming Table/KPI/Filter, error parsing (wait_end_of_query=1→ non-2xx), and cancellation are unchanged.src/core/png.ts, new, pure, 100%): signature + IHDR + limits (8192×8192, 32M pixels, 64 MiB) run inexecuteReadbefore any object URL can exist; failures surface as a normal result error.PNG query result) — with exact-bytesDownload PNG; Copy is disabled. Blob URLs come from a new injectedcreateObjectUrl/revokeObjectUrlenv seam, cached per payload, and revoked on rerun replacement, tab close, result overwrite (including the schema-lineage view), and format-error overwrite. History recordsrows: null; image bytes are never persisted anywhere.imagepanel type (fitcontain/cover/actual,backgroundtheme/transparent/checkerboard,alt; regenerated via the schema pipeline).panelExecutiongrows an image arm that requires an authoredFORMAT PNG(rejecting missing or non-PNG formats); the viewer-session blanket FORMAT rejection now exempts only image panels — every other panel/role keeps rejecting authored formats, with explicit tests for line/table +FORMAT PNG. Tiles render the image filling the body, repaint on image identity (resize never re-executes), and revoke the previous URL through the shared destroy slot in both layout engines. Refresh uses the standard loading state (owner decision: the issue's keep-last-image wording described behavior no tile type has; matching current tiles was chosen — decisions pinned in the issue comment).examples/image-panels.json+ README): spiral, RGB gradient, RGBA checkerboard glow, and an original ray-traced-sphere homage crediting ClickHouse's RayTracer demos, all small-by-default with a CPU-cost warning — validated live against a ClickHouse 26.6.1 server. Requires ClickHouse 26.6+ (older servers show their normal error).tests/e2e/image-result.*): real-browser proof that a PNG payload renders as a visible<img>behind the locked tab, downloads exact bytes, and letterboxes (pixel-sampled, not CSS-box-checked) in 3:1 and 1:3 dashboard tiles — chromium + webkit green (firefox can't launch in this sandbox, known environmental).Out-of-scope finding filed as inbox #318 (pre-existing: dashboard tile removal skips renderer cleanup — chart instances leak too).
Merge-gate riders (owner-requested, from live testing): favorite/import tile-membership fix (pre-existing Phase-8 regression), uncapped binary formats (large PNGs no longer truncated by the ROWS selector), auto-inferred Image panel for unconfigured
FORMAT PNGqueries, RayTracer README link, and an unrelated owner change riding along:build/local.py<ch-auth>basic</ch-auth>support for ch-jwt-verify OAuth connections.Checklist
npm testpasses (the per-file coverage gate is non-negotiable)npm run buildsucceeds (single-filedist/sql.html)src/core/, network insrc/net/(injected fetch), DOM insrc/ui/CHANGELOG.md([Unreleased]) updated if behavior or the deployed surface changed🤖 Generated with Claude Code
https://claude.ai/code/session_01X8d8DK5DegshRDZ8xfajWB