refactor(#359): one FilterSourceRuntime per unique Filter source query#361
Conversation
Dashboard filters sharing one Filter-source query rendered empty: the viewer ran the source once per definition and keyed each provider by the filter-definition id, so mergeDashboardFilterHelpers rejected every helper as a duplicate provider and the diagnostics were silently discarded. Split the filter runtime into per-definition state and one FilterSourceRuntime per unique sourceQueryId: - the source SQL executes exactly once per refresh wave, no matter how many filters/parameters it feeds; - the provider is keyed by the saved source-query id, so two DISTINCT sources providing the same helper still collide with no arbitrary winner; - merge diagnostics (info/warning/error, severity preserved) are published to the Dashboard instead of dropped; - non-ready results CLEAR curated options (no silent stale retention); an unresolvable sourceQueryId surfaces a visible source-error; - an active value absent from refreshed options is deactivated (value kept) before dependent panels run; - a per-filter optionsRev drives the filter-bar rebuild on option-content change (fixes non-empty -> different-non-empty not updating the combobox); - a superseded refresh wave can no longer publish options/activation over a fresher one (stale-generation guard in the terminal merge step). filterExecution no longer injects readonly: server read-only policy belongs to ClickHouse user/profile config, not feature code (panel execution unchanged). This establishes the one-source-runtime boundary #360 builds on. Closes #359. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UDsUDSPoDYa1M1rbpgdG3f
|
Review findings for
What looks correct: one runtime per unique I could not submit |
…ined provider Resolves the three P1 review findings on PR #361: 1. E2E was red and misleading. The shared-source case built two standalone comboboxes from hand-split arrays and never touched the viewer. Replaced with a REAL DashboardViewerSession integration: two filter definitions share one Filter source, the harness asserts the source executed exactly once (window.__sharedRequests === 1), both consumers resolve to `ready`, and each filter's curated column populates a real buildFilterOptionField control. Adds the @preact/signals-core import-map entry the unbundled harness needs. 2. Missing helper was silent. When a source succeeds but omits a consumer's column, applyFilterProviders now publishes a `filter-helper-missing` warning (severity warning, sourceId + helperName, naming the source and column) in addition to setting `missing-helper` — the UI renders diagnostics, not per-filter status, so the control is no longer unexplained. 3. Provider not retained per source. FilterSourceRuntime now stores `provider: FilterProvider | null`, updated only for the current generation; the terminal merge reads the COMPLETE set from every source runtime, not just the wave's returns. A future selective wave (#360) can re-merge unaffected sources' helpers instead of clearing or rerunning them. Full gate green (4421 tests, tsc, build); new E2E verified loading + one-request behavior in a local Chromium harness run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UDsUDSPoDYa1M1rbpgdG3f
The new shared-source integration E2E asserted the raw onCommit COUNT (=1) after an option click. A curated combobox whose committed label equals an exact option value re-fires a same-value commit via the blur strictCommit path, so the count is 2 — value and active are correct on all engines (Chromium/Firefox/WebKit confirmed). The commit-fires-once contract is the widget's own concern, already covered by the single-field test above; the shared-source test now asserts only the committed value+active, which is what #359 is about. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UDsUDSPoDYa1M1rbpgdG3f
|
Re-review at The three prior blockers are resolved:
I also rechecked the surrounding behavior: unique execution per CI is green ( |
…dependencies (#362) * feat(#360): allow parameters in Filter sources; add shared prepareFilterSource (core) Filter-role source SQL may now declare its own {name:Type} query parameters: the filter-source-parameters diagnostic is removed from filterSqlDiagnostics (structural rules — empty / statement-count / row-returning / owned-FORMAT — stay). Two new pure functions wrap the shared param-pipeline so both the Dashboard viewer and the Workbench Filter preview prepare a source identically (no independently-implemented binding): - analyzeFilterSource(sql, {sourceBackedParams, label}) -> {sql, analysis, dependsOn, diagnostics}: structural + single-layer cascading diagnostics (a dep backed by ANOTHER Filter source -> filter-source-cascading) and the declared param names, derived once per SQL edit. - prepareFilterSource(analyzed, {values, active, wallNowMs}) -> readiness ('runnable'|'waiting'|'error') + owned transport params (caps only, no readonly) merged with native param_<name> args + materialized execSql + bound-param snapshots. missing -> waiting; invalid/errors/diagnostics -> error; else runnable. validationMode is always 'execute'. Foundation for #360 (Wave 1). Dashboard runtime + Workbench parity follow. filter-execution.ts stays 100/100/100/100; full gate + build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(#360): parameterized Filter sources in the Dashboard viewer runtime The Dashboard viewer now runs parameterized Filter-role source queries fed by committed root filter values (native param_<name>), with single-layer dependency scheduling on top of #359's one-runtime-per-source boundary. - FilterSourceRuntime gains `analyzed` (analyzeFilterSource at construction, with the dashboard's source-backed param set for cascading rejection) and a `waiting` status; each wave prepares the source via prepareFilterSource against committedRootValues() (inactive root filters blanked so they gate) and one deps.wallNow() reading per wave (relative {from}/{to} share one clock). - A source issues NO request while waiting (missing deps) or errored (invalid value / structural / cascading); runnable sources execute the materialized execSql with the bound args, then recordBoundParams for parity. - Selective rerun: committing a root filter reruns ONLY the sources whose SQL declares it (runFilterSourceWave), then folds the changed param plus any reconciliation-deactivated names into ONE combined affected-panel wave (commitAndRerun); the selective wave preflights its own token. - applyFilterProviders returns the flipped (deactivated) names and, per the plan review, skips re-deriving a mid-flight non-plan source's consumers so an overlapping selective wave can't flip a still-loading source back to stale options. Published per-filter view gains optional `stale`/`waitingFor`. - Fixes the stray NUL byte in optionsSignature (from #361). Wave 2a of #360. Filter-bar waiting/stale UI (2b) + Workbench parity (3) follow. dashboard-viewer-session.ts: 100% stmts/fns/lines, 94.75% branches (gate ≥90). Full gate + build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(#360): filter-bar waiting/stale/error affordance for source-backed filters A source-backed Dashboard filter with no options (waiting on a root dependency, mid-flight, or errored) used to fall out of the curated rendering path and render as a bare, unlabelled plain field — the plan-review BLOCKER-2. Now: - rebuildFilterBar gates the curated path on `f.status !== 'idle'` (a plain filter stays 'idle' forever), forwarding status/stale/waitingFor, so a source-backed filter stays curated regardless of whether it has options yet. - The barSig rebuild trigger gains a sticky-status term (waiting + the three error statuses + waitingFor) so an idle→waiting transition with unchanged options actually repaints; bare 'loading' is excluded to preserve #359's same-content-refresh-never-rebuilds invariant. - filter-bar renders a STRUCTURAL, test-observable affordance (happy-dom sees no CSS): is-waiting (disabled + "Waiting for: <names>" text), is-error (disabled), is-stale (disabled, last options kept but marked not-current). - filter-option-field.ts is untouched (affordance applied onto the input it already returns) — zero cross-wave contract risk. Wave 2b of #360. filter-bar.ts 100% (branches 97.18); dashboard.ts floor held. Full gate + build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#360): clear a filter's options during a selective rerun so stale ones can't appear current Per the issue's "Error and stale-result behavior": when a dependency changes, prior options must not continue to appear current while the new source loads. runFilterSourceWave (the dependency-changed path only) now clears each affected consumer's options (setConsumerOptions null) when marking it loading, so the filter-bar repaints the is-stale/loading affordance instead of showing the old options as fresh; applyFilterProviders repopulates from the settled merge. runFilterWave (full refresh) is untouched, preserving #359's no-flicker-on-same-content invariant. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(#360): Workbench Filter preview uses the shared prepareFilterSource (parity) The Workbench Filter-tab preview now prepares its source through the exact same analyzeFilterSource/prepareFilterSource pipeline the Dashboard uses, instead of the old standalone filterExecution() — so structural diagnostics, relative-value resolution (one wave clock), optional-block materialization, validation, serialization, and native param_<name> args are identical on both surfaces (the issue's "do not independently implement parameter binding"). - WorkbenchParameterSession.prepareFilterPreview(sql, wallNowMs) wraps the shared op with the var-strip's values/active (no sourceBackedParams — the Workbench has no Dashboard topology), mirroring prepareTabSource; wired through app.ts's createWorkbenchSession hooks and the fake-app test stub. - run()'s Filter branch: error/waiting return before any request; runnable executes execSql with the bound params and records boundParams. The generic var-gate/toast stays bypassed for Filter tabs (kept the !isFilter guard) so a missing param yields the non-error 'waiting' preview, not a toast-block. - filter-preview gains a 'waiting' state ("Waiting for: <names>"). The role-agnostic variable strip still surfaces the tab's {from:DateTime} controls — the intended way to type test values. Wave 3 of #360. workbench-parameter-session.ts + filter-preview.ts 100%; workbench-session.ts 100% stmts/fns/lines. Full gate + build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#360): preflight once per commit for the selective source+panel waves (review) Code review found commitAndRerun double-preflighted: when a committed root param feeds a Filter source, both runFilterSourceWave and runAffectedWave called preflight() — so ensureFreshToken ran twice per commit and, on a stale token, onAuthFailed fired twice (double toast/redirect). Now the affected path preflights exactly once in commitAndRerun; runFilterSourceWave (only ever reached from that path) no longer preflights, and runAffectedWave keeps its own guard (it is still reached directly on the no-affected-source fast path, whose timing is unchanged). The regression test now asserts ensureFreshToken and onAuthFailed fire exactly once for one affected commit; added a test that clearFilter on a dependency root re-gates the dependent source to 'waiting' (the retained value is blanked, not stale-executed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(#360): skip redundant tab analysis for Filter tabs; style the filter affordance (review) - workbench-session run(): a Filter tab prepares via the shared filterPrep, so it no longer also runs the generic prepareTabSource over the same SQL (`src` is null for Filter tabs — it was only ever read on the non-Filter path). - styles.css: distinguish the source-backed filter transport states so a disabled field's reason reads at a glance — is-error (red), is-waiting (dashed + "Waiting for: …" note), is-stale (dimmed/italic while reloading); plus the waiting filter-preview message. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(#360): migrate the query-log-explorer example + CHANGELOG (reconcile) - examples/query-log-explorer.json: the qle-filter source now scopes its user / query_kind option lists to the selected half-open {from:DateTime}/{to:DateTime} window (matching the panels, which already use from/to); from/to default to -1d/now active so the whole example runs out-of-box and demonstrates the waiting→ready transition when the range changes. Re-normalized. - CHANGELOG [Unreleased]: document #360 (Added) and the stray-NUL fix (Fixed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(#360): e2e — parameterized Filter source waits then runs once with native params Extends the raw filter-source harness with a parameterized source (`geo`) depending on two root filters (from/to). Asserts, in a real browser (chromium + webkit): both deps inactive → status 'waiting', waitingFor [from, to], zero requests; committing from+to → status 'ready', exactly one executeRead, params carry param_from/param_to (no textual interpolation). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#360): Filter source param type-conflicts are errors, not runnable (review) analyzeFilterSource now folds the shared pipeline's own analysis.diagnostics (param type-conflict, e.g. {x:UInt8} vs {x:String}) into its diagnostics as filter-source-param-type-conflict — so prepareFilterSource classifies such a source 'error' (no request) instead of runnable, restoring parity with the generic param pipeline on both Dashboard and Workbench. Also extract filterOwnedParams() so prepareFilterSource no longer calls filterExecution just to recover the transport caps (which re-ran the structural diagnostics). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#360): a superseded or destroyed selective commit no longer runs its panel wave (review) Findings 1+2: commitAndRerun ran the affected-panel wave unconditionally after the source wave, so a commit superseded by a newer one (or by destroy()) still issued panel requests and could bind stale source-backed values. Now: - a commit-level generation guards both phases: after the source wave, commitAndRerun bails (no panel wave) if destroyed, if a newer commit bumped the generation, or if the source wave reports 'superseded' — flushing any genuinely-applied source merge via publish() first (a superseded result mutated nothing, so that's a harmless replay); - applyFilterProviders / runFilterSourceWave now return a discriminated { status:'applied', flipped } | { status:'superseded' } (an empty array couldn't distinguish stale from applied-nothing-flipped); - runAffectedWave gets an unconditional destroyed guard (the preflighted:true fast path previously had none). Finding 6: extract executeFilterSourcePlan shared by runFilterWave (keep options, reset diagnostics) and runFilterSourceWave (clear options, keep diagnostics), removing the duplicated clock/plan/supersede/loading/publish/pool/ merge scaffolding. Also publishes explicit ViewerFilterState.sourceId so the filter bar can choose the curated renderer by topology, not transient status (UI fix follows). dashboard-viewer-session.ts: 100% stmts/fns/lines, 94.24% branches. Gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#360): render source-backed filters by topology, update status imperatively (review) Maintainer review: a source-backed filter's curated identity was inferred from transient status (`f.status !== 'idle'`), so it rendered as a plain text control on initial load, and the loading/stale affordance never showed on a full refresh with existing options (barSig excludes 'loading'). Status is execution state, not topology. - rebuildFilterBar now gates the curated renderer on the explicit `ViewerFilterState.sourceId` (set once at construction) — a source-backed filter is curated at every status, including its initial 'idle'. - buildFilterBar returns `updateStatus(states)`; each curated field keeps a handle and a shared applyFieldStatus() maps status → affordance (ready=enabled; waiting=disabled+is-waiting+note; error states=disabled+ is-error; loading/idle/stale=disabled+is-stale). Applied at build time and on status-only changes WITHOUT rebuilding, so the same input persists and in-progress typing survives. - The render effect splits its signature: barSig is now purely structural ([id, active, value, optionsRev, sourceId!=null]); a separate statusSig drives the imperative updateStatus. Preserves #359's no-flicker-on-same-content invariant while actually showing loading/stale/waiting. filter-bar.ts 100/100/100 (97.53 branches); dashboard.ts floor held. Gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(#360): make the example qle-filter {to} optional to match its panels (review) Maintainer review: the migrated source made both from and to required, but the dashboard's panels treat to as optional (/*[ ... <= {to} ]*/, blank = up to now). Align the source to the same contract — from required, to optional and inclusive — so clearing to no longer forces the source into waiting while the panels keep running. CHANGELOG wording updated to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#360): drop over-broad commit-generation guard; rely on per-source supersession (review round 2) The commit-level generation counter (added last round) was bumped by EVERY commit — including unrelated fast-path ones — so an unrelated overlapping commit (e.g. Tab-then-Enter across two fields) made a legitimate, non-superseded commit skip its affected-panel wave, leaving its tile silently bound to the old value. The per-source generation check in applyFilterProviders (surfaced as result.status === 'superseded') already covers every real supersession — the same-param overlap (the newer commit supersede()s the shared source), a concurrent full refresh() (runFilterWave supersedes all sources), and destroy() — without penalizing two commits that touch genuinely unrelated sources. So commitAndRerun now guards only on `destroyed || result.status === 'superseded'`; the commitGeneration counter and its publish() workaround are removed. Added a regression test: two unrelated overlapping commits each run their own panel wave. dashboard-viewer-session.ts: 100% stmts/fns/lines, 94.21% branches. Gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#360): full refresh defers affected panels when its Filter wave was superseded (review round 3) runFilterWave discarded its SourceWaveResult, claiming a full refresh has no later phase — but refresh() runs the affected-panel wave after the filter wave. So a refresh whose Filter wave was superseded by a concurrent selective commit still ran its affected panels before the superseding wave's source updates + reconciliation, unprotected by tile generations (the selective commit reserves its own tile generations only later, inside its runAffectedWave). runFilterWave now returns SourceWaveResult, and refresh() skips its affected-panel wave when that result is 'superseded' — the superseding selective commit is the current source-of-truth and runs those panels after it settles. Regression test added (verified it fails without the guard). dashboard-viewer-session.ts: 100% stmts/fns/lines, 93.6% branches. Gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(#360): trim review-history comments to stable invariants (review) Maintainer review: several production comments documented review rounds, "finding N", and abandoned approaches rather than stable invariants — notably commitAndRerun's ~30-line description of the reverted commit-generation counter. Condensed the #360-introduced review narrative to present-tense invariants that link #360, leaving the review history in GitHub. commitAndRerun's doc comment: 49→17 lines. Comment-only (build byte-identical); comments belonging to other issues (#359/#170/#173/#276/#291/#321/#344/#347) left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What & why
Closes #359.
Dashboard filters that shared one Filter-source query rendered empty. The
viewer ran the source query once per filter definition and keyed each
provider by the filter-definition id, so
mergeDashboardFilterHelperssaw Nidentical providers for every helper name and rejected them all as
filter-duplicate-provider— and the explanatory diagnostics were silentlydiscarded. (
examples/query-log-explorer.jsonhits this:userandquery_kindboth point atqle-filter.)This refactor splits the filter runtime into per-definition state and one
FilterSourceRuntimeper uniquesourceQueryId:filters/parameters it feeds;
sources providing the same helper still collide with no arbitrary winner;
published to the Dashboard instead of dropped;
unresolvable
sourceQueryIdsurfaces a visiblesource-error;before dependent panels run;
optionsRevdrives the filter-bar rebuild on option-contentchange (fixes non-empty → different-non-empty not updating the combobox);
one (stale-generation guard in the terminal merge step — found in self-review).
Per the issue's Settings ownership section,
filterExecutionno longerinjects
readonly(server read-only policy belongs to ClickHouse user/profileconfig, not feature code). Panel execution is unchanged. This establishes the
one-source-runtime boundary that #360 (parameterized sources + single-layer
dependency scheduling) builds on.
Scope / limitations
core/dashboard-filters.ts) is unchanged — the fix is inviewer source planning, state publication, and the UI rebuild/diagnostics.
existing
filter-sourceharness (two curated fields from one bundle renderingindependently); no e2e harness in the repo mounts the full viewer session, so
the "executes once / merge / dedup" behavior is covered by the unit suite.
dashboard-viewer-session.tsbranch coverage is ~94% (clears the enforced≥90 floor); the uncovered branches are pre-existing defensive ones plus a few
synchronous stale-gen entry guards that only a contrived concurrency race
would exercise — the config's own note prefers this over artificial padding.
Checklist
npm testpasses (the per-file coverage gate is non-negotiable) — 157 files / 4421 testsnpm run buildsucceeds (single-filedist/sql.html)src/core/, network insrc/net/(injected fetch), DOM insrc/ui/CHANGELOG.md([Unreleased]) updated if behavior or the deployed surface changed🤖 Generated with Claude Code