Follow-up agreed in the #343 review thread (#343 (comment), continuing #343 (comment)): degraded cache tracking is currently only observable when a pipe first executes (debug logs + the wavehouse_pipe_dep_resolution_fallback_total / wavehouse_pipe_dep_unresolved_total metrics). An operator authoring a pipe that reads s3(), a cross-database table, or a non-local dictionary — or one that can't be analyzed at all — should hear about it when the pipe is created or loaded, not discover a 10-second TTL cap in production.
Agreed design (from the thread)
The invalidation path stays exactly as #343 shipped it — lazy, per bound query, because the dependency set is parameter-dependent. The diagnostic doesn't need real parameter values: a table function / cross-database read shows up in EXPLAIN QUERY TREE regardless of what's bound, so we can dummy-bind the pipe's SQL and run the same EXPLAIN purely as a warning check:
PUT /v1/pipes/{name} returns 200 with a warnings array (synchronous, best-effort — requires ClickHouse reachable; no warning path blocks the save, pipes are never rejected).- Bootstrap (
WH_PIPES_DIR seed files) logs a warning per affected pipe at load.
- A schema refresh that newly degrades a pipe (e.g. a view it reads became unfoldable) logs it.
- A pipe whose EXPLAIN fails outright at dummy-bind time (a write/DDL pipe, a parameter used as a whole table name) gets a "can't be cache-tracked; falls back to database-version invalidation" warning the same way.
Notes
Follow-up agreed in the #343 review thread (#343 (comment), continuing #343 (comment)): degraded cache tracking is currently only observable when a pipe first executes (debug logs + the
wavehouse_pipe_dep_resolution_fallback_total/wavehouse_pipe_dep_unresolved_totalmetrics). An operator authoring a pipe that readss3(), a cross-database table, or a non-local dictionary — or one that can't be analyzed at all — should hear about it when the pipe is created or loaded, not discover a 10-second TTL cap in production.Agreed design (from the thread)
The invalidation path stays exactly as #343 shipped it — lazy, per bound query, because the dependency set is parameter-dependent. The diagnostic doesn't need real parameter values: a table function / cross-database read shows up in
EXPLAIN QUERY TREEregardless of what's bound, so we can dummy-bind the pipe's SQL and run the same EXPLAIN purely as a warning check:PUT /v1/pipes/{name}returns200with awarningsarray (synchronous, best-effort — requires ClickHouse reachable; no warning path blocks the save, pipes are never rejected).WH_PIPES_DIRseed files) logs a warning per affected pipe at load.Notes