The lakehouse is one protocol away: Trino, Databricks and Snowflake as query engines - #239
Merged
Conversation
This was referenced Sep 3, 2026
…s query engines Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
WaylandYang
force-pushed
the
feat/lakehouse-engines
branch
from
September 3, 2026 02:56
fca6e7b to
a0a9949
Compare
# Conflicts: # docs/decisions/README.md
This was referenced Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three HTTP-protocol query engines beside Postgres, each behind the same
QueryEnginetrait:trino://, alsopresto://) —POST /v1/statement+nextUripaging. One catalog away from Iceberg, Delta Lake, Hive and Hudi, and Starburst speaks the same protocol.databricks://) — SQL Statement Execution API with INLINE / JSON_ARRAY results and PENDING → SUCCEEDED polling; the path is the console's httpPath.snowflake://) — SQL API v2 with 202 → handle polling; programmatic access token or OAuth (key-pair JWT deliberately not yet).The scheme picks the engine (
engine_from_conn), so the data-source page keeps its single connection-string field; the shape is validated at registration with the exact form in the error. The SQL gate parses with the engine's own dialect (guard_sql_for), LIMIT wrapping and JSON Lines are shared, and string-typed values from Databricks / Snowflake are restored to numbers and booleans from the column type. The engine client has an explicit proxy policy (loopback andNO_PROXYdirect,HTTPS_PROXY/HTTP_PROXYotherwise) because reqwest's Windows system-proxy detection routed loopback through the proxy. Migration0021widens theengineCHECK.Tests: 17 new unit tests (connection-string parsing, dialect gates, JSON assembly, type coercion) and wiremock replays of each protocol including their error bodies. End to end against a stand-in Trino on the isolated deployment: register → test → grant → mount (schema document says
Engine: trino) → chat runsquery_datathrough the Trino path and answers with the lake's numbers. None of the three has run against a real cluster; follow-up issues ask for that.Decision record 0018 (English, per the new convention) explains why Trino rather than format readers, why MaxCompute waits, and what is missing. README's Ontology-driven querying row and roadmap updated (English only, by request).
🤖 Generated with Claude Code