Repository navigation
feat: Trino and Amazon Athena warehouse connectors - #427
Merged
Merged
Conversation
One Trino SQL layer (trino_sql / trino_expr / trino_queries) behind two drivers: - TrinoAdapter (Trino and Starburst, `trino` client): https with basic auth, or an unauthenticated coordinator; the database field is the catalog, plus a default schema and schema allowlist. Session time zone UTC and query_max_run_time from the source's timeout. Every request, result pages included, goes to the vetted address with the configured name kept for TLS; other hosts and redirects are refused; a password is only sent over HTTPS. - AthenaAdapter (pyathena over boto3) subclasses it: region-only host (boto3 only ever reaches athena.<region>.amazonaws.com), workgroup, S3 result location, catalog, access key. The deadline is enforced client side with StopQueryExecution. SQL: date_trunc for 1h/1d and integer epoch-second arithmetic for 15m/6h/weeks anchored at WEEK_ORIGIN, on the UTC wall clock (zoned columns AT TIME ZONE 'UTC'); buckets are zone-less timestamp(3). Window literals follow the column family. json columns via json_extract / json_extract_scalar and map_keys(CAST(... AS map(varchar, json))); varchar columns can be parsed as JSON. count_if conditional aggregates, try_cast to double for range contracts, regex patterns probed per expectation with regexp_like (Joni). GROUP BY by ordinal (Trino refuses output aliases). Identifiers double-quoted; values written as single-quoted literals with the quote doubled - Trino literals have no escapes, the client would render bound parameters into the same literal, and Athena's execution parameters take literal text. db_type gains trino and athena (migration), typed connection settings, measure-validator dialect rules, fact-condition LIKE escaping, the data-source form, starter templates, docs and an e2e spec. A `trino` CI job runs the value suite against a digest-pinned trinodb/trino 483 coordinator; the Athena value suite runs on release tags once ATHENA_VALUE_CONFORMANCE_ENABLED is set and has not been run live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stops it The cursor has no query id until StartQueryExecution returns; a deadline that fired first was a silent no-op and the query ran on to the workgroup's limit. The deadline now keeps looking until the id appears. A BaseException during execute (a worker soft time limit) also stops the timer and closes the cursor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two new warehouse types,
trinoandathena.trinoclient:timestamp(3)on the UTC wall clock:1h/1dusedate_trunc;15m,6hand weeks floor onto the epoch grid (weeks anchored at WEEK_ORIGIN, a Monday).GROUP BYuses ordinals.(?P<name>…).jsoncolumns are documents.varcharcolumns can be parsed as JSON.pyathena:athena.<region>.amazonaws.com(.cn).StopQueryExecution).httpis refused.'doubled. Trino has no backslash escapes, and the client renders bound parameters the same way.e6a8c0d2f4b6.schemas/trino_settings.pyandconnection_settings_base.py. The frontend has connection fields, SQL dialect, formatter and metric templates for both engines.trinojob runs the value-conformance suite against digest-pinnedtrinodb/trino:483, with a JUnit check that nothing was skipped.athena-value-conformance.ymlruns on release tags onceATHENA_VALUE_CONFORMANCE_ENABLEDis set.Test plan
trino-athena-data-source.spec.ts, in CI.pyathena, and the docs mark Athena as believed, not proven.🤖 Generated with Claude Code