Repository navigation
feat: Support configuring Iceberg REST catalog #950
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
97c60b1
3c6dd88
36b1189
9b93545
d80e9e5
5af8d9e
8d8c750
2baa4f4
3373167
b3e3e54
a4e1d83
a84519d
96844a5
eec3c71
a65c8bd
c153ce8
1290db7
be0311a
324bbd4
c93b7f5
1b9f0a8
a4c7ead
bfec62a
9ec4755
96a27b0
1a93e62
1b7c94c
7e6a42e
c950aef
49a4a67
c10d014
e38564d
d5a2f28
c0e5113
9f86a8b
d9f11d5
e9aedb9
eedbc35
76f2f83
a08e9cb
7148bd1
a18857d
df77f11
312b0d1
8dff9ad
e653279
d030733
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,104 @@ | ||
| = Apache Iceberg | ||
|
|
||
| Apache Iceberg is a format for huge analytic tables designed to address some of the scaling issues with traditional Hive tables. | ||
| Iceberg depends on a Hive metastore being present and makes use of the same metastore ConfigMap used by the Hive connector. | ||
| Iceberg stores table metadata in a metadata catalog, which is configured in `spec.connector.iceberg.catalog`. | ||
| The following catalogs are supported: | ||
|
|
||
| You can deploy a Hive Metastore Stacklet with the xref:hive:index.adoc[]. | ||
| * `rest`: An https://trino.io/docs/current/object-storage/metastores.html#rest-catalog[Iceberg REST catalog] | ||
| * `hiveMetastore`: A Hive metastore, using the same metastore ConfigMap as the xref:usage-guide/catalogs/hive.adoc[] connector | ||
| * `userProvided`: The operator doesn't configure a catalog, you need to configure it yourself using `configOverrides` | ||
|
|
||
| == Example Iceberg REST catalog configuration | ||
|
|
||
| == Example Iceberg catalog configuration | ||
| [source,yaml] | ||
| ---- | ||
| apiVersion: trino.stackable.tech/v1alpha2 | ||
| kind: TrinoCatalog | ||
| metadata: | ||
| name: iceberg # <1> | ||
| labels: | ||
| trino: simple-trino | ||
| spec: | ||
| connector: | ||
| iceberg: | ||
| catalog: | ||
| rest: | ||
| hostname: iceberg-rest-catalog.example.com # <2> | ||
| port: 9001 # <3> | ||
| path: /catalog # <4> | ||
| tls: # <5> | ||
| verification: | ||
| server: | ||
| caCert: | ||
| secretClass: tls | ||
| authentication: # <6> | ||
| oAuth2: | ||
| server: # <7> | ||
| hostname: idp.example.com | ||
| path: /oauth2/token | ||
| clientCredentials: # <8> | ||
| secretName: trino-iceberg-client # <9> | ||
| scopes: # <10> | ||
| - lakehouse | ||
| s3: | ||
| inline: | ||
| host: test-minio | ||
| port: 9000 | ||
| accessStyle: Path | ||
| credentials: | ||
| secretClass: minio-credentials | ||
| --- | ||
| apiVersion: v1 | ||
| kind: Secret | ||
| metadata: | ||
| name: trino-iceberg-client | ||
| stringData: | ||
| clientId: trino | ||
| clientSecret: my-client-secret | ||
| ---- | ||
| <1> The name of the catalog as it will appear in Trino | ||
| <2> The hostname of the REST catalog server | ||
| <3> Optional, defaults to 443 if TLS is used and to 80 otherwise | ||
| <4> Optional, defaults to `/` | ||
| <5> Optional, uses the same TLS configuration as xref:usage-guide/s3.adoc#tls[S3 connections]. | ||
| If not specified, TLS is used and the server certificate is verified against the web PKI (`webPki`). | ||
| Set `tls: null` to connect without TLS. Disabling the TLS verification (`verification.none`) is not supported. | ||
| `tls: null` is kept when creating or replacing a TrinoCatalog, but `kubectl apply` and server-side apply remove it from an existing TrinoCatalog, which brings back the `webPki` default. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tests revealed that
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As it's a common config mechanism we use and more complicated than though, are you ok with removing the sentence regarding how to set it to null? |
||
| Use `kubectl replace` or recreate the TrinoCatalog to switch an existing catalog to a connection without TLS. | ||
| <6> Optional, defaults to no authentication | ||
| <7> The token endpoint of the OAuth2 server, see <<oauth2-server>>. | ||
| It takes the same `hostname`, `port`, `path` and `tls` fields as the REST catalog, so it uses TLS with `webPki` verification by default. | ||
| <8> The client credentials are exchanged for a token using the OAuth2 client credentials flow. | ||
| Alternatively, use `token.secretName` to pass a Secret containing a static bearer token in the `token` key. | ||
| <9> Name of a Secret containing the `clientId` and `clientSecret` keys | ||
| <10> Optional, the scopes to request from the OAuth2 server. | ||
| If not specified, the `catalog` scope is requested. Specifying scopes replaces this default. | ||
|
|
||
| The operator only supports a subset of the https://trino.io/docs/current/object-storage/metastores.html#rest-catalog[REST catalog properties]. | ||
| Use `configOverrides` to set additional ones. | ||
|
|
||
| [#oauth2-server] | ||
| === OAuth2 server | ||
|
|
||
| `oAuth2.server` points to the token endpoint of the OAuth2 server. | ||
| What it is used for depends on the credential: | ||
|
|
||
| * `clientCredentials`: Trino fetches its token from this endpoint on startup and refreshes it there before it expires. | ||
| * `token`: Trino uses the token as is. | ||
| If the token is a JWT with an `exp` claim, Trino tries to refresh it before it expires by exchanging it at this endpoint (OAuth2 token exchange). | ||
| If the refresh fails, Trino logs a warning and keeps using the current token. | ||
| A token that isn't a JWT is never refreshed, so the endpoint is never called. | ||
|
|
||
| The OAuth2 server is required even for tokens that are never refreshed. | ||
| Without it, the Iceberg client used by Trino would fall back to the deprecated `/v1/oauth/tokens` endpoint of the REST catalog, which is planned to be removed (see https://github.com/apache/iceberg/issues/10537[apache/iceberg#10537]). | ||
|
|
||
| == Example Iceberg Hive metastore configuration | ||
|
|
||
| You can deploy a Hive Metastore Stacklet with the xref:hive:index.adoc[]. | ||
|
|
||
| [source,yaml] | ||
| ---- | ||
| apiVersion: trino.stackable.tech/v1alpha1 | ||
| apiVersion: trino.stackable.tech/v1alpha2 | ||
| kind: TrinoCatalog | ||
| metadata: | ||
| name: iceberg # <1> | ||
|
|
@@ -19,8 +108,9 @@ spec: | |
| connector: | ||
| # Specify iceberg here when defining a iceberg catalog | ||
| iceberg: | ||
| metastore: | ||
| configMap: simple-hive # <2> | ||
| catalog: | ||
| hiveMetastore: | ||
| configMap: simple-hive # <2> | ||
| s3: | ||
| inline: | ||
| host: test-minio | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docs say webPki verifies against the web PKI, but in practice it also trusts every SecretClass CA used by any other connection of the TrinoCluster. All connections share one JVM truststore (config/jvm.rs:83, seeded with the system bundle in https://github.com/stackabletech/trino-operator/blob/main/rust/operator-binary/src/controller/build/command.rs#L54, extended per connection in config/tls.rs:30. See the FIXME in https://github.com/stackabletech/trino-operator/blob/main/rust/operator-binary/src/controller/build/command.rs#L52).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are absolutely correct! We have that problem at many places already: Lot's of products don't support configuring the ca cert for explicit (S3, LDAP, whatnot) connections. So we don't have any other choice as to add the certs to the JVM truststore.
Ideally we should mention this globally for every affected product, as this applies not only here, but also to a bunch of other places e.g. in Trino. Not sure how to best proceed here, as you are right, the docs are wrong and trino-specific. E.g. on S3Connection there might be some product that do it correctly, here it's only for Trino. The docs describe how it should work, but we currently(!) have the Trino limitations - future Trino versions might support it. The entire platform TlsClientVerification struct is build on this assumption, so from my perspective we have a platform-global problem, this is just continuing status quo.