Repository navigation
feat: authenticate to REST catalogs with --auth - #40
Merged
Merged
Conversation
`bergr rest` could only authenticate to AWS Glue's REST endpoint, via SigV4. It now takes `--auth TYPE` (or `BERGR_REST_AUTH`), where the type decides which environment variables bergr reads its settings from: - `auto` (default): `sigv4` for Glue endpoints, otherwise `none` - `none` - `sigv4`: AWS credentials, as before - `bearer`: BERGR_REST_BEARER_TOKEN - `basic`: BERGR_REST_BASIC_USER, BERGR_REST_BASIC_PASSWORD - `oauth2`: BERGR_REST_OAUTH2_CREDENTIAL, and optionally BERGR_REST_OAUTH2_SCOPE and BERGR_REST_OAUTH2_SERVER_URI Secrets only ever come from the environment, never from arguments, which are visible in `ps` and end up in shell history. A missing variable is an error before any request is made. `bearer` and `oauth2` pass through to iceberg-catalog-rest's OAuth2 support (`token`; `credential`, `scope`, `oauth2-server-uri`), with `rest.auth.type` set explicitly. `basic` is a small custom `AuthManager`, like the SigV4 one. Settings are resolved through a lookup function rather than read from the environment directly, so that tests needn't modify the process environment, and so that #39's profiles can supply settings too. Closes #38
Polaris rejects the default `catalog` scope, and needs `PRINCIPAL_ROLE:ALL`. Found while testing --auth against the Polaris quickstart.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bergr restcan now authenticate to any REST catalog, not just AWS Glue's, using--auth TYPE(orBERGR_REST_AUTH), as proposed in #38. Each type reads its settings, secrets included, from environment variables, never from arguments. Closes #38.BERGR_REST_BEARER_TOKEN="$(cat token-file)" bergr rest --auth bearer https://catalog.example.com namespaces--authauto(default)sigv4for AWS Glue endpoints, otherwisenonenonesigv4bearerBERGR_REST_BEARER_TOKENbasicBERGR_REST_BASIC_USER,BERGR_REST_BASIC_PASSWORDoauth2BERGR_REST_OAUTH2_CREDENTIAL(client_id:secret); optionallyBERGR_REST_OAUTH2_SCOPEandBERGR_REST_OAUTH2_SERVER_URIHow it works:
bearerandoauth2pass through to iceberg-catalog-rest's own OAuth2 support (token;credential,scope,oauth2-server-uri). bergr setsrest.auth.typeexplicitly, which also avoids the library's "inferring oauth2" warning.basicis a small customAuthManager(src/basic_auth.rs), like the existing SigV4 one.src/rest_auth.rsresolves the type and its settings into either SigV4 (which needs AWS config) or plain HTTP auth. It reads settings through a lookup function rather thanstd::env::var, so tests don't modify the process environment, and Named catalog profiles, read from files (bergr profile NAME …) #39's profiles can plug in later.SensitiveString, so they're redacted in--debugoutput.Choices the issue didn't spell out:
$(cat missing-file)fails with a clear error instead of sending an empty token.BERGR_REST_BASIC_USERmay not contain:, which RFC 7617 forbids.--auth sigv4with a non-Glue URI is an error, until signing name and region overrides exist.--auth noneturns off signing even for a Glue URI.Testing:
mockitotests check theAuthorizationheader each type sends on/v1/configand/v1/namespaces, including the OAuth2 client-credentials exchange.oauth2andbearerboth list namespaces and tables, and an explicitBERGR_REST_OAUTH2_SERVER_URIis honoured. A wrong secret, a bad token, and no auth are each rejected with 401. No secret or token appears in--debugoutput, even on failure. Polaris needsBERGR_REST_OAUTH2_SCOPE=PRINCIPAL_ROLE:ALL, so the README now shows that.basicwas only tested against a local stub server, since Polaris doesn't support it.Polaris testing also showed that
bergr rest … table …fails for every catalog except Glue's, because the plain REST catalog has no storage factory. That predates this PR, and is #41.