Skip to content

feat: authenticate to REST catalogs with --auth - #40

Merged
mdub merged 2 commits into
mainfrom
mdub-20261111-rest-auth
Nov 11, 2026
Merged

mdub merged 2 commits into
mainfrom
mdub-20261111-rest-auth

Conversation

@mdub

@mdub mdub commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

bergr rest can now authenticate to any REST catalog, not just AWS Glue's, using --auth TYPE (or BERGR_REST_AUTH), as proposed in #38. Each type reads its settings, secrets included, from environment variables, never from arguments. Closes #38.

BERGR_REST_BEARER_TOKEN="$(cat token-file)" bergr rest --auth bearer https://catalog.example.com namespaces
--auth Reads
auto (default) Behaves as sigv4 for AWS Glue endpoints, otherwise none
none nothing
sigv4 the standard AWS credential variables and profile
bearer BERGR_REST_BEARER_TOKEN
basic BERGR_REST_BASIC_USER, BERGR_REST_BASIC_PASSWORD
oauth2 BERGR_REST_OAUTH2_CREDENTIAL (client_id:secret); optionally BERGR_REST_OAUTH2_SCOPE and BERGR_REST_OAUTH2_SERVER_URI

How it works:

  • bearer and oauth2 pass through to iceberg-catalog-rest's own OAuth2 support (token; credential, scope, oauth2-server-uri). bergr sets rest.auth.type explicitly, which also avoids the library's "inferring oauth2" warning.
  • basic is a small custom AuthManager (src/basic_auth.rs), like the existing SigV4 one.
  • src/rest_auth.rs resolves the type and its settings into either SigV4 (which needs AWS config) or plain HTTP auth. It reads settings through a lookup function rather than std::env::var, so tests don't modify the process environment, and Named catalog profiles, read from files (bergr profile NAME …) #39's profiles can plug in later.
  • Secrets are held as iceberg's SensitiveString, so they're redacted in --debug output.

Choices the issue didn't spell out:

  • An empty variable counts as unset, so $(cat missing-file) fails with a clear error instead of sending an empty token.
  • BERGR_REST_BASIC_USER may not contain :, which RFC 7617 forbids.
  • --auth sigv4 with a non-Glue URI is an error, until signing name and region overrides exist.
  • --auth none turns off signing even for a Glue URI.

Testing:

  • New mockito tests check the Authorization header each type sends on /v1/config and /v1/namespaces, including the OAuth2 client-credentials exchange.
  • Against the Apache Polaris quickstart, oauth2 and bearer both list namespaces and tables, and an explicit BERGR_REST_OAUTH2_SERVER_URI is honoured. A wrong secret, a bad token, and no auth are each rejected with 401. No secret or token appears in --debug output, even on failure. Polaris needs BERGR_REST_OAUTH2_SCOPE=PRINCIPAL_ROLE:ALL, so the README now shows that.
  • basic was only tested against a local stub server, since Polaris doesn't support it.

Polaris testing also showed that bergr rest … table … fails for every catalog except Glue's, because the plain REST catalog has no storage factory. That predates this PR, and is #41.

`bergr rest` could only authenticate to AWS Glue's REST endpoint, via
SigV4. It now takes `--auth TYPE` (or `BERGR_REST_AUTH`), where the type
decides which environment variables bergr reads its settings from:

- `auto` (default): `sigv4` for Glue endpoints, otherwise `none`
- `none`
- `sigv4`: AWS credentials, as before
- `bearer`: BERGR_REST_BEARER_TOKEN
- `basic`: BERGR_REST_BASIC_USER, BERGR_REST_BASIC_PASSWORD
- `oauth2`: BERGR_REST_OAUTH2_CREDENTIAL, and optionally
  BERGR_REST_OAUTH2_SCOPE and BERGR_REST_OAUTH2_SERVER_URI

Secrets only ever come from the environment, never from arguments,
which are visible in `ps` and end up in shell history. A missing
variable is an error before any request is made.

`bearer` and `oauth2` pass through to iceberg-catalog-rest's OAuth2
support (`token`; `credential`, `scope`, `oauth2-server-uri`), with
`rest.auth.type` set explicitly. `basic` is a small custom
`AuthManager`, like the SigV4 one.

Settings are resolved through a lookup function rather than read from
the environment directly, so that tests needn't modify the process
environment, and so that #39's profiles can supply settings too.

Closes #38
Polaris rejects the default `catalog` scope, and needs
`PRINCIPAL_ROLE:ALL`. Found while testing --auth against the Polaris
quickstart.
@mdub
mdub marked this pull request as ready for review November 11, 2026 02:14
@mdub
mdub merged commit fffc145 into main Nov 11, 2026
4 checks passed
@mdub
mdub deleted the mdub-20261111-rest-auth branch November 11, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support authentication for REST catalogs (bergr rest --auth)

1 participant