Skip to content

Support authentication for REST catalogs (bergr rest --auth) #38

Closed

Description

@mdub

bergr rest can't authenticate to REST catalogs, other than AWS Glue's (signed with SigV4, detected from the URI, since #37). This proposes a --auth TYPE option, where the type determines which environment variables bergr reads its settings from. Secrets only ever come from the environment, never from command-line arguments.

Proposal

bergr rest --auth bearer https://catalog.example.com namespaces
--auth Reads
auto (default) Behaves as sigv4 for AWS Glue URIs (glue.<region>.amazonaws.com), otherwise none
none nothing
sigv4 the standard AWS credential variables and profile; region and signing name from the hostname
bearer BERGR_REST_BEARER_TOKEN
basic BERGR_REST_BASIC_USER, BERGR_REST_BASIC_PASSWORD
oauth2 BERGR_REST_OAUTH2_CREDENTIAL (client_id:secret); optionally BERGR_REST_OAUTH2_SCOPE and BERGR_REST_OAUTH2_SERVER_URI
  • Type-specific variables are named BERGR_REST_<TYPE>_<NAME>, so it's clear which type each applies to, and settings left over from another type are ignored.
  • --auth itself can also be set with BERGR_REST_AUTH, so a shell configured with direnv (or similar) needs no flags.
  • A missing required variable is an error before any request is made, e.g. --auth bearer requires BERGR_REST_BEARER_TOKEN.
  • bergr rest --help lists the variables each type reads.

Why environment variables

Command-line arguments end up in shell history, and other local users can see them in ps while bergr runs. A process's environment is better protected: on Linux and macOS, only the same user and root can read it. Taking all settings for a type from the environment means there's no secret-bearing syntax to parse or escape, and no way to pass a secret as an argument by mistake. It's the same model as the AWS SDKs (AWS_PROFILE, AWS_ACCESS_KEY_ID).

The environment isn't the safest option, though: security guidance generally prefers a file only the user can read (OWASP Secrets Management Cheat Sheet). #39 proposes adding that.

Settings that aren't secret, such as a SigV4 region or signing-name override, can become flags later if needed.

Implementation notes

  • bearer and oauth2 map onto iceberg-catalog-rest's existing properties (token; credential, scope, oauth2-server-uri), so bergr only needs to pass them through. --help should mention the correspondence, for people who know those property names.
  • basic, like sigv4, would be a small custom AuthSession, setting Authorization: Basic ….
  • If upstream iceberg-rust ships rest.auth.type=sigv4 (apache/iceberg-rust#2660), bergr's sigv4 type could pass through to it as well.

Out of scope, for now

  • Arbitrary request headers (iceberg-catalog-rest's header.* properties). Header names don't map cleanly onto environment variable names; if a catalog needs a custom auth header pair (e.g. Cloudflare Access's CF-Access-Client-Id/-Secret), a dedicated type is clearer.
  • Reading settings from files. That's proposed separately, in Named catalog profiles, read from files (bergr profile NAME …) #39.
  • Vended credentials (X-Iceberg-Access-Delegation). That controls data access rather than authentication, and would be its own flag.
  • SigV4 for endpoints auto doesn't recognise (S3 Tables, FIPS and China endpoints, API Gateway with IAM auth). These need a signing-name and region override, as above.

Edited 2026-11-11: dropped a comparison with psql's PGPASSWORD, which PostgreSQL's own docs advise against, and moved reading settings from files to #39.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions