Skip to content

Read Glue REST tables with Glue's vended credentials - #47

Merged
mdub merged 1 commit into
mainfrom
mdub-20261111-glue-vended-credentials
Nov 15, 2026
Merged

mdub merged 1 commit into
mainfrom
mdub-20261111-glue-vended-credentials

Conversation

@mdub

@mdub mdub commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

bergr rest against AWS Glue's Iceberg REST endpoint now reads table files with the short-lived S3 credentials that Glue vends for each table, instead of your own AWS credentials. So tables you can read through Lake Formation, but not directly in S3, now work. Fixes #44.

Why it didn't work

Glue's loadTable response always includes s3.access-key-id, s3.secret-access-key and s3.session-token in the table's config, whether or not the request asks for them. bergr ignored these keys. Its Glue path gives opendal's S3 storage a custom credential loader, and opendal's credential_provider_chain replaces the whole credential chain with that loader, keys in the table's config included.

Changes

  • VendedFirstS3StorageFactory (src/aws.rs) wraps OpenDalStorageFactory::S3, and the Glue REST catalog now uses it. It decides per table: when the table's config has vended keys, it builds the storage without the custom loader, so opendal uses them; otherwise it uses bergr's own credential chain as before. StorageFactory is a typetag trait, so the wrapper derives serde, skipping the process-local loader, and typetag becomes a direct dependency. It was already in the lockfile.
  • files --verify lists data files with an S3 client built from the vended keys (vended_s3_client), when there are any. The client reads region, endpoint and path-style access from the table's properties, as opendal does.
  • create_existence_checker now falls back to per-file checks when the bulk ListObjectsV2 fails, and logs a warning. Listing is only an optimisation, and vended keys can't always list: Polaris's quickstart keys can read objects but get AccessDenied on ListObjectsV2. Without the fallback, the change above would have broken --verify against Polaris.
  • README: replaced "AWS Glue's table files are always read with your AWS credentials".

The Glue path still doesn't send X-Iceberg-Access-Delegation, since Glue vends either way. Plain bergr glue (the Glue API) is unchanged.

Testing

  • New unit tests cover the storage loader choice, the vended credentials and region, and the listing fallback, using mockito as a fake S3 that returns 403. cargo fmt, cargo clippy --all-targets -- -D warnings and cargo test (73 tests) pass.
  • Against a real Glue REST catalog, using a temporary patch that gave bergr's storage fallback and its listing client bogus AWS keys while catalog requests stayed signed with real credentials:
    • main: files and files --verify fail with InvalidAccessKeyId.
    • This branch: both succeed (516 files, none missing), and --verify still uses the bulk listing, so the vended keys allow ListObjectsV2.
  • Without the patch, Glue files and --verify work as before.
  • Polaris: files --verify falls back to per-file checks after the denied listing, and finds every file.

Not tested: a principal that can only reach the data through Lake Formation. The bogus-key test exercises the same path.

AWS Glue's Iceberg REST endpoint vends short-lived S3 credentials with every
table, through Lake Formation, but bergr read Glue tables' files with the
user's own AWS credentials: its custom credential loader replaces opendal's
whole credential chain, including keys in the table's config. So tables
granted only through Lake Formation couldn't be read.

- Give the Glue REST catalog a storage factory that leaves out the custom
  loader when a table's config has vended keys, and keeps bergr's own
  credential chain as the fallback.
- List files for `files --verify` with an S3 client built from the vended
  keys, when there are any.
- Fall back to per-file existence checks when listing fails: vended keys
  can't always list (Polaris's can't), and listing is only an optimisation.

Fixes #44
@mdub
mdub marked this pull request as ready for review November 15, 2026 22:10
@mdub
mdub merged commit 854274f into main Nov 15, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use AWS Glue's vended credentials for bergr rest against Glue

1 participant