Skip to content

Rebase polynya patches onto v0.11.0; SigV4 as an AuthManager - #7

Merged
hasyimibhar merged 4 commits into
polynya-patches-v0.11.0from
rebase-v0.11.0
Nov 16, 2026
Merged

hasyimibhar merged 4 commits into
polynya-patches-v0.11.0from
rebase-v0.11.0

Conversation

@hasyimibhar

Copy link
Copy Markdown

The fork's patches from polynya-patches-v0.10.1, replayed onto upstream v0.11.0. polynya-patches-v0.11.0 is upstream's v0.11.0 commit (20eb0fe), so this PR's diff is exactly the fork's changes. polynya-patches-v0.10.1 is untouched.

Upstream since 0.10.1 brings 182 commits. Among those relevant to pg2iceberg:

Patches

Commit Change on 0.11.0
rebase polynya patches onto v0.11.0 (#3) Public TransactionAction, ReplaceSchemaAction, equality/position deletes through FastAppendAction, RewriteFilesAction, and Table::properties(). Upstream's new per-batch path dedupe in FastAppendAction (apache#2509) now runs over the data files and the delete files each. Manifests are read through Table::manifest_reader(), since apache#2884 removed ManifestFile::load_manifest. SigV4 is ported, see below.
one concurrent manifest scan per rewrite; Transaction::with_fresh_base (#4) Its scan reads through Table::manifest_reader().
stage a fast append, to write a chain of snapshots together (#5) The manifest list writer encrypts the list on a table with an encryption manager, as upstream's commit path does, and the snapshot names the list's key once it's written. A staged snapshot is planned before its list is written, so it refuses encrypted tables, as it already refuses v3 ones.
style: rustfmt and clippy the fork's patches The pinned nightly rustfmt over the fork's code, plus five needless borrows.

SigV4 is an AuthManager now

Upstream's REST client authenticates through AuthManager/AuthSession (apache#2838), mirroring Iceberg Java. The fork's signer was a hook in HttpClient::execute; it is now SigV4AuthManager, as Java's is.

  • Selection: rest.auth.type=sigv4, or the older rest.sigv4-enabled=true when no rest.auth.type is set, as Java does. Without that inference a pg2iceberg config would have silently sent unsigned requests.
  • Signing target: the region comes from rest.signing-region, falling back to s3.region and then us-east-1. The service comes from rest.signing-name, defaulting to execute-api. The catalog session also takes either one from the server's /v1/config.
  • Credentials: the AWS default chain, loaded on first use. with_credentials_provider overrides it.
  • Signing order: the session signs the request after the client has set every other header.

Tests

  • iceberg lib: 1688 passed. iceberg-catalog-rest lib: 85 passed.
  • New:
    • auth::sigv4 tests: signs for the configured region and service, takes them from the server config, falls back to s3.region then us-east-1, and the legacy property enables it.
    • test_auth_type_sigv4: explicit type or legacy property; an explicit type wins over the legacy property.
  • cargo fmt --check and cargo clippy --all-targets --all-features -D warnings for both crates are clean.
  • pg2iceberg: workspace tests, DST and integration suite on this branch: see its PR.

🤖 Generated with Claude Code

hasyimibhar and others added 4 commits November 17, 2026 02:13
The fork's patches from polynya-patches-v0.10.1 (#3), replayed onto
upstream v0.11.0:

- Public `TransactionAction`, `ReplaceSchemaAction`, equality/position
  deletes through `FastAppendAction` (its new per-batch path dedupe now
  runs over the data and the delete files each), `RewriteFilesAction`
  with its data sequence number, removed-file check and summary totals,
  and `Table::properties()` for vended credentials. Manifests are read
  through `Table::manifest_reader()` (upstream apache#2884 removed
  `ManifestFile::load_manifest`).
- SigV4 request signing is now an `AuthManager`, as upstream's REST
  client authenticates through `AuthManager`/`AuthSession` (apache#2838):
  `SigV4AuthManager`, selected by `rest.auth.type=sigv4` or, as Iceberg
  Java does, by the older `rest.sigv4-enabled=true`; region and service
  from `rest.signing-region` (falling back to `s3.region`) and
  `rest.signing-name`, which the server's config may also name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replayed onto v0.11.0. The manifest list writer now encrypts the list on
a table with an encryption manager, as upstream's commit path does, and
the snapshot names the list's key once it's written. A staged snapshot
is planned before its list is written, so it refuses encrypted tables,
as it refuses v3 ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hasyimibhar
hasyimibhar merged commit 429f3a0 into polynya-patches-v0.11.0 Nov 16, 2026
hasyimibhar added a commit to polynya-dev/pg2iceberg that referenced this pull request Nov 16, 2026
polynya-dev/iceberg-rust#7 merged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant