Repository navigation
Rebase polynya patches onto v0.11.0; SigV4 as an AuthManager - #7
Merged
Merged
Conversation
The fork's patches from polynya-patches-v0.10.1 (#3), replayed onto upstream v0.11.0: - Public `TransactionAction`, `ReplaceSchemaAction`, equality/position deletes through `FastAppendAction` (its new per-batch path dedupe now runs over the data and the delete files each), `RewriteFilesAction` with its data sequence number, removed-file check and summary totals, and `Table::properties()` for vended credentials. Manifests are read through `Table::manifest_reader()` (upstream apache#2884 removed `ManifestFile::load_manifest`). - SigV4 request signing is now an `AuthManager`, as upstream's REST client authenticates through `AuthManager`/`AuthSession` (apache#2838): `SigV4AuthManager`, selected by `rest.auth.type=sigv4` or, as Iceberg Java does, by the older `rest.sigv4-enabled=true`; region and service from `rest.signing-region` (falling back to `s3.region`) and `rest.signing-name`, which the server's config may also name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replayed onto v0.11.0. The manifest list writer now encrypts the list on a table with an encryption manager, as upstream's commit path does, and the snapshot names the list's key once it's written. A staged snapshot is planned before its list is written, so it refuses encrypted tables, as it refuses v3 ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hasyimibhar
added a commit
to polynya-dev/pg2iceberg
that referenced
this pull request
Nov 16, 2026
polynya-dev/iceberg-rust#7 merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The fork's patches from
polynya-patches-v0.10.1, replayed onto upstream v0.11.0.polynya-patches-v0.11.0is upstream's v0.11.0 commit (20eb0fe), so this PR's diff is exactly the fork's changes.polynya-patches-v0.10.1is untouched.Upstream since 0.10.1 brings 182 commits. Among those relevant to pg2iceberg:
first_row_idinheritance on manifest read (feat: inherit data-file first_row_id on manifest read apache/iceberg-rust#2922). This is the first commit of v3 row lineage: REST create sends format-version, RewriteFilesAction::validate_no_row_ids #6, which then drops it._row_idreads (feat: synthesize _row_id metadata column on read apache/iceberg-rust#3058) and deletion-vector reads (feat(scan): [3/N] read and apply V3 deletion vectors apache/iceberg-rust#3035), which pg2iceberg's later v3 phases build on.Varianttype, and a reworked REST client.Patches
TransactionAction,ReplaceSchemaAction, equality/position deletes throughFastAppendAction,RewriteFilesAction, andTable::properties(). Upstream's new per-batch path dedupe inFastAppendAction(apache#2509) now runs over the data files and the delete files each. Manifests are read throughTable::manifest_reader(), since apache#2884 removedManifestFile::load_manifest. SigV4 is ported, see below.Transaction::with_fresh_base(#4)Table::manifest_reader().SigV4 is an
AuthManagernowUpstream's REST client authenticates through
AuthManager/AuthSession(apache#2838), mirroring Iceberg Java. The fork's signer was a hook inHttpClient::execute; it is nowSigV4AuthManager, as Java's is.rest.auth.type=sigv4, or the olderrest.sigv4-enabled=truewhen norest.auth.typeis set, as Java does. Without that inference a pg2iceberg config would have silently sent unsigned requests.rest.signing-region, falling back tos3.regionand thenus-east-1. The service comes fromrest.signing-name, defaulting toexecute-api. The catalog session also takes either one from the server's/v1/config.with_credentials_provideroverrides it.Tests
iceberglib: 1688 passed.iceberg-catalog-restlib: 85 passed.auth::sigv4tests: signs for the configured region and service, takes them from the server config, falls back tos3.regionthenus-east-1, and the legacy property enables it.test_auth_type_sigv4: explicit type or legacy property; an explicit type wins over the legacy property.cargo fmt --checkandcargo clippy --all-targets --all-features -D warningsfor both crates are clean.🤖 Generated with Claude Code