Skip to content

REST: send Idempotency-Key on mutations and retry them on transient failures #3671

Open

Description

@YuangGao

Is your feature request related to a problem or challenge?

The REST spec lets a server advertise idempotency-key-lifetime in GET /v1/config. Clients then send an Idempotency-Key (UUIDv7) on mutations and reuse it when retrying, so a retried commit or create can't apply twice.

iceberg-rust does neither: CatalogConfig ignores the field, and HttpClient::query_catalog sends each request once, so a 5xx on a mutation reaches the caller (CommitStateUnknown for update_table) even when the server could have deduplicated a retry. Java (RESTSessionCatalog, ExponentialHttpRequestRetryStrategy) and PyIceberg (apache/iceberg-python#4310) already support it.

Describe the solution you'd like

  • Parse idempotency-key-lifetime into CatalogConfig. Without it, nothing changes.
  • When it's present, add a fresh Idempotency-Key (Uuid::now_v7()) to each mutation: namespace create/update/drop, table create/drop/purge/rename/register/update, and commit_transaction once feat(rest): support atomic multi-table commits #3398 lands. Reads carry none.
  • Retry keyed requests on 429/500/502/503/504 with exponential backoff, reusing the key, and stop once the lifetime has passed. backon and uuid's v7 feature are already workspace dependencies.

Open question: should these retries be configurable, like Java's rest.client.* properties, or only apply to keyed requests by default?

Willingness to contribute

I can contribute a PR for this.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions