Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4428
Rationale for this change
#4310 sends
Idempotency-Keyon mutations and letsrest.client.*retry keyedPOSTs, but it never looks at theidempotency-key-lifetimethe server returns from/v1/configbeyond checking it's there. urllib3 sleeps for the fullRetry-After(and backoff can add up too), so a retry can go out after the key has expired, and the server may then apply the same mutation twice. The spec says clients SHOULD NOT reuse a key after that window.This follows what the Java client does (
HTTPClient.parseKeyLifetime+ExponentialHttpRequestRetryStrategy, apache/iceberg#18070):idempotency-key-lifetimeis parsed as an ISO-8601 duration (TypeAdapter(timedelta))._RetryTimeoutHTTPAdapter.sendrecords a deadline (now + lifetime, monotonic clock) in aContextVarwhen the request carries anIdempotency-Key.Retrysubclass checks it inincrement: ifnow + next_sleep >= deadlinethe retry is declined.next_sleepis whatRetry.sleepwould wait:Retry-Afterwhen urllib3 would honor it, otherwise the backoff plus the max jitter.MaxRetryError, i.e. it behaves exactly like running out of retries: withraise_on_status=Falseurllib3 hands back the last response, so_handle_non_200_responsestill raises the typed error (e.g.ServiceUnavailableError); a connection error is re-raised as before.Requests without a key (all
GET/HEAD, and everything when the server doesn't advertise a lifetime) are untouched.Are these changes tested?
Yes, in
tests/catalog/test_rest.py, using the existing loopback server (now with an optionalRetry-Afterheader):Retry-After: 60withPT30Slifetime: no retry,ServiceUnavailableError, 1 callPT5S, factor 10): stops before the second retryRetry-After: 0withinPT30M: still retried and succeedsPT0Sand-PT5M: keyed POST never retried, key still sentPT0Sdoesn't affect unkeyed GETs, they still retry_parse_idempotency_key_lifetimeunit casesmake lintpasses.pytest tests/catalog -m unitpasses except 4 kerberos tests intest_hive.pythat fail the same way onmainhere (kerberos extra not installed locally).Are there any user-facing changes?
Only for catalogs whose server advertises
idempotency-key-lifetimeand that setrest.client.max-retries: a keyed mutation is no longer retried once the next attempt would land outside the key's lifetime; the error that would have been raised after exhausting retries is raised earlier instead.Limitations
table/updatecreates a new key per attempt, so that's fine).POSTon status codes, so it isn't changed here.#4310 cc @YuangGao. Happy to adjust if u'd prefer a different shape (e.g. doing this in the adapter instead of a
Retrysubclass). thx!