Feat: add request.extensions["retry"]

[vgavro]

This feature allows to:

1. Override Retry configuration per request, the same way as Timeout configuration may be overriden per request, see https://www.python-httpx.org/advanced/extensions/

client.get("https://example.com", extensions={"retry": Retry(...)})

2. Allows introspection on each request of actual retries made and time spent , because response.request.extensions["retry"] is always set (the same way as response.request.extensions["timeout"] is always set, so it's consistent across httpx api expectations)

3. This is also pretty useful with feat: Add validate_response option #74 - considering you decide what request responses you want to pre-validate (for example, to read body in RetryTransport to retry timeouts), and what requests you will use with streaming.
   NOTE - these two merge requests are in conflict, some of them should be merged first, and then i'll adopt other IF you're willing to merge both hopefully.

NOTES ON SOME DESIGN DECISIONS:

1. Retry.copy_with was implemented for easier use to copy global retry configuration with per-request retry configuration - naming like httpx.URL.copy_with, but instead of **kwargs in signature we're using full typing with _UNSET sentinel -for better type checker autocompletions - same pattern used in httpx._config

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (8c7b9c9) to head (41e6288).

Additional details and impacted files

@@            Coverage Diff            @@
##              main       #75   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            6         6           
  Lines         1132      1220   +88     
=========================================
+ Hits          1132      1220   +88     

Files with missing lines	Coverage Δ
httpx_retries/retry.py	100.00% <100.00%> (ø)
httpx_retries/transport.py	100.00% <100.00%> (ø)
tests/test_retry.py	100.00% <100.00%> (ø)
tests/test_transport.py	100.00% <100.00%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[will-ockmore]

Thanks again @vgavro for this, appreciate the though that you've taken here. Splitting my feedback by the two halves:

copy_with(): This is great. It's the right fix for the field-list duplication in increment(). Two things I'd like:

1. add a test that round-trips every __init__ field through copy_with() so the two signatures can't silently drift
2. we'll need to thread validate_response through once #74/#75 ordering is sorted.

request.extensions["retry"]: I'd like to take this as a separate PR so we can get the semantics right. The blocker for me is that the transport writes the incremented retry back into request.extensions. Request objects can be reused, so a reused request comes back with a depleted budget. Can we

1. read the override in from request.extensions["retry"] but not mutate it, and
2. if we want post-hoc introspection of attempts_made, put the final retry on response.extensions instead?
   That keeps input config and output state on the right-lifetime objects.

Additionally, as it's squatting httpx's namespace here in the extensions dict, we should proceed with caution. It's a valid use case, but we may want to guard against a future version where this key is set externally.

[will-ockmore]

As we're using the sentinel pattern here, the more performant check is x is _UNSET instead of isinstance(x, _UnsetType)

[vgavro]

it's totally for type-checkers on using Sentinel, it's either - we'll do as typechekers wants us to do with _UnsetType = _UNSET pattern and using isinstance() (because, formally, you can pass OTHER _UnsetType, not only _UNSET sentinel), or adding mypy/pyright/ty ignores there.

[vgavro]

if you prefer typing ignores for performance - i'll fix, let me know.

[will-ockmore]

That's a fair argument. I haven't measured the performance impact, I doubt it is worth changing.

[vgavro]

copy_with(): This is great. It's the right fix for the field-list duplication in increment(). Two things I'd like:

1. add a test that round-trips every __init__ field through copy_with() so the two signatures can't silently drift
2. we'll need to thread validate_response through once feat: Add validate_response option #74/Feat: add request.extensions["retry"] #75 ordering is sorted.

Agreed.

request.extensions["retry"]: I'd like to take this as a separate PR so we can get the semantics right. The blocker for me is that the transport writes the incremented retry back into request.extensions. Request objects can be reused, so a reused request comes back with a depleted budget. Can we

1. read the override in from request.extensions["retry"] but not mutate it, and
2. if we want post-hoc introspection of attempts_made, put the final retry on response.extensions instead?
   That keeps input config and output state on the right-lifetime objects.

This is great point! I'll do.

Additionally, as it's squatting httpx's namespace here in the extensions dict, we should proceed with caution. It's a valid use case, but we may want to guard against a future version where this key is set externally.

Well, that's a valid point but tbh i don't see better approach here, having "httpx_retries" namespace instead of "retry" is worse from DX point of view, and i don't see cases where developer would want to use TWO retries libraries. I think it's just unavoidable risk. Use "retries" namespace instead of "retry" is just as good wild guess as keep "retry" and hope nobody else wants to use it. I think "retry" is fine.

[vgavro]

plz review, thank you for you time!

• fixed not mutating request.extensions["retry"] but setting response.extensions["retry"] instead.
• added Retry.copy_with signature related tests.

I didn't fixed, but answered:

• isinstance(_UnsetType) check instead of is _UNSET - if you prefer typing ignores for performance - i'll fix, let me know.
• namespace extensions["retry"] issue

[will-ockmore]

Once conflicts are sorted, I'll merge this one as well, thanks

[vgavro]

@will-ockmore merged, thanks!