Transparent 429 retry with Retry-After + exponential backoff#75
Conversation
b8e1d06 to
be7921d
Compare
c532b8f to
350de33
Compare
| ) | ||
| session = requests.Session() | ||
| adapter = HTTPAdapter(max_retries=retry) | ||
| session.mount("http://", adapter) |
There was a problem hiding this comment.
Any point in using adapter for http:// schema? All trafic is https for the Wallbox api?
There was a problem hiding this comment.
Fixed: dropped the http:// mount from the production session. The lib only ever talks to api.wall-box.com / user-api.wall-box.com (both https), so retry-arming a scheme we never use in production was dead code — agreed.
The pytest-httpserver test infrastructure was the only consumer of the http:// mount, since the test server runs plaintext localhost. To preserve full behavioural test coverage I considered two approaches:
- Switch tests to https with a self-signed cert (via pytest-httpserver's SSL support +
trustmeorverify=False) - Mirror the retry adapter onto http:// inside the test session only, leaving production https-only
Went with (2) — two lines in the test fixture (and one in test_connection_error_is_not_retried, which builds its Wallbox manually):
w._session.mount("http://", w._session.get_adapter("https://example.com/"))Reasoning: (1) requires generating self-signed certs and either disabling cert verification or trusting the cert chain — adds test infrastructure and a new dev dependency to solve what amounts to a localhost test setup detail. (2) keeps the test setup self-contained, doesn't add deps, and the small mirror line documents itself with a comment explaining why pytest-httpserver requires it.
Happy to switch to (1) if you'd rather not have any test-side mount mirroring.
| self._session = self._build_session(maxRetries, backoffFactor) | ||
|
|
||
| @staticmethod | ||
| def _build_session(maxRetries, backoffFactor): |
There was a problem hiding this comment.
Can we use Camel Case for _build_session (_buildSession)
There was a problem hiding this comment.
Renamed: _buildSession. Snake_case slipped through — same pattern as usedRefreshToken on #74. Consistent with the rest of the codebase now.
350de33 to
a68a0ce
Compare
The Wallbox API enforces rate limits on third-party clients (per the maintainer's note on cliviu74#68: 3 GET / 5 non-GET per minute). When the limit is hit the server returns 429 with a Retry-After header. Today the library raises HTTPError on the first 429, which knocks long-running consumers (e.g. the Home Assistant integration) offline until the integration is manually reloaded. This change introduces a single requests.Session, mounted with an HTTPAdapter that uses urllib3.Retry to handle 429 responses. The Retry-After header is honored when present; otherwise exponential backoff is applied. raise_on_status is disabled so callers continue to receive requests.HTTPError once retries are exhausted, preserving the existing exception contract — no behaviour change for callers, only added resilience. Scoping decisions: * 429 only. Transient 5xx is deliberately not in the forcelist because two POST endpoints (restartCharger, updateFirmware) are not idempotent — a transient 5xx after the device has already accepted the action could otherwise trigger a duplicate. 429 is always safe to retry: the request was not processed by definition. * Auth failures (401/403) are excluded so bad credentials surface immediately rather than burning the retry budget. * Network errors (connect=0, read=0) are not retried either. The retry budget is for 429 only — silently retrying network failures would be a behaviour change beyond this PR's stated scope and would mask real configuration errors behind extra wait time. * The retry adapter is mounted on https:// only, since both baseUrl and authUrl point at the Wallbox API which is https only. The test suite mirrors the adapter onto http:// in the test session, because pytest-httpserver runs plaintext localhost. Defaults: maxRetries=3, backoffFactor=1.0. Both are configurable via new constructor kwargs (named in camelCase to match the existing style of requestGetTimeout and jwtTokenDrift). The internal builder helper follows the same convention (_buildSession). Behaviour and observability are documented in README so callers know exactly what they will see for each failure mode. The library itself does not emit log records; callers who want retry visibility can configure the urllib3.connectionpool logger at WARNING level (urllib3 already logs each retry there). A test suite is added under tests/ in three layers: * TestRetryConfiguration — adapter and Retry config (9 tests) * TestExistingErrorContract — non-retryable HTTPError contract is preserved (3 tests, requests-mock) * TestRetryBehaviour — end-to-end against a localhost HTTP server so the full urllib3 Retry path is actually exercised (5 tests, pytest-httpserver) Each behavioural test was verified by mutating the relevant Retry config to break it and confirming the test caught the regression before being committed. The connect/read disabling specifically followed strict red-green TDD: the test failed naturally without the fix, then passed after it was applied. A urllib3>=1.26.0 floor is added to install_requires because this change uses the allowed_methods kwarg (renamed from method_whitelist in urllib3 1.26.0). Without the floor, a user installing on the existing requests>=2.22.0 lower bound could land on urllib3 1.25.x and hit a TypeError at construction time. Refs: cliviu74#68 ("Wallbox API has changed"), README "WARNING - Ratelimits"
a68a0ce to
5908a60
Compare
Summary
Adds transparent retry on 429 responses, addressing the rate-limit problem flagged in #68 and the README warning. Long-running consumers (notably the Home Assistant integration) currently lose connectivity on the first 429 and stay offline until manually reloaded — this PR makes the library survive normal rate-limit weather without any caller-side changes.
How
A single
requests.Sessionis mounted onWallbox.__init__with anHTTPAdapterwhosemax_retriesis aurllib3.Retryconfigured for:status_forcelist=(429,)— see scoping note belowallowed_methods=GET, POST, PUT, DELETE— see note belowconnect=0, read=0— network errors are not retried (see scoping note)respect_retry_after_header=True— honors the server'sRetry-Afterbackoff_factor=1.0— exponential fallback whenRetry-Afteris absent (~1s, 2s, 4s)raise_on_status=False— preserves the existingrequests.HTTPErrorcontract once retries are exhaustedEvery
requests.get/post/putcall site inwallbox.pyis mechanically swapped forself._session.get/post/put. No method signatures change.Note on
allowed_methodsincluding DELETEThe library does not currently issue any DELETE requests, but DELETE is kept in
allowed_methodsfor two reasons:allowed_methodsincludes DELETE for exactly this reason._sessionis the canonical retry-enabled HTTP session for the wallbox API. Enumerating it by HTTP-spec property (idempotent verbs + the POST we explicitly opted into) reads more cleanly than enumerating by current-usage. If someone adds a DELETE-using method later, retries fire automatically rather than silently regressing.This is a small judgment call — happy to tighten to
("GET", "POST", "PUT")if you prefer the whitelist to track actual usage strictly. Either is defensible.Scoping decisions
429 only — 5xx is intentionally not in the forcelist. Two POST endpoints in this lib are not idempotent:
restartCharger(action 3) andupdateFirmware(action 5). Retrying these on a transient 5xx after the device has already accepted the request could fire the action twice, withupdateFirmwarecarrying the same risk the README already warns about ("can brick your charger"). 429 is always safe to retry — by definition the server has not processed the request.Auth failures (401/403) are excluded so bad credentials surface immediately rather than burning the retry budget.
Network errors are also excluded (
connect=0, read=0). urllib3.Retry would otherwise retry connection errors, DNS failures, and read timeouts by default. Silently retrying these is a behaviour change beyond this PR's stated scope and would mask real configuration errors behind extra wait time. The retry budget is reserved for 429 only.Backwards compatibility and behaviour
The lib's failure semantics, post-PR:
requests.HTTPError(status 429)requests.HTTPErrorimmediatelyrequests.HTTPErrorimmediatelyrequests.HTTPErrorimmediatelyrequests.ConnectionError/TimeoutimmediatelyTwo new constructor kwargs make the budget tunable for callers who want different behaviour (named in camelCase to match
requestGetTimeout/jwtTokenDrift):Observability
The lib itself does not emit log records — preserving parity with the rest of the codebase. urllib3 logs each retry at
WARNINGlevel on theurllib3.connectionpoollogger, so callers who want visibility can configure that logger:In Home Assistant this surfaces in the integration's debug log when the user enables debug logging. Adding lib-native logging is plausible as a follow-up if you'd prefer that, but it felt out of scope for this PR.
Dependency floor
A
urllib3>=1.26.0floor is added tosetup.py install_requires. TheRetry(allowed_methods=...)kwarg this PR uses was renamed frommethod_whitelistin urllib3 1.26.0; without the floor, an end-user installing on the existingrequests>=2.22.0lower bound could land onurllib3 1.25.xand hit aTypeErrorat construction time. Anyone with a modernrequestsalready satisfies>=1.26.0.Tests
The repo had no test scaffolding, so this PR adds one in three layers:
TestRetryConfigurationTestExistingErrorContractHTTPErrorcontract preservedrequests-mockTestRetryBehaviourpytest-httpserverThe behavioural layer matters because
requests-mockswaps in its own adapter and bypasses the urllib3 Retry machinery.pytest-httpserverspins up a real localhost server so requests travel through the full urllib3 Retry path and the retry behaviour is actually exercised. The behavioural layer covers:429 → 200triggers exactly one retry and returns the decoded payload429 foreverexhausts the retry budget and raisesrequests.HTTPError(notRetryError)Retry-After: 1header is honored (timing-bounded assertion)401is not retried (single request, immediateHTTPError)Each behavioural test was verified before commit by mutating the relevant Retry config (e.g. dropping 429 from the forcelist, flipping
raise_on_status, disablingrespect_retry_after_header, adding 401 to the forcelist, or removingconnect=0/read=0) to confirm the test fails for the expected reason. Theconnect=0, read=0change specifically followed strict red-green TDD: the test failed naturally before the fix was applied, then passed after.requirements-dev.txtis added withpytest,requests-mock, andpytest-httpserveron top of the runtime requirements. Test code uses Python's standard snake_case convention for test method names and local variables (pytest discovery norm); production code is uniformly camelCase.Tested locally
Smoke-tested against the live Wallbox API (Home Assistant context) — normal calls behave exactly as before, no regressions visible. I do not have a reliable way to force-trigger a 429 from the live API on demand, so the retry path itself is exercised through the test suite rather than against production.
Out of scope (happy to file separately)
getChargerStatusto the v4 endpoint that@jorisdrenthdocumented in Wallbox API has changed #68 — different concern, larger surfaceRefs: #68