Broker: thread-safe rate limiters + stale-key eviction#118
Conversation
RateLimiter and ByteRateLimiter mutated shared deques without a lock under ThreadingHTTPServer, letting concurrent requests exceed limits, and their key dicts grew without bound. Guard allow() with a per-limiter lock, opportunistically evict stale keys, and compare session tokens with hmac.compare_digest to avoid timing leaks. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2f484dd475
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for session_token in self.config.session_tokens: | ||
| # Check every configured token so the comparison time does not | ||
| # depend on where (or whether) the match occurs. | ||
| token_matches |= hmac.compare_digest(token, session_token) |
There was a problem hiding this comment.
When any static session token is configured, a malformed Authorization: Bearer ... value containing non-ASCII bytes now reaches hmac.compare_digest(token, session_token), which raises TypeError for non-ASCII str inputs. That turns an unauthenticated request to /v1/responses or /v1/audio/transcriptions into an unhandled handler exception/connection drop with no audit event instead of the previous 401 path; validate or encode the bearer token before comparing so bad tokens are rejected normally.
Useful? React with 👍 / 👎.
| return False | ||
| events.append(now) | ||
| return True | ||
| with self._lock: |
There was a problem hiding this comment.
Because now and cutoff are captured before entering this newly added lock, a request that waits here while another thread is evicting/checking a large limiter map can make its decision using an already-stale window. If entries expire during that wait, they remain counted for this request and can incorrectly return 429 or byte-limit a valid request; the same pre-lock timestamp pattern exists in ByteRateLimiter.allow().
Useful? React with 👍 / 👎.
1 participant
Summary
RateLimiter.allow()andByteRateLimiter.allow()with a per-instancethreading.Lockso concurrent requests underThreadingHTTPServercannot race past the limit checks.allow(), preventing unbounded memory growth without background threads.hmac.compare_digest(no short-circuit on match position) instead of a non-constant-timeinset lookup.Closes #82
Test plan
./scripts/check.shpassespython3 -m py_compile services/model-broker/openphone_model_broker.pyallow()calls against a 30/min limit admit exactly 30 events🤖 Generated with Claude Code