[SLOP(claude-opus-4-8)] feat(util): add rate limiter primitive and ingress throttles for actor create and gateway websocket

[MasterPtato]

No description provided.

[MasterPtato]

Stack for rivet-dev/rivet

• #5552 [slopfix] fix(universaldb): match fdb conflict semantics in …
• #5551 [slopfix] fix(universaldb,util,pegboard-envoy): index dedup …
• #5398 feat(kitchen-sink): add churnDb actor and seed driver for co …
• #5397 chore: Clean up
• #5382 [slopfix] docs(self-hosting): correct postgres config format …
• #5356 [slopfix] fix(pegboard): decrement kv_storage_used metric on …
• #5354 [slopfix] refactor(universaldb,ups): single/multi-node udb o …
• #5353 [slopfix] perf(universaldb): fold postgres drain_batch into …
• #5342 [SLOP(claude-opus-4-8-high)] perf(universaldb): batch leader …
• #5341 [slopfix] docs(self-hosting): promote postgres from experime …
• #5340 [slopfix] fix(self-host): repair dev compose generation, kit …
• #5337 [SLOP(claude-opus-4-8-high)] feat(universaldb): graceful pos …
• #5336 [slopfix] test(universaldb): postgres leader failover + abor …
• #5335 [SLOP(claude-opus-4-8-high)] feat(ups): table-backed postgre …
• #5329 [SLOP(claude-opus-4-8-high)] feat(universaldb): postgres lea …
• #5327 [SLOP(claude-opus-4-8)] feat(pegboard-envoy): rate-limit env …
• #5326 [SLOP(claude-opus-4-8)] feat(util): add rate limiter primiti … 👈
• #5322 [SLOP(claude-opus-4-8)] feat(rivetkit-core): add serde durat …
• #5321 Add observation for serde_bare
• #5320 Add observation and ser/de metrics for serde calls
• #5314 [SLOP(claude-opus-4-8-medium)] chore: add profiling cargo pr …
• #5306 [SLOP(claude-opus-4-8)] fix(kitchen-sink): serve built front …
• main

Get stack: forklift get 5326
Push local edits: forklift submit
Merge when ready: forklift merge 5326

_{change pnyswwtm}

[claude]

Review

Nice consolidation of the ad hoc backoff.rs + guard-core RateLimiter into a single shared rivet_util::throttle primitive, with a LeakyBucket mode added on top and good unit test coverage for the new module. A few issues worth addressing.

Bug: RateLimiter::acquire() leaks a token every cycle for FixedWindow

engine/packages/util/src/throttle.rs (acquire(), FixedWindow arm, ~line 240-245):

} else {
    tokio::time::sleep(*period).await;
    *requests_remaining = *requests_limit;
    *reset_time = Instant::now() + *period;
}

When the bucket is empty, this sleeps a full period and then resets requests_remaining to the full limit, but never decrements it to account for the request that just waited and is being admitted. Compare with the LeakyBucket arm just below, which correctly leaves requests_remaining at 0 after the deficit sleep (the woken caller consumes the newly produced token instead of getting a bonus one).

Concretely, for requests: 2, period: 100ms: two acquire() calls drain remaining to 0 instantly; a third acquire() waits 100ms then resets remaining to 2 (should be 1); a fourth and fifth acquire() then both return instantly since remaining is 2 again; only the sixth call waits. So every time the window has to block, one extra request is admitted for free, and the sustained rate converges to requests + 1 per period instead of requests per period. try_acquire() does not have this problem since it decrements immediately after refilling in the same call.

This path is not exercised by production callers in this PR (proxy_service.rs and pegboard_actor_create both use try_acquire, and the only .acquire() caller uses LeakyBucket), so it is latent rather than actively causing a bypass today. But it is part of the new shared, tested public API, so it should be fixed now by decrementing requests_remaining after the reset, mirroring the LeakyBucket arm. The existing test fixed_window_acquire_blocks_until_window_resets only asserts on elapsed time and would not catch this; worth adding a follow-up test that drives a couple of blocking cycles and asserts the achieved rate matches requests per period.

ws_to_tunnel_task: rate-limit wait is not cancellable by the abort signal

In engine/packages/pegboard-gateway2/src/ws_to_tunnel_task.rs, rate_limit.acquire().await is called at the top of the loop, before the tokio::select! that races ws_rx.try_next() against ws_to_tunnel_abort_rx.changed(). Because the acquire sits outside the select, an abort signal cannot preempt it. With the current defaults (drip_rate_ms = 10) the worst-case delay to observe an abort is tiny, but it scales linearly with gateway_websocket_rate_limit_drip_rate_ms, so a larger configured drip rate would directly translate into slower shutdown/abort responsiveness for that connection. Consider racing the acquire inside the select as well so abort always wins immediately.

Per-namespace actor-creation limiter is process-local

engine/packages/pegboard/src/ops/actor/create.rs keys the rate limiter cache off a static OnceLock<moka::Cache<...>>, which is per-process state. If pegboard_actor_create (invoked from api-peer) runs behind multiple replicas, the effective cap for a namespace becomes N times the configured burst/rate rather than a global namespace-wide limit, since each replica keeps its own independent bucket. That may be an accepted trade-off, similar to how the guard-core per-IP limiter is also node-local, but the doc comments ("Max burst of actor creations per namespace before throttling") read as if it is a hard global cap. Worth a one-line comment noting the per-process scope, or confirming that is intentional.

Nice fix worth calling out

Switching the guard-core rate_limiters / in_flight_counters moka caches from time_to_live to time_to_idle in proxy_service.rs looks like a real correctness improvement bundled into this refactor. With time_to_live, a sustained high-rate client's limiter entry would periodically expire mid-attack and silently reset their quota, letting them bypass the limit every TTL window regardless of activity. time_to_idle closes that gap since an actively-hammering client's entry never idles out.

Test coverage

The new throttle.rs unit tests are thorough for try_acquire/acquire across both modes. The two new production integration points, the actor.creation_rate_limit rejection path in pegboard_actor_create and the leaky-bucket throttle wired into ws_to_tunnel_task, do not have any test coverage in this PR. Even a basic operation-level test asserting CreationRateLimit is returned once the configured burst is exhausted would help guard the wiring, and would exercise the config plumbing added in pegboard.rs.

Everything else, including config plumbing, error wiring for actor.creation_rate_limit to 429, and the Backoff rename/move, looks straightforward and consistent with existing conventions: hard tabs, imports at top, exhaustive enum matches, no _ => fallback.