Bound and evict stale buckets in the in-memory rate limiter

[Lakes41]

Difficulty: Advanced
Type: performance

Background

lib/rate-limit.ts implements a token-bucket rate limiter for the integration gateway routes, keyed by `ip:${ip}` and `wallet:${address}`. Buckets are stored in a module-level Map<string, Bucket> (const buckets = new Map<string, Bucket>()), created lazily on first request via getBucket() and never removed.

Problem

There is no eviction of stale entries. Every unique IP or wallet address that ever calls a rate-limited route (/api/integration/membership, /api/integration/verify) creates a permanent entry in buckets for the lifetime of the Node process. On a long-running instance exposed to the public internet, this is an unbounded-growth memory leak — an attacker could also deliberately spray requests from many spoofed X-Forwarded-For values to inflate memory usage, since the IP key is derived directly from a client-controllable header with no allowlist of trusted proxies.

Expected Outcome

The rate limiter's memory footprint stays bounded regardless of how many distinct IPs/wallets have made requests over the process lifetime, without changing the observable rate-limiting behavior (RateLimitResult semantics, retryAfter, remaining) for any individual key.

Suggested Implementation

1. Add a lastAccess timestamp to the Bucket interface (or reuse last) and implement a periodic or on-access sweep that deletes buckets whose tokens are fully refilled (i.e., inactive for at least WINDOW_MS) and haven't been touched recently — e.g., prune entries idle longer than N × WINDOW_MS.
2. Choose an eviction strategy appropriate for a single Node process without adding new runtime dependencies — e.g., an LRU cap (MAX_BUCKETS) combined with lazy sweep-on-write, or a setInterval sweep guarded to only run in a real server runtime (not during tests/edge builds).
3. Ensure the sweep doesn't introduce race conditions with concurrent take() calls (Node's request handling is single-threaded per event-loop tick, but be careful with any async boundaries introduced).
4. Add a way to reset/inspect bucket count for tests (mirroring resetLiveApiResilienceState() in lib/api/live.ts for the circuit breaker).
5. Update the file's doc comment (currently describing the "SINGLE-INSTANCE CAVEAT") to describe the new eviction behavior.
6. Extend test/rate-limit.test.ts with tests asserting the bucket map does not grow unboundedly after many distinct keys go idle.

Acceptance Criteria

• After a large number of distinct IP/wallet keys are rate-limited and then go idle, the internal store size is bounded (verified via a test-only introspection export).
• Existing rate-limit behavior (limited, retryAfter, remaining for active keys) is unchanged — all current test/rate-limit.test.ts assertions still pass.
• No new runtime dependency is introduced.
• Sweep/eviction logic is covered by new unit tests, including a test that an evicted-then-returning key gets a fresh, correctly-initialized bucket (not incorrectly rate-limited or given a free pass).
• npm run typecheck, npm run lint, npm test pass.

Likely Affected Files/Directories

• lib/rate-limit.ts
• test/rate-limit.test.ts
• docs/deployment.md (if it references rate-limiter behavior)

---