fix(api): in-flight request coalescing for response cache + DB fallback cache [BUG-004]

[Morenikeoa]

Problem

Neither cacheMiddleware (src/middleware/cache.ts) nor withDbCacheFallback (src/middleware/db-cache-fallback.ts) de-duplicates concurrent in-flight requests for the same cache key. If N concurrent requests miss the cache for the same key at the same time, all N independently re-run the expensive backend call (Solana RPC or Supabase), and all N independently write to the cache on completion with no ordering guarantee — whichever happens to resolve last wins, regardless of which result actually reflects more current backend state.

This is both wasted backend load (thundering herd on every concurrency event, attack or not) and a correctness issue (an earlier-started-but-slower request can clobber a later-started-but-faster request's result).

Fix design process

This went through the full triple-solver-consensus process given the architectural complexity. All three independently proposed the same core approach — mirror the in-flight coalescing pattern this codebase already uses and trusts in src/routes/oracle-router.ts — then diverged on implementation details for cache.ts, which is harder than db-cache-fallback.ts because the "expensive work" being coalesced is await next() (a Hono middleware continuation that writes into the current request's own Context), not a plain value-returning function.

Synthesized from all three proposals:

• db-cache-fallback.ts: coalesce only queryFn() itself (new runCoalesced() helper). The cache write and staleness-header logic stay per-caller — every coalesced waiter (leader and followers alike) independently runs its own post-processing against its own Context. This was deliberately chosen over coalescing the whole write+header block, because one solver's first draft of that approach set staleness headers only on the leader's Context, silently dropping them from followers' HTTP responses — a real bug another solver caught only on a second pass.
• cache.ts: leader/follower split on cache miss. Only the first ("leader") request calls next(); concurrent ("follower") requests await the same promise and replay the leader's now-cached entry via a new serveEntry() helper (shared with the existing cache-HIT path) instead of calling next() themselves. If the leader's result isn't cacheable (error, non-JSON) or the leader throws, the follower falls back to calling next() itself for its own accurate response — verified safe because every route in this repo is a read-only GET handler with zero mutation endpoints (confirmed during the broader audit this fix came out of), so re-running a handler twice has no side-effect risk.
• No shared coalesce() helper extracted across the two files — cache.ts's Context-mutation semantics don't fit the same shape a plain promise-memoizing helper would need without adding indirection for marginal line savings. oracle-router.ts's own unabstracted version is left untouched — refactoring working code for style, with no bug-fix value, was unanimously rejected by all three solvers.

Proof of Fix

Added regression tests to both tests/middleware/db-cache-fallback.test.ts and a new tests/middleware/cache.test.ts:

• Concurrent-miss coalescing: 2 concurrent callers for the same key, handler/queryFn called exactly once.
• A test proving a slower concurrent request can no longer overwrite a faster one's cached result (the original race).
• Fallback-to-next() behavior when the leader's response isn't replayable.
• Existing HIT/304/sequential-miss behavior unchanged (no-op for the non-concurrent case).

I verified these are genuine regression tests, not tautologies: reverted just the two source files and reran — all 3 coalescing-specific tests failed with the handler/queryFn called twice instead of once. Restored the fix and they pass.

• All existing tests pass — output attached.
• tsc --noEmit clean (no separate lint script in this repo).

Test Output

✓ tests/middleware/cache.test.ts (6 tests) 101ms
✓ tests/middleware/db-cache-fallback.test.ts (4 tests) 93ms

Full suite: 301/302 passed (294 baseline + 7 new). The 1 failure (tests/sdk-smoke.test.ts) is pre-existing and unrelated — it asserts on an exact @percolatorct/sdk error-message string that has drifted from the locally-resolved SDK version in this environment.

Related

Found during a broader API audit; no existing open issue/PR covers this.

[vercel]

@Princessdada is attempting to deploy a commit to the Khubair Nasir's projects Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

Warning

Review limit reached

@Morenikeoa, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 24 minutes and 24 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b68fe6f1-7523-4369-8975-d0fc2b376fad

📥 Commits

Reviewing files that changed from the base of the PR and between b2751f4 and b8a19c4.

📒 Files selected for processing (4)

• src/middleware/cache.ts
• src/middleware/db-cache-fallback.ts
• tests/middleware/cache.test.ts
• tests/middleware/db-cache-fallback.test.ts

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}