Skip to content

Coalesce concurrent statement preparation - #498

Merged
bikeshedder merged 1 commit into
mainfrom
testable-statement-cache
Aug 4, 2026
Merged

Coalesce concurrent statement preparation#498
bikeshedder merged 1 commit into
mainfrom
testable-statement-cache

Conversation

@bikeshedder

Copy link
Copy Markdown
Collaborator

Rework the statement cache so that many tasks racing to prepare the same statement no longer each send a redundant, expensive PREPARE to the database. Each cache entry is now a tokio::sync::OnceCell: the first caller runs the preparation while the others wait for it, and if that caller fails or is cancelled, one of the waiters takes over. A failed preparation leaves the cell uninitialized, so the next caller simply retries.

The cells are held behind an Arc, which matters for eviction. A preparation in flight keeps working on its own cell, so a clear() or remove() landing mid-flight detaches that cell from the map instead of being silently undone once the preparation completes. Resurrecting an entry that way would defeat the very purpose of StatementCaches::remove(), which exists to drop statements that went stale.

The map lives in a StatementCacheInner<V>, with StatementCache as a thin StatementCacheInner<Statement> wrapper. The value type is generic purely as a test seam, not as a claim to reusability: Statement has no public constructor, so without it every test would need a live database. With it the interesting behaviour is covered by unit tests — coalescing to a single preparation under 128 concurrent tasks, eviction during preparation, size accounting against uninitialized cells, borrowed/owned key matching, and clear.

Based on the approach in #422 by @tgeoghegan.

@tgeoghegan

Copy link
Copy Markdown

For whatever it's worth: this LGTM, and with much better test coverage than I achieved in #422.

@bikeshedder

Copy link
Copy Markdown
Collaborator Author

This is my first AI assisted code change of Deadpool.

While writing the tests for it the Agent even found a real resource leak in tokio-postgres:

I'm just updating the CHANGELOG and will merge this shortly.

Rework the statement cache so that many tasks racing to prepare the same
statement no longer each send a redundant, expensive `PREPARE` to the
database. Each cache entry is now a `tokio::sync::OnceCell`: the first caller
runs the preparation while the others wait for it, and if that caller fails or
is cancelled, one of the waiters takes over. A failed preparation leaves the
cell uninitialized, so the next caller simply retries.

The cells are held behind an `Arc`, which matters for eviction. A preparation
in flight keeps working on its own cell, so a `clear()` or `remove()` landing
mid-flight detaches that cell from the map instead of being silently undone
once the preparation completes. Resurrecting an entry that way would defeat
the very purpose of `StatementCaches::remove()`, which exists to drop
statements that went stale.

The map lives in a `StatementCacheInner<V>`, with `StatementCache` as a thin
`StatementCacheInner<Statement>` wrapper. The value type is generic purely as a
test seam, not as a claim to reusability: `Statement` has no public constructor,
so without it every test would need a live database. With it the interesting
behaviour is covered by unit tests — coalescing to a single preparation under
128 concurrent tasks, eviction during preparation, size accounting against
uninitialized cells, borrowed/owned key matching, and clear.

Based on the approach in #422 by @tgeoghegan.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@bikeshedder
bikeshedder force-pushed the testable-statement-cache branch from 55f113d to 6d38f06 Compare August 4, 2026 20:10
@bikeshedder
bikeshedder merged commit 85d3405 into main Aug 4, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants