You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 403ff69
Browse filesBrowse the repository at this point in the historyBrowse files
perf: bound the two statements whose cost grew with the data
Measurement said the stall was pool contention, so the first instinct was to
make the pool bigger. That was wrong: the requests waiting for a connection
were waiting behind two statements doing far more work than they needed.
Log redaction ran on every row in the window. Four chained regexp_replace sat
in the same SELECT as ORDER BY time DESC LIMIT, and a projection there is
evaluated below the top-N — so returning a hundred entries redacted every row
the window held. Measured at 140ms for 100 rows out of 372,000 against 8ms
once the top-N runs first, and independently at 449ms against 11ms over
400,000 rows with more realistic bodies — the plan confirming the projection
moved above the top-N and EXCEPT ALL confirming the rows are identical either
way. It was the one statement whose cost grew
with retention rather than with the answer — with no search term; a search
still has to redact every row it tests, because it must match the text the
reader is shown rather than the secret underneath it.
The endpoint boundary scan read the whole window. Its bounds came from a CTE
joined against the scan, and a bound derived from a joined row is not a
constant DuckDB can push into a Parquet scan — so a query that wants the two
partial minutes at the window's edges read everything between them: 799,333
rows of 800,000 to return 7,333, and 240,000 to return 667 in a smaller
independent reproduction. The bounds are computed in Go and bound as
parameters now; the same queries scan 11,333 and 35,200.
Dashboard queries get a twenty-second deadline. The SQL API has always had one;
these endpoints, which every widget calls, had none, so a pathological query
held its connection until the client gave up. A query that exceeds it answers
503: running out of time under load is not an internal error, and reporting it
as one logs at ERROR and counts towards the 5xx rate that pages someone.
And the half of the picture the last commit lacked: how long a statement takes
to execute. The connection is acquired before the timer starts, because
DB.QueryContext would fold the wait for a free connection into the statement's
duration — and telling queueing from execution is the entire reason to measure
it. A test holds the pool's only connection to prove the two do not overlap.
Measured across the same seed and load — 96 batches of 4,000 spans over ten
hours, continuous ingest, eight concurrent clients, 150s, four-connection pool
in both arms:
req p50 p95 p99 cheap p99
before 11,970 45ms 275ms 498ms 178ms
after 15,642 31ms 191ms 358ms 122ms
Throughput up 31%, p95 down 31%. The max column is deliberately absent: with a
closed-loop generator one server-side stall appears as eight slow requests, so
a single run cannot separate a shifted distribution from one bad event.
No pool-sizing change. An earlier version of this commit rewrote
resolveDuckDBMaxConns, and the arm that appeared to justify it compared a
hand-pinned pool of four against an auto-sized seven — which is not what the
rule would have done on any host: unchanged on the four-core reference host,
and a reduction on larger ones that nothing here measured. The evidence says a
cheap query should not queue behind an expensive one; it does not say what the
pool should be. The statement histogram and the pool gauges are what should
answer that, on a real host.
Claude-Session: https://claude.ai/code/session_017ovmTezqzvxVP8P9MGDQNk
0 commit comments