Skip to content

feat: request-side backpressure via a max_in_flight pool option - #184

Merged
puzza007 merged 2 commits into
masterfrom
request-backpressure
Jul 18, 2026
Merged

puzza007 merged 2 commits into
masterfrom
request-backpressure

Conversation

@puzza007

Copy link
Copy Markdown
Owner

Admission had no bound: a request burst piled into every worker's Reqs map and the C multi without limit, so overload showed up as memory growth and eventual timeouts rather than a signal callers can act on. The new {max_in_flight, N | infinity} pool option (default infinity — existing pools are unchanged) caps concurrently in-flight requests per worker, enforced by a single capacity gate ahead of both the sync and async admission paths.

A full worker replies {overload, self()} without registering anything, and pool_call offers the request to the remaining workers in random-rotation order, skipping the one that just rejected — so a partially-loaded pool always admits, and random routing can't strand requests on a busy worker while others idle. Only when every worker is at its cap does the caller get a fast {error, #{code => overload}}, the signal for shedding load or retrying with backoff. All admission attempts flow through one helper using wpool's own call doors, so the worker-death exit shape is caught in exactly one place: a death on the first attempt still reports worker_died (preserving the #178 semantics and the unknown-pool distinction), while a death mid-spillover just means that worker has nothing to offer.

The option is validated and split from the curl-multi options in katipo_pool:start/3 itself, so a bad value fails the API call with a bad_opts error instead of surfacing as worker-init crashes under wpool, and pool options get an honest pool_opts() type rather than overloading curlmopts(). The model header notes why capacity rejection preserves the verified properties by construction: it restricts admission before a request ever enters Reqs, adding no delivery paths.

Tests cover deterministic fill-via-spillover (two slow requests occupy both workers of a size-2, cap-1 pool), sync and async overload rejection, capacity recovery after a cancel, and bad-option rejection at pool start. 198 tests, dialyzer, xref, and lint pass.

Stacked on #183 (alias-routed commands) → #182 (streaming) → #180 (timeout abort); merge bottom-up and I'll rebase each layer as the one below lands.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Vf2gxZLpxT4HvBfZKCgP3g

@codecov

codecov Bot commented Jul 18, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.75000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.16%. Comparing base (eebb5e6) to head (2244db0).

Files with missing lines Patch % Lines
src/katipo.erl 93.75% 1 Missing ⚠️
src/katipo_worker.erl 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #184      +/-   ##
==========================================
+ Coverage   77.79%   78.16%   +0.37%     
==========================================
  Files           9        9              
  Lines        1342     1365      +23     
==========================================
+ Hits         1044     1067      +23     
  Misses        298      298              
Flag Coverage Δ
c 78.16% <93.75%> (+0.37%) ⬆️
erlang 78.16% <93.75%> (+0.37%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@puzza007
puzza007 force-pushed the alias-routed-commands branch from a327ed8 to 0527523 Compare July 18, 2026 08:48
@puzza007
puzza007 force-pushed the request-backpressure branch from 098536b to d82a779 Compare July 18, 2026 08:48
@puzza007
puzza007 force-pushed the alias-routed-commands branch from 0527523 to 29748d1 Compare July 18, 2026 08:59
@puzza007
puzza007 force-pushed the request-backpressure branch from d82a779 to cc2c140 Compare July 18, 2026 08:59
@puzza007
puzza007 force-pushed the alias-routed-commands branch from 29748d1 to 29656bd Compare July 18, 2026 09:05
@puzza007
puzza007 force-pushed the request-backpressure branch from cc2c140 to 6178814 Compare July 18, 2026 09:05
Base automatically changed from alias-routed-commands to master July 18, 2026 09:08
Admission had no bound: a burst piled into every worker's Reqs map and
the C multi without limit, so overload showed up as memory growth and
timeouts instead of a signal callers can act on. The new
{max_in_flight, N | infinity} pool option (default infinity, the
previous behavior) caps concurrently in-flight requests per worker,
enforced by a single capacity gate ahead of both admission kinds.

A full worker replies {overload, self()} without registering anything,
and pool_call offers the request to the remaining workers -- in
random-rotation order, skipping the worker that just rejected -- so a
partially-loaded pool always admits; only when every worker is at its
cap does the caller get a fast {error, #{code => overload}}. All
admission attempts go through one helper (wpool doors only), so the
worker-death exit shape is caught in exactly one place: a death on the
first attempt still reports worker_died, while a death mid-spillover
just means that worker has nothing to offer. The option is validated
and split from the curl-multi options in katipo_pool:start/3 itself, so
a bad value fails the API call directly instead of surfacing as worker
init crashes, and pool options get an honest pool_opts() type instead
of overloading curlmopts().

Tests cover fill-via-spillover, sync and async overload rejection,
capacity recovery after cancel, and bad-option rejection at pool start;
the suites' white-box state matches gain the new record field.
198 tests, dialyzer, xref, and lint pass.
@puzza007
puzza007 force-pushed the request-backpressure branch 4 times, most recently from 135f601 to 7326048 Compare July 18, 2026 09:31
curl.interface was the one test that talked to the public internet
(with default 30s timeouts, so an httpbin.org outage rode into CT's
timetrap -- it flaked two merges in a row). Binding to a physical
interface cannot be verified against local destinations: Linux
device-bound sockets black-hole traffic to host-local addresses, which
is delivered via lo (macOS's IP_BOUND_IF tolerates it, which made that
approach look viable locally). Bind to the loopback interface by its
discovered name instead and talk to the local httpbin over it -- the
device bind is then self-consistent on every platform, and that the
option is genuinely passed through is proven by interface_unknown's
rejection of a bogus name.

The loopback name comes from inet:getifaddrs rather than per-platform
hardcoding, replacing the ens5/eth0/ens4 lists and the darwin special
case. Fully local, deterministic, and strict: the test asserts a 200
with the same bare group opts as its sibling tests.
@puzza007
puzza007 force-pushed the request-backpressure branch from 7326048 to 2244db0 Compare July 18, 2026 09:34
@puzza007
puzza007 merged commit 046fce5 into master Jul 18, 2026
12 checks passed
@puzza007
puzza007 deleted the request-backpressure branch July 18, 2026 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant