Skip to content

P31: HTTP listener resource limits (sc serve) - #41

Merged
tonibergholm merged 14 commits into
mainfrom
p31-listener-limits
Jul 10, 2026
Merged

tonibergholm merged 14 commits into
mainfrom
p31-listener-limits

Conversation

@tonibergholm

Copy link
Copy Markdown
Member

Implements #38 (audit High #2, from the #24 audit-response map): all four sc serve bounds, zero new dependencies, PROTOCOL_VERSION stays 3.

What ships

  • --max-connections <n> (default 32, 0 = unlimited): atomic slot count + RAII SlotGuard; at the limit a new connection gets an immediate 503 at the HTTP opening and closes — no queuing. The accept loop also survives OS thread-spawn failure (Builder::spawn, shed + backoff instead of panic).
  • --timeout <secs> (default 300, 0 = disabled): session read+write socket timeouts replace the old post-200 clear; progress-based under P25 chunking (only true zero-byte stalls trip it); connection-fatal; opening keeps its 30s. Recorded divergence from git-daemon's off-default.
  • --max-pack-size <bytes> (default 16 GiB, floor 256 MiB = MAX_OBJECT_SIZE, 0 = unlimited; applies to both --http and --stdio): counted mid-stream spool abort (receive.maxInputSize shape) → new wire code EC_TOO_LARGE (old clients degrade to message text), best-effort reply then close. Recorded divergence from git's unlimited default.
  • 8 MiB read-only drain cap (RO_DRAIN_CAP): small doomed pushes still get the clean EC_READONLY; hostile bulk is dropped mid-send — an ro token can no longer write unbounded bytes to .sc/tmp.
  • Accept-loop backoff: Go's exact 5ms→×2→1s shape, hardcoded.

Mechanism: WirePolicy (shared wire layer) + ServeLimits (listener config). Closes ADR-0036's three named operational boundaries plus the unbounded-spool gap the #27 research surfaced (never previously recorded).

Docs & proof

  • ADR-0041, THREAT-MODEL closures + the Decide: env-var secret exposure — fd/stdin injection or defer #31/Decide: signature replay boundary — position binding or defer #32 boundary re-affirmation wordings, CLAUDE.md (flags + "Phase 31 is built").
  • demo/run_limits_demo.sh: busy shed proven via deterministic raw-socket 503 probe, floor cap proven on the gated push path, below-floor startup refusal, zero .sc/tmp residue, run twice.
  • 15 new tests (wire budget/policy/drain + socket-level slot/timeout/backoff); workspace 486 green, clippy -D warnings clean, all neighboring transport demos re-run green.
  • Subagent-driven build with per-task review; final whole-branch review verdict: ready to merge (two Importants found and fixed: spawn-failure accept-loop death, PackTooLarge double-prefixed client message).

Deferred (recorded in ADR-0041): 503-shed RST race drain-before-close, demo pick_port TOCTOU sweep.

https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB

Brainstormed design for the four serve bounds: ServeLimits CLI struct,
atomic-slot connection cap with RAII guard + 503, persistent session
read/write timeouts, WirePolicy with counted spool abort (EC_TOO_LARGE),
8 MiB RO drain cap, Go-style accept backoff. Spool cap applies to both
--http and --stdio (brainstorm decision).

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
Eight TDD tasks: typed errors + EC_TOO_LARGE, read_pack_stream byte
budget, WirePolicy + capped spool/RO drain, ServeLimits + slots/503 +
session timeouts + accept backoff, CLI flags, demo, ADR-0041 + doc
sweep, final verification. Also corrects the spec: a mid-stream spool
abort is connection-fatal (stream desync), not connection-reusable.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
…am (P31)

Add max_bytes parameter to read_pack_stream with P31 Task 2 semantics:
- max_bytes == 0 means unlimited
- max_bytes > 0 caps the total bytes written to sink
- Over-budget chunk abort with Error::PackTooLarge before writing to sink
- TDD: two new tests verify budget enforcement and pass-through
- All existing call sites pass 0 (unlimited) for now; Task 3 threads real cap

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
- Add three new flags to Cmd::Serve: --max-connections (--http only),
  --timeout (--http only), --max-pack-size (both transports).
- CLI dispatch routes flags to run_serve; stdio rejects http-only flags
  with clear errors; http builds ServeLimits from flags, defaults from
  ServeLimits::default(); stdio validates and applies --max-pack-size
  via wire::WirePolicy with serve_with_policy (not legacy serve).
- Flags resolve per brief §Step 2: max_pack_size defaults to
  DEFAULT_MAX_PACK_SIZE, validated through validate_max_pack_size,
  wired into WirePolicy.read_only=false for stdio and custom limits
  for --http. Both manual verification commands pass (--timeout rejects
  on stdio, --max-pack-size=1024 fails floor check).

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
Proves sc serve --http's --max-connections/--timeout/--max-pack-size
knobs end to end: a raw held TCP connection (exec 3<>/dev/tcp) occupies
the single slot and a second connection is shed with 503/"server busy"
(a deterministic raw-socket probe plus a best-effort real `sc clone`,
which occasionally observes a benign connection-reset instead of a clean
503 due to a TCP quirk of the shed path — both outcomes are accepted and
explained inline); closing the holder frees the slot for a fresh clone.
--timeout's post-opening session-idle reap can't be cheaply driven
without a wire client, so the script says so and points at the existing
unit tests instead of faking coverage. --max-pack-size at the 256 MiB
floor serves normally while a value below the floor refuses to start.
Zero .sc/tmp residue asserted; runs clean twice in a row.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
Review Important: --max-pack-size gates only the PutPack spool (the
server's receive path); a clone's GetPack never consults it, so the
earlier clone-through-the-capped-server step passed even if the cap's
enforcement broke. Step 6 now commits in the capped-server clone and
pushes back through sc+http:// — the small pack exercises the gated
spill path end to end, verified by a fresh clone seeing the pushed
content — and the RESULT line claims exactly that ("a small push lands
through the PutPack spool under the 256 MiB floor cap") instead of
"serves normally". Also (review Minor) the raw 503 probe read is now
bounded (read -r -t 10) so a regressed accept-but-stall server fails
the demo within 10s instead of hanging until the 30s opening reap.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
…e, doc fixes

Six final-review findings against the P31 listener-resource-limits work,
all small and isolated:

1. serve_http_listener now uses std::thread::Builder::spawn instead of the
   panicking std::thread::spawn, so OS thread-creation failure (EAGAIN) is
   logged and paced with the existing accept backoff instead of killing the
   accept loop.
2. err_to_wire no longer double-prefixes Error::PackTooLarge's Display text
   on round trip; the read_pack_stream abort payload now names the cap and
   points at --max-pack-size / splitting the push.
3. Removed a stray duplicated `# justifies it (P29)` line in CLAUDE.md.
4. ADR-0041 now states precisely how the 503 pre-handshake seam differs
   from ADR-0040's 401/404 (before vs. after reading the opening), and
   records two known boundaries: the RST race on a 503 write, and
   --max-connections 0 leaving thread exhaustion reachable.
5. Fixed the ServeLimits doc comment's stray reference to ro_drain_cap,
   which actually lives on wire::WirePolicy.
6. read_frame_inner's EOF diagnostics now include the source io::Error
   instead of swallowing it, so a session-timeout trip is distinguishable
   from a generic EOF.
7. Dropped the err_from_wire delegating alias, updating its two call sites
   to call wire_to_err directly.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
@tonibergholm
tonibergholm merged commit c277634 into main Jul 10, 2026
5 checks passed
@tonibergholm
tonibergholm deleted the p31-listener-limits branch July 10, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant