Repository navigation
P31: HTTP listener resource limits (sc serve) - #41
Merged
Merged
Conversation
Brainstormed design for the four serve bounds: ServeLimits CLI struct, atomic-slot connection cap with RAII guard + 503, persistent session read/write timeouts, WirePolicy with counted spool abort (EC_TOO_LARGE), 8 MiB RO drain cap, Go-style accept backoff. Spool cap applies to both --http and --stdio (brainstorm decision). Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
Eight TDD tasks: typed errors + EC_TOO_LARGE, read_pack_stream byte budget, WirePolicy + capped spool/RO drain, ServeLimits + slots/503 + session timeouts + accept backoff, CLI flags, demo, ADR-0041 + doc sweep, final verification. Also corrects the spec: a mid-stream spool abort is connection-fatal (stream desync), not connection-reusable. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
…am (P31) Add max_bytes parameter to read_pack_stream with P31 Task 2 semantics: - max_bytes == 0 means unlimited - max_bytes > 0 caps the total bytes written to sink - Over-budget chunk abort with Error::PackTooLarge before writing to sink - TDD: two new tests verify budget enforcement and pass-through - All existing call sites pass 0 (unlimited) for now; Task 3 threads real cap Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
…t; fix stale timeout doc (P31 review) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
- Add three new flags to Cmd::Serve: --max-connections (--http only), --timeout (--http only), --max-pack-size (both transports). - CLI dispatch routes flags to run_serve; stdio rejects http-only flags with clear errors; http builds ServeLimits from flags, defaults from ServeLimits::default(); stdio validates and applies --max-pack-size via wire::WirePolicy with serve_with_policy (not legacy serve). - Flags resolve per brief §Step 2: max_pack_size defaults to DEFAULT_MAX_PACK_SIZE, validated through validate_max_pack_size, wired into WirePolicy.read_only=false for stdio and custom limits for --http. Both manual verification commands pass (--timeout rejects on stdio, --max-pack-size=1024 fails floor check). Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
Proves sc serve --http's --max-connections/--timeout/--max-pack-size knobs end to end: a raw held TCP connection (exec 3<>/dev/tcp) occupies the single slot and a second connection is shed with 503/"server busy" (a deterministic raw-socket probe plus a best-effort real `sc clone`, which occasionally observes a benign connection-reset instead of a clean 503 due to a TCP quirk of the shed path — both outcomes are accepted and explained inline); closing the holder frees the slot for a fresh clone. --timeout's post-opening session-idle reap can't be cheaply driven without a wire client, so the script says so and points at the existing unit tests instead of faking coverage. --max-pack-size at the 256 MiB floor serves normally while a value below the floor refuses to start. Zero .sc/tmp residue asserted; runs clean twice in a row. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
Review Important: --max-pack-size gates only the PutPack spool (the
server's receive path); a clone's GetPack never consults it, so the
earlier clone-through-the-capped-server step passed even if the cap's
enforcement broke. Step 6 now commits in the capped-server clone and
pushes back through sc+http:// — the small pack exercises the gated
spill path end to end, verified by a fresh clone seeing the pushed
content — and the RESULT line claims exactly that ("a small push lands
through the PutPack spool under the 256 MiB floor cap") instead of
"serves normally". Also (review Minor) the raw 503 probe read is now
bounded (read -r -t 10) so a regressed accept-but-stall server fails
the demo within 10s instead of hanging until the 30s opening reap.
Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
…e, doc fixes Six final-review findings against the P31 listener-resource-limits work, all small and isolated: 1. serve_http_listener now uses std::thread::Builder::spawn instead of the panicking std::thread::spawn, so OS thread-creation failure (EAGAIN) is logged and paced with the existing accept backoff instead of killing the accept loop. 2. err_to_wire no longer double-prefixes Error::PackTooLarge's Display text on round trip; the read_pack_stream abort payload now names the cap and points at --max-pack-size / splitting the push. 3. Removed a stray duplicated `# justifies it (P29)` line in CLAUDE.md. 4. ADR-0041 now states precisely how the 503 pre-handshake seam differs from ADR-0040's 401/404 (before vs. after reading the opening), and records two known boundaries: the RST race on a 503 write, and --max-connections 0 leaving thread exhaustion reachable. 5. Fixed the ServeLimits doc comment's stray reference to ro_drain_cap, which actually lives on wire::WirePolicy. 6. read_frame_inner's EOF diagnostics now include the source io::Error instead of swallowing it, so a session-timeout trip is distinguishable from a generic EOF. 7. Dropped the err_from_wire delegating alias, updating its two call sites to call wire_to_err directly. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #38 (audit High #2, from the #24 audit-response map): all four
sc servebounds, zero new dependencies,PROTOCOL_VERSIONstays 3.What ships
--max-connections <n>(default 32, 0 = unlimited): atomic slot count + RAIISlotGuard; at the limit a new connection gets an immediate503at the HTTP opening and closes — no queuing. The accept loop also survives OS thread-spawn failure (Builder::spawn, shed + backoff instead of panic).--timeout <secs>(default 300, 0 = disabled): session read+write socket timeouts replace the old post-200 clear; progress-based under P25 chunking (only true zero-byte stalls trip it); connection-fatal; opening keeps its 30s. Recorded divergence from git-daemon's off-default.--max-pack-size <bytes>(default 16 GiB, floor 256 MiB =MAX_OBJECT_SIZE, 0 = unlimited; applies to both--httpand--stdio): counted mid-stream spool abort (receive.maxInputSizeshape) → new wire codeEC_TOO_LARGE(old clients degrade to message text), best-effort reply then close. Recorded divergence from git's unlimited default.RO_DRAIN_CAP): small doomed pushes still get the cleanEC_READONLY; hostile bulk is dropped mid-send — anrotoken can no longer write unbounded bytes to.sc/tmp.Mechanism:
WirePolicy(shared wire layer) +ServeLimits(listener config). Closes ADR-0036's three named operational boundaries plus the unbounded-spool gap the #27 research surfaced (never previously recorded).Docs & proof
ADR-0041, THREAT-MODEL closures + the Decide: env-var secret exposure — fd/stdin injection or defer #31/Decide: signature replay boundary — position binding or defer #32 boundary re-affirmation wordings, CLAUDE.md (flags + "Phase 31 is built").demo/run_limits_demo.sh: busy shed proven via deterministic raw-socket 503 probe, floor cap proven on the gated push path, below-floor startup refusal, zero.sc/tmpresidue, run twice.-D warningsclean, all neighboring transport demos re-run green.PackTooLargedouble-prefixed client message).Deferred (recorded in ADR-0041): 503-shed RST race drain-before-close, demo
pick_portTOCTOU sweep.https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB