perf(ws): open a dedicated socket instead of queueing concurrent turns

[amondnet]

Summary

Closes #248.

With the opt-in Codex WebSocket transport (websocket = true, default off), concurrent requests sharing one WS pool key ran strictly one turn at a time: codex_ws::begin took the pooled connection's turn_lock with an unbounded lock_owned().await, so the second request parked until the first turn's terminal event.

Serializing turns on one socket is required — previous_response_id is only valid on the connection that produced it — but the behavior under contention should be "open another path", not "wait indefinitely". This implements the issue's option 1.

• begin now takes the turn slot with try_lock_owned. An idle pooled socket is probed and reused exactly as before; a busy one sends the concurrent turn over a dedicated connection.
• That overflow connection is deliberately left out of the pool (pool_key = None), which the existing machinery turns into three guarantees: run_turn only calls pool_insert for a connection carrying a pool_key, so it can never replace the session's healthy entry; evict() is a no-op for it, so its death can't evict that entry; and run_connection exits and closes the socket after its single turn, so neither the task nor the connection leaks.
• The overflow turn carries no continuation (reused == false) — the parallelism-for-continuation trade — while the pooled socket keeps its continuation state and keeps serving subsequent turns.
• Overflow is bounded: at most MAX_OVERFLOW_CONNECTIONS = 64 dedicated sockets may be live at once (RAII slot owned by the Connection, released when it drops after its single turn). At the ceiling begin refuses before any frame is sent, so the turn rides the existing AdapterFailure::BeforeHeaders path and spills over to HTTP rather than waiting — added in response to Greptile review.
• Supporting hardening: pool invalidation is now identity-checked (Arc::ptr_eq). Previously a connection that lost the pool slot in a cold-start race would evict the winner's healthy entry when its reader exited.

No config key, endpoint, CLI, or provider/model semantics change; the transport itself is already opt-in.

Milestone / spec

M7 — docs/m7-codex-websocket.md §5 "Connection pool" (reuse gate + new "Concurrent turns" bullet).

Checklist

• cargo build passes
• cargo test --all-features --workspace passes (959 unit + all integration groups green; new behavior covered)
• cargo clippy --all-targets --all-features -- -D warnings clean
• cargo fmt --all --check clean
• Source files stay under 500 lines — codex_ws.rs was already over (2.2k, transport + its test module); this PR does not add a new oversized file
• English only; matches surrounding style
• Frozen spec in docs/ updated (docs/m7-codex-websocket.md §5)
• User-facing docs considered: README.md documents no turn-serialization behavior; site/ documents only the websocket flag itself, and no config key/default/endpoint/CLI/provider semantics changed — no edit needed. wiki/ untouched (generated).
• No new GitHub Action

Notes for reviewers

• Regression evidence. concurrent_turn_opens_a_dedicated_connection_instead_of_waiting drives real loopback sockets: turn 1 pools connection 1, turn 2 reuses it and is held open mid-stream by the mock, then a third begin on the same key must return inside a 5s timeout. Reverting try_lock_owned to the old blocking lock_owned().await fails it at exactly the timeout (Elapsed(())); with the fix it passes. It further asserts the overflow socket is a second accept, closes after its single turn, and that turn 4 still lands on connection 1 (3 frames on connection 1, 1 on connection 2, 2 accepts total).
• Identity-checked eviction is covered by replaced_connection_cannot_evict_current_pool_entry. pool_insert keeps its last-writer-wins semantics — deliberately unchanged.
• The removed pub fn invalidate_pool_key had no callers left anywhere in src/, tests/, or benches/ after the conversion to invalidate_pool_entry.
• Worth a look: whether a metric (e.g. an overflow counter alongside shunt.codex_continuation) should be added to answer the issue's open question about how often concurrent agents actually share one x-claude-code-session-id. Left out here to keep the diff to the fix; happy to add as a follow-up.

---

Summary by cubic

Open a dedicated WebSocket for concurrent turns instead of queueing on the pooled session socket, with a safety cap that falls back to HTTP under heavy bursts. This removes stalls under contention; no API or config changes (transport stays opt-in via websocket = true).

• New Features

  • When the pooled socket is busy, open a one-shot overflow connection instead of waiting.
  • Overflow connections are not pooled, carry no continuation, and close after the turn; the pooled socket keeps continuation and future reuse.
  • Cap live overflow connections at 64; refuse excess before sending any frame and transparently fall back to HTTP.

• Bug Fixes

  • Identity-checked pool invalidation prevents a replaced connection from evicting the healthy pooled entry.

^{Written for commit f219f65. Summary will update on new commits.}

[gemini-code-assist]

Code Review

This pull request introduces support for concurrent turns on Codex WebSocket connections (issue #248). Instead of serializing concurrent requests on a single pooled connection and making them wait, a busy turn slot now triggers the creation of a dedicated, unpooled, one-shot connection for the concurrent turn, trading continuation reuse for parallelism. Additionally, it implements identity-checked invalidation (invalidate_pool_entry) to ensure that a stale or exiting connection only evicts itself from the pool and does not accidentally evict a newer, healthy replacement connection. Comprehensive unit tests have been added to verify these concurrent turn and race-condition behaviors. There are no review comments to assess, and I have no additional feedback to provide.

[codecov]

Codecov Report

❌ Patch coverage is 96.33803% with 13 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/adapters/responses/codex_ws.rs	96.33%	13 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

The PR replaces queued concurrent Codex WebSocket turns with bounded, dedicated overflow connections.

• Reuses an idle pooled socket while routing contention through one-shot connections.
• Limits live overflow connections to 64 and falls back to HTTP when capacity is exhausted.
• Uses identity-checked invalidation so obsolete connections cannot evict their replacements.
• Updates the WebSocket pooling specification and adds concurrency, admission, cleanup, and replacement-race tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
src/adapters/responses/codex_ws.rs	Adds bounded one-shot connections for pooled-socket contention, identity-safe pool invalidation, and comprehensive lifecycle and concurrency tests.
docs/m7-codex-websocket.md	Documents non-blocking pooled reuse, dedicated overflow behavior, the 64-connection ceiling, and HTTP fallback.

Sequence Diagram

sequenceDiagram
    participant Request
    participant Pool
    participant PooledWS as Pooled WebSocket
    participant Admission as Overflow Admission
    participant OverflowWS as Dedicated WebSocket
    participant HTTP

    Request->>Pool: begin(session key)
    alt pooled socket is idle
        Pool->>PooledWS: claim turn slot and probe
        PooledWS-->>Request: stream turn with continuation
    else pooled socket is busy
        Pool->>Admission: claim overflow slot
        alt slot available
            Admission->>OverflowWS: open one-shot connection
            OverflowWS-->>Request: stream turn without continuation
            OverflowWS->>Admission: close and release slot
        else ceiling reached
            Admission-->>Request: pre-stream transport failure
            Request->>HTTP: transparently retry over HTTP
        end
    end

Loading

_{Reviews (2): Last reviewed commit: "perf(ws): bound overflow connections wit..." | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will not alter performance

✅ 11 untouched benchmarks

---

_{Comparing amondnet/perf-ws-per-connection-turn_lock-serializes-conc (f219f65) with main (014252d)}

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
97.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud