Repository navigation
Fix idle reaper closing /ws sessions mid-step - #1618
Conversation
The /ws handler stamped activity only after reset/step returned, so the idle reaper could close a session's env while a step was still running. Reaping also freed the session's slot while its socket kept serving the closed env, letting more envs run than max_concurrent_envs. Track operations in flight per session and skip them in the reaper; stamp activity when an operation starts and ends; stop serving (SESSION_ERROR, close) when a socket's session no longer exists.
A /ws client can attach a second socket to its own session with ?session_id=. When that socket's message finished, discarding the session from the in-flight set let the reaper close the env under the first socket's step. Keep a count per session instead, and add a regression test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
burtenshaw
left a comment
There was a problem hiding this comment.
Thanks, this fixes the /ws case. I checked the parts that matter for a change like this:
- The in-flight flag is set right before the
tryand released in itsfinally. Exceptions, a failed send,close, cancellation and a disconnect mid-step all release it. I checked cancellation with an async step and with a sync step running in the executor. - None of the
_session_lockblocks awaits. So the reaper's re-check and the session pop run without yielding, and so do the handler's session check and the flag set. The two cannot interleave. - Idle sessions are still reaped, and a reaped socket can no longer run steps beyond
max_concurrent_envs. SESSION_ERRORis already part of the WebSocket protocol, so clients see no new message type.
One gap, fixed in c477f92. The flag was a set, but two sockets can share a session. A /ws client can learn its session ID with openenv/session/create and attach a second socket with ?session_id=. When the second socket's message finished, it removed the session from the set while the first socket's step was still running. Once the second socket disconnected, the reaper closed the env mid-step. The commit counts operations per session instead and adds test_second_socket_on_session_keeps_step_in_flight, which fails without the change. The test file passes: 73 passed, 1 skipped.
Two follow-ups outside this PR:
-
The
/mcpWebSocket route has the same bug onmain. Activity is stamped only aftermcp_handlerreturns, so atools/callthat outlastssession_timeouthas its env closed mid-call, and the socket keeps serving the closed env. The same count around themcp_handlercall would fix it:Sketch for the
/mcpWebSocket loopraw_message = await websocket.receive_text() if session_id not in self._sessions: error_resp = JsonRpcResponse.error_response( JsonRpcErrorCode.SERVER_ERROR, "Session expired or was closed; open a new connection", ) await websocket.send_text(error_resp.model_dump_json()) break # ... parse the request as today ... in_flight = self._session_operations_in_flight in_flight[session_id] = in_flight.get(session_id, 0) + 1 try: response = await mcp_handler( jsonrpc_request, session_env=session_env, session_id=session_id ) await websocket.send_text(response.model_dump_json()) except Exception as e: ... # unchanged finally: in_flight[session_id] -= 1 if in_flight[session_id] == 0: del in_flight[session_id]
-
After a reap,
EnvClientreports theSESSION_ERRORframe as a genericRuntimeError. A retriedstep()then raisesSessionLostError, and a retriedreset()reconnects. Onmainboth calls ran on the closed env, so this is an improvement. But a training loop that already catchesSessionLostErrorwill not catch the first error. Treating aSESSION_ERRORframe as a lost session in the client would make the two consistent.session_timeoutis off by default, so this does not block the PR.
I'll merge once CI passes on the new head.
|
@burtenshaw Thanks for the fix and the review. Happy to take both follow-ups: the /mcp WebSocket fix along the lines of your sketch, and having EnvClient raise SessionLostError on a SESSION_ERROR frame. Should they be one PR or two? |
Summary
With
session_timeoutset, the idle reaper can close a plain/wssession while itsreset/stepis still running: the handler stamps activity only after the call returns. Reaping also frees the session's slot while the socket keeps serving the closed env, so more envs can run thanmax_concurrent_envsallows. Steps that wait on slow external work (LLM calls, simulators) hit this whenever a step outlasts the timeout. Sessions attached through?session_id=and/harnessare already protected by_session_websocket_attachments; plain/wssessions are not.This PR tracks operations in flight per session so the reaper skips them, stamps activity when an operation starts and ends, and stops serving a socket whose session no longer exists (
SESSION_ERROR, then close).Found while building parley-sim, which includes a standalone repro.
Type of Change
Alignment Checklist
.claude/docs/PRINCIPLES.mdand this PR aligns with our principles.claude/docs/INVARIANTS.mdand no invariants are violated/pre-submit-pr(orbash .claude/hooks/lint.shand tests) and addressed all issuesRFC Status
Test Plan
Repro: an env whose
step_asyncsleeps 9 s, withmax_concurrent_envs=1, session_timeout=2.0; reset, wait 1.5 s, step, then open a second session.CAPACITY_REACHED)New
TestWebSocketSessionReaperintests/core/test_production_mode_routes.py:SESSION_ERRORand is closed, and a new session fits under the cap (fails on main);Rebased on
mainat ca6df26.tests/core/test_production_mode_routes.py: 72 passed, 1 skipped. The CI test command: 6171 passed, 16 failed; the same 16 fail onmainin my sandbox (TextArena can't download NLTK data through my proxy, and four Jupyter sandbox tests).bash .claude/hooks/lint.shpasses.Out of scope: the HTTP
/mcppath has the same stamp-after-call pattern for longtools/callrequests; a follow-up could cover it with a per-session counter.Claude Code Review
Alignment Review Report
Automated Checks
bash .claude/hooks/lint.shpasses. (ruff format --checkalso flagssrc/openenv/core/harness/README.md, which this PR doesn't touch; it is flagged onmaintoo.)check-debug.shfinds nothing in the changed lines; its hits are existing docstring examples in other files.Open RFCs Context
/harnessturns are bounded byHarnessConfig.session_timeout_sand are already exempt from reaping through_session_websocket_attachments. Unchanged here.Tier 1: Fixes Required
None.
Tier 2: Alignment Discussion
Principle Conflicts
None identified. Gym signatures, wire types and the agent/infrastructure boundary are unchanged. No new message types: the reaped-socket case reuses
WSErrorCode.SESSION_ERROR.RFC Conflicts
None identified. The change is in
src/openenv/core/but adds no API.One behaviour change for reviewers: a
/wssocket whose session was destroyed (by the reaper or shutdown) now getsSESSION_ERRORand is closed. Before, it kept serving the closed env outside the session cap.Summary
Note
Medium Risk
Changes session lifecycle and concurrency accounting for all
/wstraffic whensession_timeoutis set; behavior is test-covered but clients now seeSESSION_ERRORon reaped sockets instead of serving a closed env.Overview
Fixes a bug where
session_timeoutcould let the idle reaper destroy plain/wssessions whilereset/stepwere still running, because activity was only bumped after the handler returned. That could close the env mid-operation and free a concurrency slot while the WebSocket kept using the dead session.The server now maintains
_session_operations_in_flight(per-session ref counts, including attached second sockets) so the reaper skips busy sessions. Activity is refreshed when an operation starts and in afinallywhen it ends. If a message arrives after the session was reaped or shut down, the client getsSESSION_ERRORand the loop exits instead of running against a torn-down env.TestWebSocketSessionReapercovers long steps surviving a reaper pass, reaped sockets erroring and releasing capacity, failed steps still becoming idle, and a second/ws?session_id=socket during a slow step.Reviewed by Cursor Bugbot for commit e3f5d9b. Bugbot is set up for automated code reviews on this repo. Configure here.