Skip to content

Fix idle reaper closing /ws sessions mid-step - #1618

Merged
burtenshaw merged 3 commits into
openenv:mainfrom
acharyaanusha:fix/ws-idle-reaper
Nov 13, 2026
Merged

burtenshaw merged 3 commits into
openenv:mainfrom
acharyaanusha:fix/ws-idle-reaper

Conversation

@acharyaanusha

@acharyaanusha acharyaanusha commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

With session_timeout set, the idle reaper can close a plain /ws session while its reset/step is still running: the handler stamps activity only after the call returns. Reaping also frees the session's slot while the socket keeps serving the closed env, so more envs can run than max_concurrent_envs allows. Steps that wait on slow external work (LLM calls, simulators) hit this whenever a step outlasts the timeout. Sessions attached through ?session_id= and /harness are already protected by _session_websocket_attachments; plain /ws sessions are not.

This PR tracks operations in flight per session so the reaper skips them, stamps activity when an operation starts and ends, and stops serving a socket whose session no longer exists (SESSION_ERROR, then close).

Found while building parley-sim, which includes a standalone repro.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • New environment
  • Refactoring

Alignment Checklist

  • I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
  • I have checked .claude/docs/INVARIANTS.md and no invariants are violated
  • I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

  • Not required (bug fix, docs, minor refactoring)
  • RFC exists: #___
  • RFC needed (will create before merge)

Test Plan

Repro: an env whose step_async sleeps 9 s, with max_concurrent_envs=1, session_timeout=2.0; reset, wait 1.5 s, step, then open a second session.

main this PR
env closed mid-step yes no
second session admitted over the cap of 1 yes no (CAPACITY_REACHED)

New TestWebSocketSessionReaper in tests/core/test_production_mode_routes.py:

  • a step spanning a reaper check is not closed and its result is delivered (fails on main);
  • a reaped socket gets SESSION_ERROR and is closed, and a new session fits under the cap (fails on main);
  • an idle session with nothing in flight is still reaped (regression guard).

Rebased on main at ca6df26. tests/core/test_production_mode_routes.py: 72 passed, 1 skipped. The CI test command: 6171 passed, 16 failed; the same 16 fail on main in my sandbox (TextArena can't download NLTK data through my proxy, and four Jupyter sandbox tests). bash .claude/hooks/lint.sh passes.

Out of scope: the HTTP /mcp path has the same stamp-after-call pattern for long tools/call requests; a follow-up could cover it with a per-session counter.

Claude Code Review

Alignment Review Report

Automated Checks

  • Lint: PASS. bash .claude/hooks/lint.sh passes. (ruff format --check also flags src/openenv/core/harness/README.md, which this PR doesn't touch; it is flagged on main too.)
  • Debug code: CLEAN. check-debug.sh finds nothing in the changed lines; its hits are existing docstring examples in other files.

Open RFCs Context

  • RFC 002 (In Review), lifecycle section: automatic idle timeouts "can drop a live transport mid-episode". That covers provider-level timeouts; this PR stops the server's own reaper from doing the same mid-step, which is consistent with it.
  • RFC 005 (In Review): /harness turns are bounded by HarnessConfig.session_timeout_s and are already exempt from reaping through _session_websocket_attachments. Unchanged here.

Tier 1: Fixes Required

None.

Tier 2: Alignment Discussion

Principle Conflicts

None identified. Gym signatures, wire types and the agent/infrastructure boundary are unchanged. No new message types: the reaped-socket case reuses WSErrorCode.SESSION_ERROR.

RFC Conflicts

None identified. The change is in src/openenv/core/ but adds no API.

One behaviour change for reviewers: a /ws socket whose session was destroyed (by the reaper or shutdown) now gets SESSION_ERROR and is closed. Before, it kept serving the closed env outside the session cap.

Summary

  • 0 mechanical issues to fix
  • 1 behaviour change for human review (above)
  • 0 RFC conflicts to discuss

Note

Medium Risk
Changes session lifecycle and concurrency accounting for all /ws traffic when session_timeout is set; behavior is test-covered but clients now see SESSION_ERROR on reaped sockets instead of serving a closed env.

Overview
Fixes a bug where session_timeout could let the idle reaper destroy plain /ws sessions while reset/step were still running, because activity was only bumped after the handler returned. That could close the env mid-operation and free a concurrency slot while the WebSocket kept using the dead session.

The server now maintains _session_operations_in_flight (per-session ref counts, including attached second sockets) so the reaper skips busy sessions. Activity is refreshed when an operation starts and in a finally when it ends. If a message arrives after the session was reaped or shut down, the client gets SESSION_ERROR and the loop exits instead of running against a torn-down env.

TestWebSocketSessionReaper covers long steps surviving a reaper pass, reaped sockets erroring and releasing capacity, failed steps still becoming idle, and a second /ws?session_id= socket during a slow step.

Reviewed by Cursor Bugbot for commit e3f5d9b. Bugbot is set up for automated code reviews on this repo. Configure here.

acharyaanusha and others added 2 commits November 12, 2026 17:56
The /ws handler stamped activity only after reset/step returned, so the
idle reaper could close a session's env while a step was still running.
Reaping also freed the session's slot while its socket kept serving the
closed env, letting more envs run than max_concurrent_envs.

Track operations in flight per session and skip them in the reaper; stamp
activity when an operation starts and ends; stop serving (SESSION_ERROR,
close) when a socket's session no longer exists.
A /ws client can attach a second socket to its own session with
?session_id=. When that socket's message finished, discarding the
session from the in-flight set let the reaper close the env under the
first socket's step. Keep a count per session instead, and add a
regression test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@burtenshaw burtenshaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this fixes the /ws case. I checked the parts that matter for a change like this:

  • The in-flight flag is set right before the try and released in its finally. Exceptions, a failed send, close, cancellation and a disconnect mid-step all release it. I checked cancellation with an async step and with a sync step running in the executor.
  • None of the _session_lock blocks awaits. So the reaper's re-check and the session pop run without yielding, and so do the handler's session check and the flag set. The two cannot interleave.
  • Idle sessions are still reaped, and a reaped socket can no longer run steps beyond max_concurrent_envs.
  • SESSION_ERROR is already part of the WebSocket protocol, so clients see no new message type.

One gap, fixed in c477f92. The flag was a set, but two sockets can share a session. A /ws client can learn its session ID with openenv/session/create and attach a second socket with ?session_id=. When the second socket's message finished, it removed the session from the set while the first socket's step was still running. Once the second socket disconnected, the reaper closed the env mid-step. The commit counts operations per session instead and adds test_second_socket_on_session_keeps_step_in_flight, which fails without the change. The test file passes: 73 passed, 1 skipped.

Two follow-ups outside this PR:

  1. The /mcp WebSocket route has the same bug on main. Activity is stamped only after mcp_handler returns, so a tools/call that outlasts session_timeout has its env closed mid-call, and the socket keeps serving the closed env. The same count around the mcp_handler call would fix it:

    Sketch for the /mcp WebSocket loop
    raw_message = await websocket.receive_text()
    if session_id not in self._sessions:
        error_resp = JsonRpcResponse.error_response(
            JsonRpcErrorCode.SERVER_ERROR,
            "Session expired or was closed; open a new connection",
        )
        await websocket.send_text(error_resp.model_dump_json())
        break
    # ... parse the request as today ...
    in_flight = self._session_operations_in_flight
    in_flight[session_id] = in_flight.get(session_id, 0) + 1
    try:
        response = await mcp_handler(
            jsonrpc_request, session_env=session_env, session_id=session_id
        )
        await websocket.send_text(response.model_dump_json())
    except Exception as e:
        ...  # unchanged
    finally:
        in_flight[session_id] -= 1
        if in_flight[session_id] == 0:
            del in_flight[session_id]
  2. After a reap, EnvClient reports the SESSION_ERROR frame as a generic RuntimeError. A retried step() then raises SessionLostError, and a retried reset() reconnects. On main both calls ran on the closed env, so this is an improvement. But a training loop that already catches SessionLostError will not catch the first error. Treating a SESSION_ERROR frame as a lost session in the client would make the two consistent. session_timeout is off by default, so this does not block the PR.

I'll merge once CI passes on the new head.

@burtenshaw
burtenshaw merged commit ad63a0b into openenv:main Nov 13, 2026
15 checks passed
@acharyaanusha

Copy link
Copy Markdown
Contributor Author

@burtenshaw Thanks for the fix and the review. Happy to take both follow-ups: the /mcp WebSocket fix along the lines of your sketch, and having EnvClient raise SessionLostError on a SESSION_ERROR frame. Should they be one PR or two?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants