Skip to content

[Bug]: full_state WebSocket snapshot omits use_snapshot=True, breaking conversations >~1,550 LLM calls with frame_too_large #6741

Closed

Description

@podon

Summary

Long-running conversations (~1,550+ LLM calls) become permanently unreachable over /sockets/session/{id}, leaving the frontend stuck on "Unable to connect to server" in a tight reconnect loop.

PR #1308 updated ConversationStateUpdateEvent.validate_value to serialize incremental ConversationStats updates with context={"use_snapshot": True} so per-call costs, response_latencies, and token_usages lists are omitted over WebSockets. However, ConversationStateUpdateEvent.from_conversation_state (key="full_state") still calls state.model_dump(mode="json", exclude_none=True) without context={"use_snapshot": True}.

When a client connects or reconnects to /sockets/session/{id}, EventService.subscribe_to_events immediately pushes the full_state snapshot event. Once stats.usage_to_metrics accumulates enough per-call entries (~2.6 KB per LLM call), the serialized TransientFrame exceeds MAX_FRAME_BYTES (4 MiB) and _SessionWebSocketWriter.send closes the socket with reason=frame_too_large.

Affected versions

  • openhands-sdk / openhands-agent-server 1.53.0 – 1.61.0 (current main)
  • Agent Canvas 1.26.0 – 1.38.0

Reproduction / Observed behavior

In a conversation with ~1,706 LLM calls:

  • base_state.json is 4,803,779 bytes, of which stats.usage_to_metrics is 4,604,907 bytes (response_latencies: 2.16 MB, token_usages: 2.41 MB; the aggregate totals in accumulated_token_usage are <300 bytes).
  • Every WebSocket connection to /sockets/session/{id} upgrades (101) and is immediately closed by the server before any frame reaches the client:
INFO     openhands.agent_server.session_socket:session_socket.py:335 session_socket_connected: <id> after_seq=6190
WARNING  openhands.agent_server.session_socket:session_socket.py:105 session_socket_frame_too_large: 4663235 bytes > 4194304
INFO     openhands.agent_server.session_socket:session_socket.py:425 session_socket_closed: <id> reason=frame_too_large
  • The client receives an abnormal close, displays "Unable to connect to server", and reconnects every ~1.5s, hitting the exact same 4.66 MB full_state frame on every attempt.
  • Additionally, conversation_service._compose_conversation_info (openhands-agent-server/openhands/agent_server/conversation_service.py) dumps **state.model_dump(mode="json") into ConversationInfo, causing GET /api/conversations/{id} and GET /api/conversations/search to return multi-megabyte JSON responses for the same reason.

Root cause

  1. openhands-sdk/openhands/sdk/event/conversation_state.py:

    @classmethod
    def from_conversation_state(
        cls, state: "ConversationState"
    ) -> "ConversationStateUpdateEvent":
        state_snapshot = state.model_dump(mode="json", exclude_none=True)
        return cls(key=FULL_STATE_KEY, value=state_snapshot)

    Because context={"use_snapshot": True} is not passed to state.model_dump(...), ConversationStats._serialize_with_context (openhands-sdk/openhands/sdk/conversation/conversation_stats.py) preserves the full costs, response_latencies, and token_usages lists instead of converting each Metrics to MetricsSnapshot.

  2. openhands-agent-server/openhands/agent_server/conversation_service.py:
    _compose_conversation_info calls state.model_dump(mode="json") without context={"use_snapshot": True}, embedding the full per-call lists in REST ConversationInfo responses as well.

Proposed fix

Pass context={"use_snapshot": True} when serializing ConversationState for full_state events (and for ConversationInfo REST responses, if per-call lists are not needed by REST callers):

state_snapshot = state.model_dump(
    mode="json",
    exclude_none=True,
    context={"use_snapshot": True},
)

With context={"use_snapshot": True}, the serialized full_state snapshot for the 1,706-call conversation above drops from 4,803,779 bytes to 200,167 bytes (~195 KB) while keeping accumulated_cost and accumulated_token_usage intact.

Related issues / PRs


OpenHands AI triage

The following comments and acceptance criteria were added by the OpenHands AI agent.

Triage

Verified against main: ConversationStateUpdateEvent.from_conversation_state (openhands-sdk/openhands/sdk/event/conversation_state.py) serializes the full_state snapshot without context={"use_snapshot": True}, while the incremental per-field path already uses snapshot serialization (PR #1308). EventService.subscribe_to_events pushes this snapshot on every connect to /sockets/session/{id}, and _SessionWebSocketWriter enforces MAX_FRAME_BYTES (4 MiB), closing with code 1013 / reason=frame_too_large — so the reconnect loop mechanism in this report is confirmed. _compose_conversation_info (openhands-agent-server/openhands/agent_server/conversation_service.py) embeds **state.model_dump(mode="json") into ConversationInfo.stats, so REST responses carry the same per-call lists. With use_snapshot=True, ConversationStats._serialize_with_context converts each Metrics to MetricsSnapshot (model_name, accumulated_cost, max_budget_per_task, accumulated_token_usage), matching the reported ~4.8 MB → ~200 KB reduction while keeping aggregates.

The one open design question in the report — whether REST callers need per-call lists — is resolved by repository evidence: the canonical TypeScript client already types ConversationStats.usage_to_metrics as Record<string, MetricsSnapshot> (clients/typescript/src/types/base.ts), with no references to per-call costs, response_latencies, or token_usages; per-call history stays available through the durable event stream and in persisted state. Snapshot-form data also deserializes cleanly into the SDK's RemoteState.stats consumer. Both serialization call sites should reuse the existing use_snapshot context mechanism rather than introducing a new one.

Non-goals: raising MAX_FRAME_BYTES or adding frame chunking/truncation; changing the incremental per-field ConversationStats WebSocket updates (already correct since #1308); changing the base_state.json persistence format (owned by #6432); changes to clients/typescript/; removing per-call metrics history availability from persistence or the event stream.

Acceptance Criteria

  • With a ConversationState whose per-call metrics (stats.usage_to_metrics costs, response_latencies, token_usages) serialize to more than MAX_FRAME_BYTES today, the full_state ConversationStateUpdateEvent produced by from_conversation_state serializes to a JSON payload under MAX_FRAME_BYTES, and a WebSocket connect to /sockets/session/{id} delivers it without the server closing with reason=frame_too_large.
  • In that full_state snapshot, each stats.usage_to_metrics entry contains the MetricsSnapshot fields (model_name, accumulated_cost, max_budget_per_task, accumulated_token_usage) with accumulated totals equal to the live ConversationStats aggregates, and contains no per-call costs, response_latencies, or token_usages lists.
  • GET /api/conversations/{id} and GET /api/conversations/search responses no longer include per-call costs, response_latencies, or token_usages inside stats.usage_to_metrics, keep accumulated_cost and accumulated_token_usage present and correct, and their body size no longer grows with per-call history length.
  • Incremental per-field ConversationStats state updates over WebSocket remain snapshot-serialized with unchanged behavior (existing PR Fix ConversationStateUpdateEvent to use MetricsSnapshot instead of full Metrics #1308 behavior preserved).
  • Server-side persistence is unchanged: base_state.json still stores the full per-call metrics history, the event stream still exposes per-call history, and resume continues to preserve accumulated and per-call stats.
  • The existing secret non-leak property for full_state serialization continues to hold: secrets (e.g. from mcp_config) do not appear in the serialized full_state payload (as covered by tests/sdk/conversation/test_mcp_secrets_serialization_leak.py).
  • Remote clients keep working: the SDK remote conversation deserializes the snapshot-form stats so conversation.stats exposes correct aggregates, run-completion signaling via full_state execution_status is unchanged, and clients/typescript/ requires no contract change (usage_to_metrics already typed as Record<string, MetricsSnapshot>).
  • Automated validation covers both surfaces: a unit test that from_conversation_state emits snapshot-form stats, and a live-server test in tests/cross/test_remote_conversation_live_server.py that connects a WebSocket client to a conversation whose per-call metrics exceed the old 4 MiB threshold and completes the session without a frame_too_large close.

Reproduction on main (91e8b5b)

I ran Agent Canvas (main 46f20abaf) against an agent-server built from software-agent-sdk main 91e8b5b. I opened a conversation whose base_state.json holds 8,700 synthetic per-call metrics entries (4.6 MB, the same as about 8,700 real LLM calls). In ~35 s the agent-server closed 30 connections with session_socket_frame_too_large: 4361785 bytes > 4194304. The page opened 17 /sockets/session connections and stayed on "Reconnecting…". GET /api/conversations/{id} returned 4.36 MB. A live-server test that builds the same state fails on main with ConnectionClosedError: received 1013 (try again later) frame_too_large. Fix: #6823.

repro on main

This comment/PR was created by an AI agent (OpenHands) on behalf of Graham Neubig.

Activity

  1. changed the title [-]bug(sdk, agent-server): full_state WebSocket snapshot omits use_snapshot=True, breaking conversations >~1,550 LLM calls with frame_too_large[/-] [+][Bug]: full_state WebSocket snapshot omits use_snapshot=True, breaking conversations >~1,550 LLM calls with frame_too_large[/+] on Nov 11, 2026
  2. added
    priority:mediumFor bugs, a serious source of annoyance, but not blocking a large number of users.
    ready-for-devIssue meets development readiness criteria
    on Nov 11, 2026
  3. its-amann commented on Nov 11, 2026

    @its-amann

    i can take this. plan is to pass context={"use_snapshot": True} in from_conversation_state and in _compose_conversation_info, plus a unit test for the snapshot stats and a live server test with per call metrics over the 4 MiB limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:mediumFor bugs, a serious source of annoyance, but not blocking a large number of users.ready-for-devIssue meets development readiness criteria

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions