Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Routes: `POST /api/auth/workspace-session`, `DELETE /api/auth/workspace-session`
- `F02.ws-legacy-key`: the deprecated `session_api_key` query parameter and `X-Session-API-Key` header still open a socket, and a redundant auth frame sent after them is ignored.
- `F02.ws-legacy-reject`: a wrong legacy key rejects the handshake itself (HTTP 403, no close frame), even when a correct first frame would follow.
- `F02.workspace-session-mint`: `POST /api/auth/workspace-session` answers 204 with an HttpOnly `oh_workspace_session_key` cookie on `Path=/api/conversations`, `SameSite=none; Secure; Partitioned` on a loopback host, and 401 with no cookie without a valid header (the cookie itself cannot mint another).
- `F02.workspace-session-plain-http`: over plain HTTP to a non-loopback host (no `X-Forwarded-Proto: https`) the workspace cookie is minted `SameSite=lax` without `Secure` or `Partitioned`, so browsers store it; the same host over HTTPS gets `SameSite=none; Secure; Partitioned`.
- `F02.workspace-cookie-access`: the minted cookie alone (a cookie jar, no header) loads workspace files, also after a restart; a bogus cookie or no credential gets 401.
- `F02.workspace-session-delete`: `DELETE /api/auth/workspace-session` answers 204 with a `Max-Age=0` cookie on the same path, which drops the cookie from the jar so workspace files are 401 again; 401 without a header.
- `F02.multiple-keys`: every configured key (`OH_SESSION_API_KEYS_0`, `_1`, ...) authenticates HTTP and WebSocket clients; an unlisted key does not.
Expand Down Expand Up @@ -518,6 +519,25 @@ Preconditions:
`http://[::1]:PORT`. `request.url.hostname` returns the IPv6 address
unbracketed (`::1`), so `_request_is_secure_context` strips a `:port`
suffix only from a bracketed host or a host with a single colon.
- **Plain HTTP to a non-loopback host (`F02.workspace-session-plain-http`).**
Mint as a browser at a LAN address over plain HTTP would (Canvas served by
a VM, a container or another machine), then as the same host behind a
TLS-terminating proxy.
```sh
control-agent-server api POST /api/auth/workspace-session --header 'Host: 10.244.8.163:8000' --expect 204 --save F02.workspace-session-plain-http/lan-http \
--check-header set-cookie contains 'SameSite=lax' --check-header set-cookie contains HttpOnly \
--check-header set-cookie not-contains Secure --check-header set-cookie not-contains Partitioned
control-agent-server api POST /api/auth/workspace-session --header 'Host: 10.244.8.163:8000' --header 'X-Forwarded-Proto: https' --expect 204 \
--save F02.workspace-session-plain-http/lan-https --check-header set-cookie contains 'SameSite=none' \
--check-header set-cookie contains Secure --check-header set-cookie contains Partitioned
```
Over plain HTTP to a non-loopback host the cookie is `SameSite=lax`,
without `Secure` or `Partitioned`: browsers refuse to store a
`SameSite=none` cookie that is not `Secure`, while a `Lax` cookie is stored
and sent on the same-site requests Canvas makes when it shares the
server's origin, so its file previews load. Cross-site embedding on plain
HTTP stays unsupported. The same host behind HTTPS (`X-Forwarded-Proto:
https`) gets `SameSite=none; Secure; Partitioned`, like loopback.

## Gotchas

Expand Down Expand Up @@ -558,3 +578,4 @@ Preconditions:
- With keys configured and no `--host`, the server binds `0.0.0.0`; without
keys, `127.0.0.1` (`F02.default-bind`). The launcher always passes
`--host 127.0.0.1`, so only `F02.default-bind` sees the default.
- Agent Canvas bundles Agent Server 1.62.0 by default, which mints the cookie `SameSite=none` on plain HTTP too, so Canvas file previews on a plain-HTTP non-loopback origin get 401 until Canvas pins a release that has `F02.workspace-session-plain-http` ([OpenHands/OpenHands#15567](https://github.com/OpenHands/OpenHands/issues/15567), [#15519](https://github.com/OpenHands/OpenHands/issues/15519)).
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Routes: `WS /sockets/session/{conversation_id}`
- `F07.sync-first`: the first server frame is `{"type": "sync"}`; `from_seq` echoes `after_seq` (absent when live-only) and `through_seq` is the last on-disk `seq` (absent for an empty log).
- `F07.sync-first-during-run`: while a run holds the conversation's state lock, `sync` is still the first frame; an `error` reply to an invalid frame sent right after connecting comes after it.
- `F07.snapshot-transient`: every connection gets one `transient` frame with a `full_state` `ConversationStateUpdateEvent` and no `seq`, after `sync` (and after the replay when one ran); it is never persisted.
- `F07.snapshot-compact-stats`: the `full_state` snapshot carries each LLM's aggregate stats (`accumulated_cost`, `accumulated_token_usage`) but not the per-call `costs`, `response_latencies` and `token_usages` lists, so it stays small however long the conversation runs.
- `F07.snapshot-compact-stats`: the `full_state` snapshot carries each LLM's aggregate stats (`accumulated_cost`, `accumulated_token_usage`) but not the per-call `costs`, `response_latencies` and `token_usages` lists, so it stays small however long the conversation runs; `GET /api/conversations/{id}` returns the same aggregates with empty per-call lists.
- `F07.full-replay`: `after_seq=-1` (or any value below it) replays every persisted event as `durable` frames with `seq` 0..`through_seq`, in order, once each, with the same ids REST returns.
- `F07.seq-on-disk`: a durable frame's `seq` is the event's on-disk index, the `NNNNN` in `events/event-NNNNN-<event id>.json`.
- `F07.resume-cursor`: `after_seq=k` replays exactly the events with `seq > k`; `after_seq=through_seq` replays nothing.
Expand Down Expand Up @@ -448,10 +448,10 @@ Preconditions:
```
The replay carries the durable reply (its id is `ITEM`) and no progress
frame of any kind.
- **Compact snapshot stats (`F07.snapshot-compact-stats`), known bug #6741.**
- **Compact snapshot stats (`F07.snapshot-compact-stats`).**
Reconnect live-only to the streamed conversation, which has made one
model call, and read its snapshot's stats for the agent's LLM (usage id
`default`).
`default`), then the same conversation's REST `ConversationInfo`.

Requires: `F07.progress-frames`
```sh
Expand All @@ -463,17 +463,22 @@ Preconditions:
--check frames.2.frame.event.value.stats.usage_to_metrics.default.accumulated_cost exists \
--check frames.2.frame.event.value.stats.usage_to_metrics.default.response_latencies missing \
--check frames.2.frame.event.value.stats.usage_to_metrics.default.token_usages missing \
--check frames.2.frame.event.value.stats.usage_to_metrics.default.costs missing # bug: checks.0.ok eq true && checks.1.actual contains latency
--check frames.2.frame.event.value.stats.usage_to_metrics.default.costs missing
control-agent-server api GET "/api/conversations/$SID" --expect 200 --save "F07.snapshot-compact-stats/info-$SID" \
--check stats.usage_to_metrics.default.accumulated_token_usage.prompt_tokens gt 0 \
--check stats.usage_to_metrics.default.costs len-eq 0 \
--check stats.usage_to_metrics.default.response_latencies len-eq 0 \
--check stats.usage_to_metrics.default.token_usages len-eq 0
```
Expected: the snapshot keeps the aggregates (prompt tokens above 0, an
`accumulated_cost`) and drops the per-call lists, as the incremental
`stats` updates do (`use_snapshot`). Today the `full_state` snapshot is
dumped without that context, so it carries one `costs`,
`response_latencies` and `token_usages` entry per model call (about
2.6 KB each); past roughly 1,550 calls the snapshot exceeds the 4 MiB
frame cap and every connection closes with 1013 `frame_too_large`
before the client sees anything (#6741). The legacy events socket's
snapshot (F06) is built the same way.
`stats` updates do (`use_snapshot`), so its size does not grow with the
number of model calls and a long conversation's snapshot stays under the
4 MiB frame cap. `GET /api/conversations/{id}` (and search) keeps the same
`stats` schema with the aggregates and empty per-call lists. The
conversation's `base_state.json` still keeps every per-call entry; only
the wire payloads are compact. The legacy events socket's snapshot (F06)
is built the same way.
- **Interrupted stream (`F07.item-aborted`).** Start a streamed
deepseek-flash answer that takes tens of seconds (counting to 3000), wait
until its `delta` frames flow on a background capture, then interrupt the
Expand Down Expand Up @@ -786,3 +791,4 @@ Preconditions:
POSTs above use `--quiet` to keep the map-run transcript small (`--expect`
and `--check` still see the bodies); `--max-chars` only trims non-JSON
response bodies.
- Agent Canvas bundles Agent Server 1.62.0 by default, whose `full_state` snapshot still carries the per-call lists, so a very long conversation keeps closing with 1013 `frame_too_large` in Canvas until it pins a release that has `F07.snapshot-compact-stats` ([OpenHands/OpenHands#18935](https://github.com/OpenHands/OpenHands/issues/18935)).
Original file line number Diff line number Diff line change
Expand Up @@ -556,7 +556,7 @@ SIDE_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key
A second `Condensation` is recorded and ask_agent still names the codename
from the reloaded history (the condenser keeps the first two events, the
system prompt and the user's message); `A` stays `finished`.
- **Ask usage on GET and after a restart (`F10.ask-agent-usage-saved`), known bug #6844.**
- **Ask usage on GET and after a restart (`F10.ask-agent-usage-saved`).**
A conversation of its own on an `llm-stub` with fixed usage numbers: the
run's reply reports 11 prompt and 7 completion tokens, the ask's reply 5
and 3. The bullet restarts the run, so it comes last.
Expand All @@ -575,7 +575,7 @@ SIDE_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key
control-agent-server api GET "/api/conversations/$U" --max-chars 300 \
--check stats.usage_to_metrics.default.accumulated_token_usage.completion_tokens eq 7 \
--check stats.usage_to_metrics.ask-agent-llm.accumulated_token_usage.completion_tokens eq 3 \
--save F10.ask-agent-usage-saved/get # bug: status eq 200 && checks.0.ok eq true && checks.1.actual eq null
--save F10.ask-agent-usage-saved/get
control-agent-server restart
control-agent-server api GET "/api/conversations/$U" --max-chars 300 \
--check stats.usage_to_metrics.default.accumulated_token_usage.completion_tokens eq 7 \
Expand All @@ -584,12 +584,9 @@ SIDE_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key
```
The run's usage is on GET (7 completion tokens under `default`), the ask
answers, and the `stats` event the ask recorded reports 5 and 3 tokens
under `ask-agent-llm`. Expected: GET shows the same `ask-agent-llm` entry
right away and after a restart
([#6844](https://github.com/OpenHands/software-agent-sdk/issues/6844), fix in [#6845](https://github.com/OpenHands/software-agent-sdk/pull/6845)).
Today GET still lists only `default`: the ask updates the live stats
without saving the conversation's state, GET reads the saved state, and a
restart right after the ask loses the entry.
under `ask-agent-llm`. GET shows the same `ask-agent-llm` entry right
away, next to `default`, and still after a restart: the ask saves its
usage with the conversation's state.

## Gotchas

Expand Down Expand Up @@ -629,11 +626,9 @@ SIDE_AGENT="{\"llm\": {\"model\": \"openai/qa-stub\", \"api_key\": \"qa-stub-key
Each call records and pushes one `stats` `ConversationStateUpdateEvent`
whose `usage_to_metrics` includes an `ask-agent-llm` entry
(`F10.ask-agent` checks the total count), so a client that counts events
around an ask sees one more. GET's `stats` does not show the
`ask-agent-llm` entry until the conversation's state is next saved (a
message, a run), and a restart before that loses it
(`F10.ask-agent-usage-saved`,
[#6844](https://github.com/OpenHands/software-agent-sdk/issues/6844)). It passes the agent's tools to
around an ask sees one more. The same entry is saved with the
conversation's state, so GET and a restarted server show it
(`F10.ask-agent-usage-saved`). It passes the agent's tools to
the model and returns the first text block, so on a conversation with no
messages yet DeepSeek's answer came back as an empty `response` (200).
- `fork` and `navigate` return `ConversationInfo` with
Expand Down
Loading
Loading