Skip to content

[REA-6855] Serve saved step folders over HTTP - #239

Merged
Dere-Wah merged 3 commits into
mainfrom
dere/rea-6855-serve-step-folders-over-http
Oct 5, 2026
Merged

Dere-Wah merged 3 commits into
mainfrom
dere/rea-6855-serve-step-folders-over-http

Conversation

@Dere-Wah

@Dere-Wah Dere-Wah commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Why

The step folders are only useful if a caller can read them, and the caller may come for them after the session is over: a session that stopped at its steps has usually closed by the time its last folder is written.

What Changed

Three routes serve the folders by the session's own id:

GET /sessions/{id}/steps                {"steps": [1, 2, 3, 4]}
GET /sessions/{id}/steps/{n}            the step's result.json
GET /sessions/{id}/steps/{n}/{name}     a file result.json lists, with its content type

Only a step whose result.json is written is listed or served, so a reader never sees a step that is half saved. Only a name that result.json lists is served, so neither a file still being written nor a path that leaves the folder is reachable; result.json itself is read through the second route. A session that kept no folder, an id that is not a UUID, and a step that is not complete are all 404. The routes keep answering after the session ends, until each folder ages out.

Both file routes open the file inside the handler: result.json is read there, and other files are streamed from a handle opened there, with Content-Length from it. A folder that ages out between the lookup and the response answers 404 rather than failing in a deferred open, and one that ages out mid-transfer is still read whole from the open handle.

The OpenAPI document gains the three routes and the result.json schema, as additions only. The README now describes steps and step results, and the contract suite locks the surface end to end: a two-step session on a model whose process_output() returns a StepCompleted with an extra file, its step_result_ready facts, every route, the folders after the session ends, and the 404s.

Checked against examples/starter with step results on and steps: 3, with no client: four steps were saved (one reported after the stop), two of them finished after the session was back to ready, the list returned [1, 2, 3, 4], and output.mp4 decoded as an 800×500 H.264 frame at 30 fps.

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

[codex-review] - [P2] src/reactor_runtime/http/routes.py: retention cleanup can race FileResponse, producing a 500 instead of 404.

Scope: full (4a3e9e0..d7e5693).

View workflow run.

Comment thread src/reactor_runtime/http/routes.py Outdated

@tempusfrangit tempusfrangit left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with a nonblocking follow-up on the existing retention/read race: please keep an admitted file stable during delivery, or translate disappearance during response setup into a 404. Add a regression test that forces cleanup between path validation and response setup. The overall route design and contract coverage look sound.

@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-save-each-step-as-a-folder branch from 4a3e9e0 to f03b104 Compare October 4, 2026 16:39
@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-serve-step-folders-over-http branch from d7e5693 to 0b35fec Compare October 4, 2026 16:39
@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-save-each-step-as-a-folder branch from f03b104 to 2333bb0 Compare October 4, 2026 17:46
@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-serve-step-folders-over-http branch from 0b35fec to 1f3fcad Compare October 4, 2026 17:46
@Dere-Wah

Dere-Wah commented Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

@tempusfrangit done in 1f3fcad: both routes open the file inside the handler, so a folder reaped after its lookup answers 404 and one reaped mid-transfer is still sent whole. Regression tests delete the folder between validation and response, and during a transfer.

@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-save-each-step-as-a-folder branch from 2333bb0 to d976a28 Compare October 4, 2026 18:28
@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-serve-step-folders-over-http branch 2 times, most recently from 28a74c1 to d5345d2 Compare October 4, 2026 23:47
@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-save-each-step-as-a-folder branch from d976a28 to 2c56ea6 Compare October 4, 2026 23:47
@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-save-each-step-as-a-folder branch from 2c56ea6 to c0645cd Compare October 5, 2026 17:58
@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-serve-step-folders-over-http branch from d5345d2 to 2d978da Compare October 5, 2026 17:58

Dere-Wah commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Merge activity

  • Oct 5, 6:07 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Oct 5, 6:33 PM UTC: Graphite rebased this pull request as part of a merge.
  • Oct 5, 6:34 PM UTC: @Dere-Wah merged this pull request with Graphite.

@Dere-Wah
Dere-Wah changed the base branch from dere/rea-6855-save-each-step-as-a-folder to graphite-base/239 October 5, 2026 18:29
@Dere-Wah
Dere-Wah changed the base branch from graphite-base/239 to main October 5, 2026 18:31
Dere-Wah and others added 2 commits October 5, 2026 18:32
A caller collecting a session's step results reads them where the
runtime keeps them, by the session's own id, the one /clips/chunks uses:

    GET /sessions/{id}/steps                 {"steps": [1, 2, 3]}
    GET /sessions/{id}/steps/{n}             the step's result.json
    GET /sessions/{id}/steps/{n}/{name}      a file result.json lists

Only a step whose result.json is written is listed or served, so a
reader never sees a step that is half saved, and only a name that
result.json lists is served, so neither a file still being written nor a
path that leaves the folder is reachable. A session that kept no folder,
an id that is not a UUID, and a step that is not complete are all 404.
The routes keep answering after the session ends, until each folder ages
out, which is what lets the last step of a session that stopped at its
steps be collected.

The OpenAPI document gains the three routes and the result.json schema.
The README describes steps and step results.

Signed-off-by: Dere-Wah <derexcontact@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The reaper can delete a step's folder after the store has validated a
path but before a deferred FileResponse opens it, which surfaced as a
500 at the five-minute retention boundary instead of the documented 404.

Both routes now open the file inside the handler: result.json is read
there, and any other file is streamed from a handle opened there, with
its size taken from that handle. A folder that ages out after its
lookup answers 404, and one that ages out while it is being sent is
still read whole from the open handle. Contract tests delete the folder
between the lookup and the response, and during a transfer.

Signed-off-by: Dere-Wah <derexcontact@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The session paragraph now says the runtime ends a session once it has
reported its steps, so the step results paragraph keeps only what it
adds: reading the saved folders over HTTP.

Signed-off-by: Dere-Wah <derexcontact@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@Dere-Wah
Dere-Wah force-pushed the dere/rea-6855-serve-step-folders-over-http branch from 2d978da to 1f85ac6 Compare October 5, 2026 18:32
@Dere-Wah
Dere-Wah merged commit 0a1e19a into main Oct 5, 2026
10 checks passed
@Dere-Wah
Dere-Wah deleted the dere/rea-6855-serve-step-folders-over-http branch October 5, 2026 18:34
douglaseel added a commit that referenced this pull request Oct 5, 2026
…#240)

## Why

Releases what has landed since 3.6.1. It adds new public surface and moves nothing on the wire, which makes it a minor release:

- **Data-channel chunking** (#228–#230). `WebRtcConfig.dc_chunking` is on by default. The runtime answers clients that ask for chunked data channels, so a command or a model message of up to 64 MiB passes, instead of being capped at 256 KiB. Refused frames are logged once per channel, and a teardown sends what a chunked channel still has queued. A client that doesn't ask keeps the plain channel. 
- **Step sessions** (#231–#239): a session's starting input and step count on the start body, `complete_step()`, closing a session at its steps, the `runtime.step_results` manifest block, and step output saved as an MP4 or a folder of files, served over HTTP.
- **Client stats** (#201, #224): client-observed WebRTC stats decoded in the transport and journaled as metric self-loops.
- **reactor-webrtc 0.21.0** (from 0.19.1).
  - 0.21.0 keeps per-frame metadata on its own frame when libwebrtc drops a frame before render. Before, metadata was matched by queue order, so after a dropped frame every later frame got its predecessor's metadata.
  - It also keeps a `FrameTransform`'s `replace_data` through the metadata step.
  - 0.20.0 is additive.
  - Neither changes the Python API the runtime uses.
- Dependency bumps (#216–#220).

## What Changed

Two commits:
- `uv version 3.7.0`, which updates the version in `pyproject.toml` and `uv.lock`;
- `reactor-webrtc==0.21.0`, relocked. Only that package moves in `uv.lock`.

Merging tags `v3.7.0` and publishes.

Checked locally with a locked install of the published reactor-webrtc 0.21.0 wheel:
- lint, typecheck and the HTTP spec check pass;
- unit tests pass (1917);
- integration tests pass (12).

And with the release workflow's own gates:
- `mise run http-breaking-release`: "3.6.1 -> 3.7.0 (minor) satisfies the mandated patch";
- `mise run wire-check-release`: the pin matches `proto/`;
- `mise run build`: builds the 3.7.0 wheel and sdist.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants