Skip to content

feat: add Brook as a second event stream beside Workflow Streams - #8

Open
moetemp wants to merge 4 commits into
mainfrom
moe/brook/review/h1-streams
Open

moetemp wants to merge 4 commits into
mainfrom
moe/brook/review/h1-streams

Conversation

@moetemp

@moetemp moetemp commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

What changed?

This PR adds Brook, the generated sdk-python Streams API, as a second event stream beside Workflow Streams.

  • Each agent picks its event stream with the required stream=. ContribStream() is Brook, and WorkflowStream() is Workflow Streams. The examples pass WorkflowStream().
  • STREAMS_STORE=redis or memory picks the Store.
  • One Stream holds the events of the whole agent tree, and a reader filters by agent_id. Its id comes from the Workflow id and the first run id, so a retry or a continue-as-new keeps the same Stream.
  • Workflow code writes with the SDK's workflow.create_stream and append, which the SDK runs in a Local Activity that retries with no limit. A Store outage pauses the run and loses no event.
  • Clients resume from an opaque resume point. Upgrade the workers first.
  • The harness never closes the Stream, so a reset of a finished agent still works.

The justfile pins temporalio/sdk-python fc6b8d3b, the head of temporalio/sdk-python#2129, until an sdk-python release has Streams. Part of the Brook review series.

Why?

The harness is the first user of the Streams API. Agent events go to a Store such as Redis instead of the Workflow's history, and a client reads the whole agent tree in one read.

How did you test it?

The Workflow Streams lane, the Brook memory and Redis lanes, the Nexus adapter, and the Go and TypeScript tests. The failures also happen on main, except test_attach_to_an_agent_terminated_mid_turn_ends, which fails on the Workflow Streams lane with temporalio 1.35.0 and also failed before this rebase.

The main missing element is the idle-timeout close on Brook. Its agent_closed event doesn't reach the reader yet. That fix and Brook tests for the newer main features come in the next commits.

Link to a test plan if any -

  • Unit Tests
  • Staging
  • End to End Tests

@moetemp
moetemp force-pushed the moe/brook/review/h1-streams branch 2 times, most recently from 5858443 to e241a68 Compare October 25, 2026 23:46
@moetemp
moetemp changed the base branch from moe/brook/review/base to moe/brook/review/h0-nexus-wait-for-stage October 25, 2026 23:47
@moetemp
moetemp force-pushed the moe/brook/review/h1-streams branch 8 times, most recently from 0aea335 to b1ca4c3 Compare October 28, 2026 08:22
@moetemp moetemp changed the title [Brook H1] Ported the harness to Streams in temporalio.common. [Brook H1] Ported the harness to Streams in temporalio.streams. Nov 5, 2026
@moetemp moetemp changed the title [Brook H1] Ported the harness to Streams in temporalio.streams. [Brook H1] Ported the harness to the generated Streams API. Nov 13, 2026
Each agent picks its event stream with the required `stream=`. `ContribStream()` writes the agent
tree to one Stream through the generated Streams API, and `WorkflowStream()` keeps Workflow Streams.
@moetemp
moetemp force-pushed the moe/brook/review/h1-streams branch from 54f520a to b3e56ae Compare November 13, 2026 20:36
@moetemp
moetemp changed the base branch from moe/brook/review/h0-nexus-wait-for-stage to main November 13, 2026 20:36
@moetemp moetemp changed the title [Brook H1] Ported the harness to the generated Streams API. feat: add Brook as a second event stream beside Workflow Streams Nov 14, 2026
The pinned SDK build has `append` for Workflow code again, so the two Local Activities of the
harness are gone. The Store helper gives a `StreamConfig` whose Workflow calls retry until the
Store is back.
The recorded session had markers of the removed Local Activities, so its replay failed. The tests
pass `stream_config=` and check the SDK's error for a Worker with no Store.
A user registers nothing for Streams except the Stream config on the client.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant