[Bridge] Moved the stream runtime to sdk-streams. by moetemp · Pull Request #272 · moetemp/sdk-python

moetemp · 2026-10-17T09:55:18Z

What changed?

This PR moves Bridge's stream runtime to sdk-streams. It merges Jetty's jp28, pins Core to kb1 84dc65fd and takes the temporalio-streams crate at sb3 472e700f.

Nexus progress comes in the Workflow Task attachments as Any(NexusOperationProgress), keyed by endpoint, service, operation and token. The fold and the deliveries pass the crate's nexus_progress_folds.json.
The Nexus progress and sync call messages come from the crate's api_nexus copy. The system Nexus services come from the crate's WIT, which no longer exports the four notifier ops, so temporalio.workflow drops them.
StreamReader records each batch in History by default. replay_from_store=True is the opt-in for direct reads. A direct read cuts its Workflow Task like a follow step, and its recheck caps at 5 minutes.
The direct-read caches share the byte-bounded BatchCache. The Nexus follower's batch is FollowerBatch.
notify_on_append is on by default, so Bridge progress needs no setup.
The Log, the driver, the archive and PostgreSQL left with jp28's move-out. They're on the optional layers [FROZEN][Bridge][Optional] Restored the notification log, the stream driver and the archive. #273 and [FROZEN][Bridge][Optional] Restored the PostgreSQL store. #274. The Bridge demos left too, and the live tests keep their helpers.
Both Nexus generators pin nexgen 3c60ecc9.

Why?

sdk-rust holds nothing stream-specific now, so the Bridge pieces of the stream runtime and its API live in sdk-streams. Recorded batches work on every server, Temporal Cloud included.

How did you test it?

Link to a test plan if any -

Unit Tests
Staging
End to End Tests

Core k1 holds no stream code. The bridge links temporalio-streams from moetemp/sdk-streams, patched onto the submodule's sdk-rust crates so both see one Connection type, and the stream protos come from the crate checkout.

ss3 adds the Workflow publish, eviction and shutdown calls and the OutputRecord message that left Core's workflow commands.

The Worker runs the payload codec on the records and calls PublishWorkflowTask before it completes the activation, and a failed call fails the Workflow Task. Each eviction calls RunEvicted and Worker shutdown calls ShutdownWorker, keyed by a fresh worker id per Worker.

The patch hides the crate's own sdk-rust rev from the build. The FIPS tree check now counts only normal and build edges.

Each Worker keys its run state by its own worker id, so the stream keeps every record in order across the restarts and the chain still closes.

The store, the reader rules and the chain close now run in the temporalio-streams crate, and the Worker publishes the records.

ss3 now closes an ended run's chain at Worker shutdown too, and ss4 adds the Core mock Worker tests.

… it. ss3 now proves the chain close from the run's History, and ss2 adds the crate's fips feature.

Moved the Core pin to 969a0883 and the sdk-streams dependency to sj1 e86e63a5, and regenerated the protos.

Core now hands the server's attachments in job 17. The Worker decodes each wake's close outcome with the reader's codec, and the instance folds the wakes with the shared rule.

sj2 holds the notifier and owner clients that Core served before.

…inst its vectors.

…s in the publish call. The Worker knows an owner Activity by the header it set. It sends each live resolution as the server gave it, before the decode pass, so readers decode the failure with their own codec.

…checkout.

It restores the one-line ACL command in the crate's Redis docs.

kj5 vendors no stream API, so the stream messages, enums and notifier messages now generate from the crate's api_stream copy. The workflow service client lost the notifier RPCs.

…mNotifier.

The workflow service client has no notifier RPCs on kj5, since the stream runtime holds the notifier client.

The Core pin is kb1 and the sdk-streams dependency is sb1. The generated files come from them.

The stream service's Nexus path note in _body.py came back. The tests read the contract and the token vectors from the crate's nexus directory.

…rom the crate. The Nexus progress and sync call messages come from the crate's api_nexus copy. The system Nexus generator reads the workflow and notification service WIT from the crate.

The instance folds a task's progress to one per endpoint, service, operation and token, and gives it to each started operation with that key. The fold and the deliveries pass the crate's vectors.

Core's client has no stub for it, so ClientTransport calls the crate's call by method path on the client's connection.

The server caps the recheck at 5 minutes, not an hour.

The core Worker reads it, so it shouldn't import contrib.streams to get it.

The Core entries described a stream runtime that left Core. The Worker now publishes through PublishWorkflowTask, and temporalio-streams runs the store.

The owner's codec encoded it, so a codec that keys on the context failed the reader's task. No reader reads a wake's close result, so it stays as sent.

Demos don't ship in an SDK. The push test wakes through producer notifications, since the driver moved to the optional stack.

Only a read or a replay moves the hold, so a long run must Continue-as-New in time.

# Conflicts: # temporalio/bridge/Cargo.lock # temporalio/bridge/Cargo.toml # temporalio/bridge/proto/streams/streams_pb2.py # temporalio/bridge/sdk-core # temporalio/contrib/streams/README.md # temporalio/contrib/streams/__init__.py # temporalio/contrib/streams/_body.py # temporalio/contrib/streams/_capabilities.py # temporalio/contrib/streams/_cursor.py # temporalio/contrib/streams/_handles.py # temporalio/contrib/streams/_plugin.py # temporalio/contrib/streams/_supersession.py # temporalio/contrib/streams/_wire.py # temporalio/contrib/streams/_workflow.py # temporalio/contrib/streams/docs/migration.md # temporalio/contrib/streams/docs/store-capabilities.md # temporalio/contrib/streams/memory.py # temporalio/contrib/streams/redis.py # tests/contrib/streams/test_unsupported_owners.py

The new crate makes each append wait for its stream notification and drops the optional stack.

The crate's WIT no longer exports the four System Nexus notifier ops, so temporalio.workflow and the outbound interceptor lose them. The stream contract regenerates with nexgen 3c60ecc9.

With notifications on by default, the notifier wait found no owner and ended the read inside its owner start grace.

The Workflow reader ships in this stage, the Nexus notes belong to a later one, and the harness table and lane notes aren't user docs.

No released marker lacks the run that read it.

The entries described a stream runtime in Core and an attempt shift that changed since.

The crate script defaults to the shared cluster on 7101. The server ref pointed at an old commit.

# Conflicts: # scripts/streams_live_lane.sh

moetemp · 2026-10-18T07:12:49Z

Closing this in favor of the re-cut Bridge stack (#296 to #320). Its content is now in #296, #297.

moetemp added 30 commits October 16, 2026 23:04

Moved the sdk-streams dependency to ss3 and regenerated its protos.

e260ab2

ss3 adds the Workflow publish, eviction and shutdown calls and the OutputRecord message that left Core's workflow commands.

Read the crate's vectors and Redis docs from its Cargo checkout.

b79993b

Checked in CI that sdk-streams and sdk-core pin the same sdk-rust.

c1be611

The patch hides the crate's own sdk-rust rev from the build. The FIPS tree check now counts only normal and build edges.

Tested a Worker restart in one process against the stream.

417e6e2

Each Worker keys its run state by its own worker id, so the stream keeps every record in order across the restarts and the chain still closes.

Named the stream runtime, not Core, in the stream docs.

faeb495

The store, the reader rules and the chain close now run in the temporalio-streams crate, and the Worker publishes the records.

Moved the sdk-streams dependency to ss4.

1506d0c

ss3 now closes an ended run's chain at Worker shutdown too, and ss4 adds the Core mock Worker tests.

Moved the sdk-streams dependency to the new ss4 and forwarded fips to…

a67f91f

… it. ss3 now proves the chain close from the run's History, and ss2 adds the crate's fips feature.

Merged h10-sdk-streams into jp28-sdk-streams.

9fa1a80

Moved the Core pin to 969a0883 and the sdk-streams dependency to sj1 e86e63a5, and regenerated the protos.

Read stream wakes from the Workflow Task attachments.

0a92773

Core now hands the server's attachments in job 17. The Worker decodes each wake's close outcome with the reader's codec, and the instance folds the wakes with the shared rule.

Tested that the reader's codec decodes a wake's close outcome.

06ca110

Moved the sdk-streams dependency to sj2.

9374ae9

sj2 holds the notifier and owner clients that Core served before.

Moved the sdk-streams dependency to sj3 and checked the wake fold aga…

ad576bb

…inst its vectors.

Moved the sdk-streams dependency to sj4 and passed owner Activity end…

8c7d61d

…s in the publish call. The Worker knows an owner Activity by the header it set. It sends each live resolution as the server gave it, before the decode pass, so readers decode the failure with their own codec.

Moved the sdk-streams dependency to sj5.

ef4444d

Read the Activity attempt vectors and PostgreSQL docs from the crate …

acae0ca

…checkout.

Took the live lane's Redis cluster script from the crate checkout.

ada6525

Named the stream runtime, not Core, in Jetty's stream docs.

ccaa19e

Moved the sdk-streams dependency to the new sj5.

a2df034

It restores the one-line ACL command in the crate's Redis docs.

Moved the Core pin to kj5 and the sdk-streams dependency to its sj5.

dacb539

kj5 vendors no stream API, so the stream messages, enums and notifier messages now generate from the crate's api_stream copy. The workflow service client lost the notifier RPCs.

Moved the sdk-streams dependency to the sj5 that serves DescribeStrea…

25110aa

…mNotifier.

Described stream notifiers through the stream service in the tests.

22c802f

The workflow service client has no notifier RPCs on kj5, since the stream runtime holds the notifier client.

Merged jp28-sdk-streams into dr40-sdk-streams.

41cc87b

The Core pin is kb1 and the sdk-streams dependency is sb1. The generated files come from them.

Restored the Bridge parts of two files that the merge took from jp28.

f36ad3d

The stream service's Nexus path note in _body.py came back. The tests read the contract and the token vectors from the crate's nexus directory.

Restored the Nexus body helpers that the merge took out of _body.py.

8d344c4

Moved the sdk-streams dependency to sb3 and generated the Nexus API f…

9b8b08d

…rom the crate. The Nexus progress and sync call messages come from the crate's api_nexus copy. The system Nexus generator reads the workflow and notification service WIT from the crate.

Took Nexus operation progress from the Workflow Task attachments.

dcf6e57

The instance folds a task's progress to one per endpoint, service, operation and token, and gives it to each started operation with that key. The fold and the deliveries pass the crate's vectors.

Called ExecuteSyncNexusOperation through the stream runtime.

43044cc

Core's client has no stub for it, so ClientTransport calls the crate's call by method path on the client's connection.

Described stream notifiers by method path in the Nexus tests.

62aa7d9

moetemp added 28 commits October 17, 2026 22:33

Said that the notifier rechecks a Workflow reader every 5 minutes.

959ade7

The server caps the recheck at 5 minutes, not an hour.

Defined the stream owner header in the bridge streams module.

caa7bb8

The core Worker reads it, so it shouldn't import contrib.streams to get it.

Rewrote the stream changelog for the Worker publish and the crate.

d82a795

The Core entries described a stream runtime that left Core. The Worker now publishes through PublishWorkflowTask, and temporalio-streams runs the store.

Merged the jp28 move-out and notify default into dr40.

6da1cd5

Decoded a wake's close failure in the stream owner's context.

6476e75

The owner's codec encoded it, so a codec that keys on the context failed the reader's task. No reader reads a wake's close result, so it stays as sent.

Shared one task cutoff check between the Workflow reader and follower.

9b80d6d

Removed the Bridge demos and kept the live tests' helpers.

d7a8e71

Demos don't ship in an SDK. The push test wakes through producer notifications, since the driver moved to the optional stack.

Stated the floor_lease rule in the workflow_reader docs.

e940003

Only a read or a replay moves the hold, so a long run must Continue-as-New in time.

Pinned sdk-streams sj5 9250d6c7 and sdk-rust kj5 869a190a.

d61c662

The new crate makes each append wait for its stream notification and drops the optional stack.

Pinned sb3 and kb1, and dropped the notifier ops from the bindings.

9f00163

The crate's WIT no longer exports the four System Nexus notifier ops, so temporalio.workflow and the outbound interceptor lose them. The stream contract regenerates with nexgen 3c60ecc9.

Kept a sleeping client read alive while its owner hasn't started.

a1ecf08

With notifications on by default, the notifier wait found no owner and ended the read inside its owner start grace.

Cut the Nexus direct reader's Workflow Task like a follow step's.

5c14d4d

Said that Bridge progress is on by default with notify_on_append.

d0e74b8

Dropped stale stream docs and moved the live test notes to the tests.

4b66a37

The Workflow reader ships in this stage, the Nexus notes belong to a later one, and the harness table and lane notes aren't user docs.

Dropped the read marker fallback and an internal term from the Worker.

87e288e

No released marker lacks the run that read it.

Patched the workflow module itself in the direct read cut test.

612530f

Named the stream runtime, not Core, in the Jetty stream code and tests.

3c42015

Rewrote the Jetty stream changelog for the crate and the Worker.

8421590

The entries described a stream runtime in Core and an attempt shift that changed since.

Dropped the store kind the lost-notification tests no longer vary.

c8fe94a

Merged jp28-sdk-streams into dr40-sdk-streams.

8a270b5

Fixed the bridge client generator that the jp28 merge broke.

8b8593d

Called the shared task cutoff from the Nexus direct reader.

8cd80b4

Left the task cutoff out of a reader test that runs no Workflow.

cb2e271

Gated the renamed direct reader case in the live lane.

7339883

Moved the live lane to its own Redis ports and the js14 server.

fcec105

The crate script defaults to the shared cluster on 7101. The server ref pointed at an old commit.

Merge commit 'fcec105fecd42ce0b2e3a4576d0fa2966639dbd0' into HEAD

cd5870c

# Conflicts: # scripts/streams_live_lane.sh

Named the dr40 branch in the lane script.

d35ac14

moetemp closed this Oct 18, 2026

Navigation Menu

Repository navigation

[Bridge] Moved the stream runtime to sdk-streams. - #272

[Bridge] Moved the stream runtime to sdk-streams.#272
moetemp wants to merge 88 commits into
moe/bridge/v2/dr39-follow-lanefrom
moe/bridge/v2/dr40-sdk-streams

moetemp commented Oct 17, 2026 •

edited

Loading

Uh oh!

moetemp commented Oct 18, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

moetemp commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed?

Why?

How did you test it?

Uh oh!

moetemp commented Oct 18, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

moetemp commented Oct 17, 2026 •

edited

Loading