Repository navigation
Conversation
Each script checks the fence, the dedupe key, the close status and the format version before it writes, so a stale or copied batch never goes into the Store. The append script also keeps the task index and flags the first batch that withdraws.
A stale attempt could close a Stream, and a Local Activity of a failed task that wrote nothing here was not fenced. The scripts also refuse a sequence below 1, keep the trim watermark from moving back, and let a Workflow create take over a Stream that an Activity created.
WAIT held every Store call to the node behind it, so durable flushes get a connection of their own. A Stream whose newest id is near the 20-bit sequence limit now refuses a write before it goes in, and c with no attempt withdraws by started time.
Owner
Author
|
Closing as stale. This layer is folded into the v5 tip, #41, and the upstream review uses a fresh series instead. Plan: https://www.notion.so/4048fc567738814990b7c796ac28fe3f |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
This PR adds the Redis Store. One Lua script runs each create, append and close, and a read-only script reads the state and the log in one step. Before it writes, the append script checks the create key, the format version
v, the expiry, the dedupe key, the close status, the withdrawn-run fence, the epoch fence per (kind, id) and the withdrawn-task fence for Local Activities. It flags the first batch of a new epoch that withdraws, and it keeps the run's task index, the trim watermark and the TTLs. A close from a Producer other than the Workflow has the same epoch fence, and a Workflow create takes over a Stream that an Activity created first.docs/redis.mdhas the layout.Why?
A stale or copied batch must never go into the Store, and Redis doesn't roll a script back. So each script refuses before its first write, and the fence is atomic with the append. The flag is in the log, so a Reader keeps no state.
How did you test it?
crates/streams/scripts/redis-test-servers.sh, thenSTREAMS_REQUIRE=redis cargo test -p temporalio-streams, on standalone Redis and a 3-primary cluster. It covers resends, every fence, the closed and expired Streams, the format version, the TTLs, the Retention trim, and Appendix E examples 1 to 6 read through the reader rule.