Batch example#175
Open
ddieruf wants to merge 5 commits into
Open
Conversation
Processes a batch by having a single Activity heartbeat its progress, so a worker restart resumes from the last recorded offset instead of starting over. Mirrors samples-java/.../batch/heartbeatingactivity.
Processes a page of records at a time via child workflows run in parallel, continuing-as-new between pages so a dataset of any size can be processed. Mirrors samples-java/.../batch/iterator.
Keeps a fixed number of record processing child workflows running in parallel at all times, using completion Signals and continue-as-new to keep history bounded indefinitely. Multiple partitions run side by side via BatchWorkflow for higher total throughput. Continues-as-new on Workflow.ContinueAsNewSuggested in addition to a fixed page size, matching StreamPay's ProcessCustomerDataWorkflow, since a fixed count alone can be outgrown by other history (e.g. completion Signals) before it's reached. A TestContinueAsNew input flag makes that branch deterministically testable. Mirrors samples-java/.../batch/slidingwindow.
Adds the Batch solution folder and its three project references to TemporalioSamples.sln and their test-project references to TemporalioSamples.Tests.csproj, lists Batch in the root README's sample index, and adds the overview README for src/Batch.
This was referenced Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
Added a
.NETport of the Javabatchsamples (samples-java/core/src/main/java/io/temporal/samples/batch), under a newsrc/Batchdirectory with one project per pattern:BatchWorkflowfor higher total throughput.Each sample follows this repo's existing conventions (
dotnet run worker/dotnet run workflow, per-sample README,TemporalioSamples.slnentries) and mirrors the Java sample's file/class structure closely so it's easy to compare side by side.One deliberate deviation from the Java version:
SlidingWindowBatchWorkflowcontinues-as-new onWorkflow.ContinueAsNewSuggestedin addition to a fixed page size, rather than relying on the fixed count alone. A fixed count can be outgrown by other accumulated history (e.g. completion Signals) before it's reached — this pattern is based on how we handle the same problem in production. ATestContinueAsNewinput flag makes that branch deterministically testable without generating a large amount of real history.Why?
There was no .NET equivalent of these samples, which came up in a community discussion about batch-processing workarounds (paging through large datasets, bounding history size, safely reporting child-Workflow completion across continue-as-new). This fills that gap using patterns we've validated in a production import pipeline.
Checklist
Closes
How was this tested:
dotnet build/dotnet format --verify-no-changeson the full solution.tests/Batch/*(xUnit, usingWorkflowEnvironment.StartTimeSkippingAsyncfor the Iterator and SlidingWindow Workflow tests, andActivityEnvironmentfor a targeted heartbeat-resume test on HeartbeatingActivity). All pass.temporalCLI. This caught and fixed a real bug during development:SlidingWindowBatchWorkflow's original return value summed to double the actual record count across partitions; it now returns the Signal-trackedprogresscount instead.Any docs updates needed?
src/Batch/README.md(overview) plus a README per sample.Batchentry to the rootREADME.mdsample index.