Skip to content

Batch example#175

Open
ddieruf wants to merge 5 commits into
temporalio:mainfrom
kisasa:batch-example
Open

Batch example#175
ddieruf wants to merge 5 commits into
temporalio:mainfrom
kisasa:batch-example

Conversation

@ddieruf

@ddieruf ddieruf commented Jul 13, 2026

Copy link
Copy Markdown

What was changed

Added a .NET port of the Java batch samples (samples-java/core/src/main/java/io/temporal/samples/batch), under a new src/Batch directory with one project per pattern:

  • Iterator — processes a page of records at a time via child Workflows run in parallel, continuing-as-new between pages.
  • HeartbeatingActivity — processes the whole batch inside a single heartbeating Activity, resuming from the last recorded offset after a worker restart instead of starting over.
  • SlidingWindow — keeps a fixed number of record-processing child Workflows running in parallel at all times, using completion Signals and continue-as-new to keep history bounded indefinitely; multiple partitions run side by side via BatchWorkflow for higher total throughput.

Each sample follows this repo's existing conventions (dotnet run worker / dotnet run workflow, per-sample README, TemporalioSamples.sln entries) and mirrors the Java sample's file/class structure closely so it's easy to compare side by side.

One deliberate deviation from the Java version: SlidingWindowBatchWorkflow continues-as-new on Workflow.ContinueAsNewSuggested in addition to a fixed page size, rather than relying on the fixed count alone. A fixed count can be outgrown by other accumulated history (e.g. completion Signals) before it's reached — this pattern is based on how we handle the same problem in production. A TestContinueAsNew input flag makes that branch deterministically testable without generating a large amount of real history.

Why?

There was no .NET equivalent of these samples, which came up in a community discussion about batch-processing workarounds (paging through large datasets, bounding history size, safely reporting child-Workflow completion across continue-as-new). This fills that gap using patterns we've validated in a production import pipeline.

Checklist

  1. Closes

  2. How was this tested:

    • dotnet build / dotnet format --verify-no-changes on the full solution.
    • Added automated tests under tests/Batch/* (xUnit, using WorkflowEnvironment.StartTimeSkippingAsync for the Iterator and SlidingWindow Workflow tests, and ActivityEnvironment for a targeted heartbeat-resume test on HeartbeatingActivity). All pass.
    • Manually ran each sample's worker + starter against a local Temporal dev server end to end and inspected results/history via the temporal CLI. This caught and fixed a real bug during development: SlidingWindowBatchWorkflow's original return value summed to double the actual record count across partitions; it now returns the Signal-tracked progress count instead.
  3. Any docs updates needed?

    • Added src/Batch/README.md (overview) plus a README per sample.
    • Added a Batch entry to the root README.md sample index.
    • No docs.temporal.io updates needed.

ddieruf added 4 commits July 13, 2026 12:32
Processes a batch by having a single Activity heartbeat its progress,
so a worker restart resumes from the last recorded offset instead of
starting over. Mirrors samples-java/.../batch/heartbeatingactivity.
Processes a page of records at a time via child workflows run in
parallel, continuing-as-new between pages so a dataset of any size
can be processed. Mirrors samples-java/.../batch/iterator.
Keeps a fixed number of record processing child workflows running in
parallel at all times, using completion Signals and continue-as-new
to keep history bounded indefinitely. Multiple partitions run side by
side via BatchWorkflow for higher total throughput.

Continues-as-new on Workflow.ContinueAsNewSuggested in addition to a
fixed page size, matching StreamPay's ProcessCustomerDataWorkflow,
since a fixed count alone can be outgrown by other history (e.g.
completion Signals) before it's reached. A TestContinueAsNew input
flag makes that branch deterministically testable.

Mirrors samples-java/.../batch/slidingwindow.
Adds the Batch solution folder and its three project references to
TemporalioSamples.sln and their test-project references to
TemporalioSamples.Tests.csproj, lists Batch in the root README's
sample index, and adds the overview README for src/Batch.
@ddieruf ddieruf requested a review from a team as a code owner July 13, 2026 16:39
@CLAassistant

CLAassistant commented Jul 13, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants