Skip to content

Move the eager state map into the invocation states - #93

Merged
slinkydeveloper merged 3 commits into
restatedev:mainfrom
jackyzha0:release-invocation-memory-at-terminal
Aug 11, 2026
Merged

slinkydeveloper merged 3 commits into
restatedev:mainfrom
jackyzha0:release-invocation-memory-at-terminal

Conversation

@jackyzha0

Copy link
Copy Markdown
Contributor

Problem

CoreVM keeps the invocation's eager state map (Context.eager_state, the complete Virtual Object KV state) alive until the CoreVM value is dropped. On native SDKs that's fine — the VM is dropped promptly at end of invocation. In the WASM SDKs it leaks by default: nothing calls WasmVM::free(), so the wrapper is reclaimed only when the host garbage collector's FinalizationRegistry fires. Under load those finalizers lag, completed invocations' state maps accumulate inside the shared WebAssembly.Memory, and since wasm linear memory never shrinks, the high-water mark becomes a permanent external-memory floor for the process.

We hit this in production with @restatedev/restate-sdk@1.16.4 (Node, Virtual Objects with multi-MB state maps): a flat ~319 MiB (external) ArrayBuffer plateau out of ~615 MiB live heap, alongside GC-overhead and CPU-starvation signals. wasm32 also caps the shared linear memory at 4 GiB, and allocator exhaustion there fails every in-flight invocation at once, so host memory sizing can't fully mitigate.

Why this shape

The state machine already has the right drop mechanism: Transition consumes self by value, which is why Replaying's command buffer and Processing's async results never had this problem. The eager state map was simply owned by Context, which outlives the machine. This PR moves it into the states that can observe it (WaitingReplayEntries/Replaying/Processing), so State::Closed structurally cannot retain it and failed transitions drop it automatically — no release hook, no terminal-edge analysis, no host-language lifecycle coordination.

Changes

  • eager_state moves from Context into the three non-terminal State variants.
  • State writes get dedicated SysStateSet/SysStateClear/SysStateClearAll transitions (mirroring SysStateGet), which journal the command and then update the map inside the machine, following the delegate-then-mutate shape of SysNonCompletableEntryWithCompletion. This also fixes a small pre-existing quirk: the map used to be mutated before the state check, so a write in an invalid state mutated it and then errored — it now journals nothing and mutates nothing.
  • The replay helpers borrow commands/async_results and return the notification handle; the triplicated continue-replaying-or-start-processing construction is deduplicated into replaying_or_processing().

Behavior

Unobservable to user code: reads consult the map only inside the Processing arm, post-terminal sys_* calls already fail with ClosedError/UnexpectedStateError before reaching it, and take_output is untouched. The existing test suite passes unchanged.

Happy to follow up with the sdk-typescript dependency bump + wasm bindings regeneration, and with before/after production memory numbers.

🤖 Generated with Claude Code

CoreVM kept the invocation's eager state map (the complete Virtual
Object KV state) on Context, alive until the CoreVM value was dropped.
The state machine's transitions consume self by value — Replaying's
command buffer and Processing's async results are dropped the moment a
terminal transition happens — but Context outlives the state machine,
so the eager state survived State::Closed and failed transitions.

That placement is invisible on native SDKs, where the VM is dropped
promptly, but it leaks by default in the WASM SDKs: the VM object is
reclaimed by host garbage-collector finalizers, which lag under load,
and wasm linear memory never shrinks, so eager state retained past the
end of the invocation permanently raises the process' memory floor.
Observed in production as a ~319 MiB external ArrayBuffer plateau in a
Node service with large VO state maps.

Move the map into the states that can observe it
(WaitingReplayEntries/Replaying/Processing). Terminal transitions now
drop it by construction. State writes get dedicated
SysStateSet/SysStateClear/SysStateClearAll transitions (mirroring
SysStateGet) that journal the command and then update the map inside
the state machine, which also fixes the previous behavior of mutating
the map before the state check — a write in an invalid state now
journals nothing and mutates nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@jackyzha0

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

@slinkydeveloper

Copy link
Copy Markdown
Collaborator

Thanks for the contribution!

@slinkydeveloper
slinkydeveloper merged commit 80ef6c2 into restatedev:main Aug 11, 2026
5 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants