Repository navigation
Move the eager state map into the invocation states - #93
Merged
slinkydeveloper merged 3 commits intoAug 11, 2026
Merged
slinkydeveloper merged 3 commits into
slinkydeveloper merged 3 commits into
Conversation
CoreVM kept the invocation's eager state map (the complete Virtual Object KV state) on Context, alive until the CoreVM value was dropped. The state machine's transitions consume self by value — Replaying's command buffer and Processing's async results are dropped the moment a terminal transition happens — but Context outlives the state machine, so the eager state survived State::Closed and failed transitions. That placement is invisible on native SDKs, where the VM is dropped promptly, but it leaks by default in the WASM SDKs: the VM object is reclaimed by host garbage-collector finalizers, which lag under load, and wasm linear memory never shrinks, so eager state retained past the end of the invocation permanently raises the process' memory floor. Observed in production as a ~319 MiB external ArrayBuffer plateau in a Node service with large VO state maps. Move the map into the states that can observe it (WaitingReplayEntries/Replaying/Processing). Terminal transitions now drop it by construction. State writes get dedicated SysStateSet/SysStateClear/SysStateClearAll transitions (mirroring SysStateGet) that journal the command and then update the map inside the state machine, which also fixes the previous behavior of mutating the map before the state check — a write in an invalid state now journals nothing and mutates nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
All contributors have signed the CLA ✍️ ✅ |
Contributor
Author
|
I have read the CLA Document and I hereby sign the CLA |
…e order of when the eager state map gets updated
Collaborator
|
Thanks for the contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
CoreVMkeeps the invocation's eager state map (Context.eager_state, the complete Virtual Object KV state) alive until theCoreVMvalue is dropped. On native SDKs that's fine — the VM is dropped promptly at end of invocation. In the WASM SDKs it leaks by default: nothing callsWasmVM::free(), so the wrapper is reclaimed only when the host garbage collector'sFinalizationRegistryfires. Under load those finalizers lag, completed invocations' state maps accumulate inside the sharedWebAssembly.Memory, and since wasm linear memory never shrinks, the high-water mark becomes a permanent external-memory floor for the process.We hit this in production with
@restatedev/restate-sdk@1.16.4(Node, Virtual Objects with multi-MB state maps): a flat ~319 MiB(external)ArrayBuffer plateau out of ~615 MiB live heap, alongside GC-overhead and CPU-starvation signals. wasm32 also caps the shared linear memory at 4 GiB, and allocator exhaustion there fails every in-flight invocation at once, so host memory sizing can't fully mitigate.Why this shape
The state machine already has the right drop mechanism:
Transitionconsumesselfby value, which is whyReplaying's command buffer andProcessing's async results never had this problem. The eager state map was simply owned byContext, which outlives the machine. This PR moves it into the states that can observe it (WaitingReplayEntries/Replaying/Processing), soState::Closedstructurally cannot retain it and failed transitions drop it automatically — no release hook, no terminal-edge analysis, no host-language lifecycle coordination.Changes
eager_statemoves fromContextinto the three non-terminalStatevariants.SysStateSet/SysStateClear/SysStateClearAlltransitions (mirroringSysStateGet), which journal the command and then update the map inside the machine, following the delegate-then-mutate shape ofSysNonCompletableEntryWithCompletion. This also fixes a small pre-existing quirk: the map used to be mutated before the state check, so a write in an invalid state mutated it and then errored — it now journals nothing and mutates nothing.commands/async_resultsand return the notification handle; the triplicated continue-replaying-or-start-processing construction is deduplicated intoreplaying_or_processing().Behavior
Unobservable to user code: reads consult the map only inside the
Processingarm, post-terminalsys_*calls already fail withClosedError/UnexpectedStateErrorbefore reaching it, andtake_outputis untouched. The existing test suite passes unchanged.Happy to follow up with the
sdk-typescriptdependency bump + wasm bindings regeneration, and with before/after production memory numbers.🤖 Generated with Claude Code