feat(ocap-kernel): carry out a peer incarnation change on the run loop

[sirtimid]

Stacked on #1103. Review that one first; this branch's base is sirtimid/remote-inbound-run-queue-item.

A peer's incarnation change took a peerIncarnation_* savepoint of its own, which nested inside whichever crank happened to be open — the same defect #1103 just removed from the inbound message path. It becomes a run queue item, carried out by the run loop in a crank of its own.

The handshake still gets its answer immediately. Whether this is a restart is a read of what the store already says, and the transport awaits that answer to decide whether to reset the connection; it cannot wait for a crank. Only the writes are queued.

Changes

• RemoteManager.#handleIncarnationChange answers from getPeerIncarnation and calls KernelQueue.acceptPeerIncarnation. applyIncarnationChange does the writes in a crank.
• The change is queued behind anything that peer has already sent, in the same in-memory arrival buffer as inbound messages. That ordering is what keeps the two incarnations apart: a message ahead of the change belongs to the incarnation that is ending and is recorded against it, one behind it to the incarnation that is starting. So feat(ocap-kernel): take an inbound remote message in a crank of its own #1103's eager discard of queued arrivals goes away — with the change itself ordered, the discard would now throw away the new incarnation's messages instead of the old one's.
• applyIncarnationChange re-reads the stored incarnation and returns early if it already matches, so a peer that re-dials while its first change is still waiting does not get its c-list torn down twice.
• An incarnation change that cannot be recorded is logged and dropped rather than killing the run loop — the containment feat(ocap-kernel): take an inbound remote message in a crank of its own #1103 added for inbound messages.

Carries the incarnation half of #1079.

What review changed

afterCommit was doing the promise rejections, and resolvePromises writes the kernel store — promise state, reference counts, and a notify row per subscriber, all in autocommit once endCrank has committed. That is exactly what the contract added in #1101 forbids, and I had made the same mistake there. Two reviewers found it independently, and one traced a second consequence: the transport's give-up handling rejects the same promises from a send continuation, so whichever arrived second would Fail out of a post-commit hook and take the kernel with it.

The rejections now happen inside the crank, buffered with immediate: false so the notifies they produce still wait for the commit — the mechanism a vat's own syscalls already use. afterCommit keeps only finalizePeerRestart, which is in-memory counters and nothing else.

Restoring the router's exhaustiveness check also came out of review: this PR had deleted the @ts-expect-error on KernelRouter.deliver's default case, because peerIncarnation made the never reachable. deliver now takes Exclude<RunQueueItem, RunQueueItemPeerIncarnation>, which says in the type what the kernel does at runtime and brings the check back.

Testing

The handshake is answered without its writes having happened; the rejections are buffered and finalizePeerRestart waits for the commit; a peer's messages and its incarnation change come out in arrival order; a change queued twice is recorded once; the kernel carries out the item itself rather than routing it, and survives one it cannot record; and acceptPeerIncarnation is refused once the run loop has died. Every fix was mutation-checked.

Six existing incarnation tests now go through a handshakeAndRunCrank helper that does what the transport and the run loop do between them.

@metamask/ocap-kernel is green, as is @ocap/kernel-test's remote-comms suite against a rebuilt dist. That suite intermittently crashes the Node worker with Assertion failed: (env) != nullptr; it reproduces on origin/main and is not from this change — two consecutive clean runs here.

🤖 Generated with Claude Code

---

Note

High Risk
Changes remote peer restart bookkeeping, promise rejection timing, and message/incarnation ordering—core kernel run-loop and distributed comms correctness paths with breaking semantics.

Overview
BREAKING: Peer incarnation changes no longer run inside a nested peerIncarnation_* savepoint during an arbitrary open crank. They become a peerIncarnation run-queue item, delivered in its own crank like remoteInbound messages.

The transport handshake still answers immediately from getPeerIncarnation (restart vs not). Only the durable work is queued via KernelQueue.acceptPeerIncarnation, ordered in the same in-memory arrival buffer behind that peer’s already-accepted inbound messages—so old-incarnation sequence numbers are recorded before the change, and new-incarnation traffic after it. discardRemoteInbound on restart is removed in favor of that ordering.

RemoteManager.#handleIncarnationChange queues the change; applyIncarnationChange performs persist/restart logic without the old savepoint wrapper, skips duplicate work if incarnation already matches, rejects decider promises inside the crank with buffered notifies (immediate: false), and defers finalizePeerRestart to afterCommit. Kernel’s run-loop deliver handles peerIncarnation itself (not KernelRouter); recording failures log and return { abort: true } instead of killing the loop.

Adds RunQueueItemPeerIncarnation and narrows KernelRouter.deliver to exclude it. CHANGELOG documents the breaking behavior shift.

^{Reviewed by Cursor Bugbot for commit 2e949a4. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 29dae26. Configure here.}

[cursor]

Re-dial still reports a restart

High Severity

#handleIncarnationChange still returns restart whenever the store lags the observed incarnation, including on the re-dial the transport makes after that first verdict. The transport closes the new channel and expects the next handshake to see the persisted value and proceed; that write only happens later in applyIncarnationChange, so inbound re-dials keep being rejected and outbound sends keep throwing PeerRestartedError until a later crank commits. Each of those failures is terminal on the send path.

 

^{Reviewed by Cursor Bugbot for commit 29dae26. Configure here.}

[cursor]

Give-up now precedes restart writes

High Severity

The handshake now returns restart before persistPeerRestart or the in-crank PEER_RESTARTED rejections run. The transport then throws PeerRestartedError, and the send continuation’s give-up path actually runs: it writes startSeq and rejects decider promises as CONNECTION_LOST outside the incarnation crank. Those kernel-store writes are autocommitted, and the later crank finds nothing left to reject as a peer restart.

Additional Locations (2)

• packages/ocap-kernel/src/remotes/kernel/RemoteHandle.ts#L687-L702
• packages/ocap-kernel/src/remotes/kernel/RemoteManager.ts#L288-L308

 

^{Reviewed by Cursor Bugbot for commit 29dae26. Configure here.}

[cursor]

Failed change drops the incarnation barrier

Medium Severity

A peerIncarnation item that applyIncarnationChange cannot record is shifted off #arrivedFromRemotes and then aborted with no re-queue. Messages already sitting behind that item — the new incarnation’s — are then delivered against the old seq and c-list. The handshake already returned restart, so the peer will not replay those arrivals.

 

^{Reviewed by Cursor Bugbot for commit 29dae26. Configure here.}