Implement incremental catch-up and heartbeat for sync resilience#605
Open
noahm wants to merge 1 commit into
Open
Implement incremental catch-up and heartbeat for sync resilience#605noahm wants to merge 1 commit into
noahm wants to merge 1 commit into
Conversation
Implements roadmap step 2 for event-mode sync.
- Incremental catch-up: the server retains an in-memory tail of the last 500
stamped actions; a client that sees a seq gap on a live socket sends
{type:"catchup", since} and gets the missing actions replayed, instead of
reconnecting for a full snapshot. Live broadcasts are buffered while the
repair is in flight and drained once the gap closes. Falls back to a
reconnect when the gap predates the tail or catch-up goes unanswered.
- Application-level heartbeat: the client pings every ~10s and forces a
reconnect after 2 unanswered pongs, surfacing a stalled-but-open socket
that would otherwise go unnoticed until an ack timeout.
- Durable sequencer: seq and the dedupe id set now persist to room storage
(syncMeta key), so hibernation or a restart can't reset seq or forget
applied ids, closing the dedupe-across-hibernation hole.
New wire messages (catchup/ping/pong) are additive; older servers ignore
them and clients degrade to the previous reconnect-for-roomstate behavior.
Docs updated in partykit-sync-design.md and partykit-sync-roadmap.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tuc1MxeNwT1rU9bxSuVMUX
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements two critical improvements to the PartyKit sync protocol to make brief network disruptions nearly free and detect stalled connections:
Incremental catch-up: When a client detects a gap in action sequence numbers on a live socket, it now asks the server to replay just the missing actions instead of forcing a full reconnect. The server maintains an in-memory tail of recent stamped actions and responds with the gap-filling actions, or falls back to a full roomstate if the gap predates the retained tail.
Application-level heartbeat: Clients now ping the server every ~10 seconds and treat 2 unanswered pings as a dead connection, forcing a reconnect. This surfaces stalled-but-open sockets (e.g., frozen server, half-open TCP) that would otherwise go unnoticed until an ack timeout.
Persistent sequencer metadata: The server now persists
seqand the dedupe id set to room storage (syncMetakey) so hibernation/restarts don't reset the sequencer or forget applied ids, closing a small dedupe window.Key Changes
Client (
src/party/sync-manager.ts):resyncingflag andbufferarray to hold live broadcasts while a catch-up repair is in flighthandleCatchup()method to apply replayed stamped actions and drain buffered broadcastsingest()method extracted fromhandleRemoteAction()to apply a single action with gap detectionrequestCatchup(since)instead of immediately callingresync()MAX_CATCHUP_ATTEMPTS(3) before falling back to full reconnectdispose()to clean up timers and buffersServer (
src/party/server.ts):tailarray storing the last 500 stamped actions for catch-up replieshandleCatchup()method that returns missing actions from the tail or a full roomstate if the gap is too oldseqandseenIdsto room storage underSYNC_META_KEYon startup and after each actiononMessage()to dispatch by message type (action,catchup,ping)rememberActionId()renamed torememberStampedAction()and now maintains the tailClient socket manager (
src/party/client.tsx):HEARTBEAT_INTERVAL_MS(10s)missedPongsRefand forces reconnect afterMAX_MISSED_PONGS(2) unanswered pingsrequestCatchup()handler to send catchup messagesProtocol types (
src/party/types.ts):StampedActiontype: a ReduxAction with guaranteedidandseqCatchupRequestandCatchupResponsemessage typesPingandPongmessage typesClientMessageandBroadcastunion typesDocumentation (
docs/partykit-sync-design.mdanddocs/partykit-sync-roadmap.md):catchup/pingand degrade gracefullyNotable Implementation Details
seqorder once the gap closes. If a gap reopens mid-drain, another catch-up round begins.https://claude.ai/code/session_01Tuc1MxeNwT1rU9bxSuVMUX