Skip to content

Implement incremental catch-up and heartbeat for sync resilience#605

Open
noahm wants to merge 1 commit into
partykitfrom
claude/partykit-sync-roadmap-step-2-feld6c
Open

Implement incremental catch-up and heartbeat for sync resilience#605
noahm wants to merge 1 commit into
partykitfrom
claude/partykit-sync-roadmap-step-2-feld6c

Conversation

@noahm

@noahm noahm commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary

This PR implements two critical improvements to the PartyKit sync protocol to make brief network disruptions nearly free and detect stalled connections:

  1. Incremental catch-up: When a client detects a gap in action sequence numbers on a live socket, it now asks the server to replay just the missing actions instead of forcing a full reconnect. The server maintains an in-memory tail of recent stamped actions and responds with the gap-filling actions, or falls back to a full roomstate if the gap predates the retained tail.

  2. Application-level heartbeat: Clients now ping the server every ~10 seconds and treat 2 unanswered pings as a dead connection, forcing a reconnect. This surfaces stalled-but-open sockets (e.g., frozen server, half-open TCP) that would otherwise go unnoticed until an ack timeout.

  3. Persistent sequencer metadata: The server now persists seq and the dedupe id set to room storage (syncMeta key) so hibernation/restarts don't reset the sequencer or forget applied ids, closing a small dedupe window.

Key Changes

Client (src/party/sync-manager.ts):

  • Added resyncing flag and buffer array to hold live broadcasts while a catch-up repair is in flight
  • New handleCatchup() method to apply replayed stamped actions and drain buffered broadcasts
  • New ingest() method extracted from handleRemoteAction() to apply a single action with gap detection
  • On gap detection, calls requestCatchup(since) instead of immediately calling resync()
  • Catch-up requests retry on 5s timeout with MAX_CATCHUP_ATTEMPTS (3) before falling back to full reconnect
  • Added dispose() to clean up timers and buffers

Server (src/party/server.ts):

  • Added in-memory tail array storing the last 500 stamped actions for catch-up replies
  • New handleCatchup() method that returns missing actions from the tail or a full roomstate if the gap is too old
  • Persists seq and seenIds to room storage under SYNC_META_KEY on startup and after each action
  • Refactored onMessage() to dispatch by message type (action, catchup, ping)
  • rememberActionId() renamed to rememberStampedAction() and now maintains the tail

Client socket manager (src/party/client.tsx):

  • Added heartbeat loop that pings every HEARTBEAT_INTERVAL_MS (10s)
  • Tracks missedPongsRef and forces reconnect after MAX_MISSED_PONGS (2) unanswered pings
  • Pong counter resets on any pong or fresh roomstate arrival
  • Wired requestCatchup() handler to send catchup messages

Protocol types (src/party/types.ts):

  • New StampedAction type: a ReduxAction with guaranteed id and seq
  • New CatchupRequest and CatchupResponse message types
  • New Ping and Pong message types
  • Updated ClientMessage and Broadcast union types

Documentation (docs/partykit-sync-design.md and docs/partykit-sync-roadmap.md):

  • Documented the catch-up flow, including buffer/drain mechanics and fallback to reconnect
  • Explained why the tail is memory-only (safe because clients only catch up on live sockets)
  • Documented the heartbeat mechanism and the stalled-socket failure mode it addresses
  • Updated version interop notes: older servers ignore catchup/ping and degrade gracefully

Notable Implementation Details

  • Buffer and drain: While a catch-up is in flight, live broadcasts are buffered and replayed in seq order once the gap closes. If a gap reopens mid-drain, another catch-up round begins.
  • Memory-only tail: The tail is deliberately not persisted to room storage. A client can only observe a live-socket gap while the same actor has been running

https://claude.ai/code/session_01Tuc1MxeNwT1rU9bxSuVMUX

Implements roadmap step 2 for event-mode sync.

- Incremental catch-up: the server retains an in-memory tail of the last 500
  stamped actions; a client that sees a seq gap on a live socket sends
  {type:"catchup", since} and gets the missing actions replayed, instead of
  reconnecting for a full snapshot. Live broadcasts are buffered while the
  repair is in flight and drained once the gap closes. Falls back to a
  reconnect when the gap predates the tail or catch-up goes unanswered.
- Application-level heartbeat: the client pings every ~10s and forces a
  reconnect after 2 unanswered pongs, surfacing a stalled-but-open socket
  that would otherwise go unnoticed until an ack timeout.
- Durable sequencer: seq and the dedupe id set now persist to room storage
  (syncMeta key), so hibernation or a restart can't reset seq or forget
  applied ids, closing the dedupe-across-hibernation hole.

New wire messages (catchup/ping/pong) are additive; older servers ignore
them and clients degrade to the previous reconnect-for-roomstate behavior.

Docs updated in partykit-sync-design.md and partykit-sync-roadmap.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tuc1MxeNwT1rU9bxSuVMUX
@vercel

vercel Bot commented Jul 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
ddr-tools Ready Ready Preview Jul 8, 2026 11:18pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants