XIP-83: Mutable subscription streams with liveness

[tylerhawkes]

Draft XIP, opened for circulation/discussion.

Summary

Defines a single bidirectional subscription RPC (Subscribe) on the MLS API. A client opens one long-lived stream and mutates its topic set in place (add/remove deltas) instead of tearing down and reopening on every membership change; the server delivers messages plus a periodic liveness heartbeat (StatusUpdate{ WAITING }) so clients can detect silent stream death that transport keepalives miss — a terminating L7 proxy answers HTTP/2 pings at the edge while the backend subscription is gone. One connection can carry the union of many topics, the enabling primitive for multi-tenant agent gateways.

Compatibility

Additive and backward-compatible: existing SubscribeGroupMessages / SubscribeWelcomeMessages are untouched, and WASM/browser clients (no bidirectional gRPC) stay on them. A client calling Subscribe against a node that lacks it gets UNIMPLEMENTED and falls back.

Status

Draft, for discussion. The client-side liveness floor — a WatchdogStream combinator that reconnects a stale subscription from its persisted cursor — is already implemented in libxmtp against the existing server-streaming subscriptions (xmtp/libxmtp#3718). The Subscribe RPC + node heartbeat handler standardized here are the remaining protocol work.

Note

Add XIP-83 specification for mutable subscription streams with liveness

Adds xip-83-mutable-subscription-streams.md, a new protocol specification defining a bidirectional Subscribe RPC that supports in-place mutation of topic subscriptions without reconnection.

• The spec defines protobuf message shapes including SubscribeRequest/SubscribeResponse with versioned V1, Mutate (add/remove topics, history_only, mutate_id), and lifecycle messages (Started, TopicsLive, CatchupComplete)
• Application-level liveness is handled via Ping/Pong messages with server-controlled keepalive_interval_ms, separate from transport-level keepalives
• The document covers server and client requirements, per-subscription auth, version pinning, bounded catch-up, half-close behavior, rationale, backward compatibility, test cases, and security considerations
• Adds keepalive and keepalives to cspell.json to suppress spell-check warnings

^{Macroscope summarized 196fb3d.}

[macroscopeapp]

Approvability

Verdict: Needs human review

9 blocking correctness issues found. This new protocol specification has multiple unresolved review comments including high-severity issues around liveness semantics and authorization design. The XIP file is owned by @jhaaaa who should review these protocol design concerns.

^{You can customize Macroscope's approvability policy. Learn more.}

[neekolas]

Love this direction. Would be a huge unlock for Herald and also better for all mobile apps.

Main concern is that we have an even-more-different code path for the browser and everything else, and that browser issues can go unnoticed. Not a blocker, just an unfortunate side effect. Also means that our client streaming implementation needs to handle both mutable and immutable streams, which makes things harder to maintain. Such is life.

[macroscopeapp]

🟠 High XIPs/xip-83-mutable-subscription-streams.md:179

The Mutate.Subscription message (line 185) contains only topic and id_cursor, but server requirement 5 states that a single Subscribe stream may carry topics for multiple identities and that authorization MUST be evaluated per subscription. With no per-subscription identity or credential field, the node lacks data to authenticate which identity a newly added topic belongs to. An implementation must either authorize everything against the connection-level principal (breaking the gateway use case) or accept unauthenticated cross-identity adds, weakening authorization. Consider adding an installation_key or similar identity field to Subscription so the node can enforce per-subscription authorization.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @XIPs/xip-83-mutable-subscription-streams.md around line 179:

The `Mutate.Subscription` message (line 185) contains only `topic` and `id_cursor`, but server requirement 5 states that a single `Subscribe` stream may carry topics for multiple identities and that authorization MUST be evaluated per subscription. With no per-subscription identity or credential field, the node lacks data to authenticate which identity a newly added topic belongs to. An implementation must either authorize everything against the connection-level principal (breaking the gateway use case) or accept unauthenticated cross-identity adds, weakening authorization. Consider adding an `installation_key` or similar identity field to `Subscription` so the node can enforce per-subscription authorization.