Delivery durability for runs that aren’t served over HTTP or WebSocket

[Dan6erbond]

StreamDurability is great, but right now the only way to drive it is through a transport helper (toServerSentEventsResponse, toHttpResponse, the WebSocket helpers). The function that does the work, durableStreamSource in stream-to-response.ts, isn’t exported. It batches chunks, appends before forwarding, flushes early at tool calls and terminal events, and calls close() on every producer exit.

That leaves out runs whose producer never returns a response. In our case, chat() runs inside a Convex action. The action writes the run’s chunks to a database table, and every client (the sender, a second tab, a colleague viewing the same chat) follows the run through a reactive query on that table. There’s no HTTP response to attach durability to, so we ended up writing our own copy of durableStreamSource: a buffer, a flush timer, ordered appends and a terminal close. The same applies to queue workers, cron jobs, or any background run that clients join from a database or pub/sub instead of a response stream.

We can use the durability log without the transport by exporting the producer side, something like:

import { chat, pipeToDurability } from '@tanstack/ai'

const durability = convexRunLog(ctx, runId) // our StreamDurability over a Convex table

await pipeToDurability(
  chat({ adapter, messages, threadId }),
  durability,
  { abortController, batch: 64, batchWaitMs: 200 },
)
// append → close on completion, cancellation and failure, same as the transport helpers

pipeToDurability could just be a thin wrapper around durableStreamSource that drains the source. Exporting durableStreamSource itself (plus RUN_ACCEPTED_EVENT) would work just as well.

That way the batching, the early flush and the close-on-every-exit guarantee stay in one place, instead of every non-HTTP runtime re-implementing them and getting the edge cases slightly wrong. For example, a terminal batch can get skipped when persistence throws after the stream ends; we hit exactly that.