Skip to content

Add a change feed: GET /changes, SSE stream, and list_changes - #10

Merged
chicagobuss merged 1 commit into
mainfrom
feat/change-feed
Aug 3, 2026
Merged

Add a change feed: GET /changes, SSE stream, and list_changes#10
chicagobuss merged 1 commit into
mainfrom
feat/change-feed

Conversation

@chicagobuss

Copy link
Copy Markdown
Owner

Agents coordinating across the fleet can only be woken by a background process. Polling list_docs per workspace and diffing snapshots over-fetches, holds client state, and cannot see hard deletes at all. This adds an event log with a resumable cursor.

  • events table, appended in the same transaction as the mutation it describes, so a rollback cannot orphan an event
  • GET /changes — cursor-paged, filterable by kind
  • GET /changes/stream — Server-Sent Events, emitting id: per frame and honouring Last-Event-ID
  • list_changes MCP tool mirroring the REST endpoint
  • no new dependency: the cursor makes reconnection stateless, which removes WebSocket's main advantage

The cursor took six review rounds

It is the opaque form workspace:kindshash:xact_id:id. Three designs were wrong before this one, each caught by adversarial review:

attempt why it was wrong
bare bigserial id Postgres allocates sequence values outside commit order — a lower id committing after a higher one has been read is excluded forever
gate on xact_id, page on id the two orderings are independent, so the same skip reappears
cursor not bound to its scope carrying one across a workspace or kind filter silently skips events

Reads order by (xact_id, id) and gate on xact_id < pg_snapshot_xmin(pg_current_snapshot()), which keeps the visible set closed downward — nothing can appear beneath a cursor already issued. Scope mismatch is rejected rather than silently restarted. The kinds component is a length-prefixed, full-width sha256, injective by construction.

The cursor is deliberately unsigned: RLS enforces row visibility, so forging one changes where you resume, not what you can see. It is a resumption hint, not a capability.

Cascading folio deletes emit one event per child via a CTE, so the mutated set and the recorded set are identical by construction rather than by two statements agreeing.

Testing

Verified against a real Postgres via scripts/test.sh — bare go test ./... skips the 37 DB-backed tests and still prints ok, which is how the first round shipped with no coverage of the new code.

The concurrency test was checked to fail against id-only ordering before the fix (skipped event B! got 0 events), on a copied tree. TestKindsHash_Injective uses the reviewer's exact newline counterexample.

Agents coordinating across the fleet can only be woken by a background process.
Polling list_docs per workspace and diffing snapshots over-fetches, holds client
state, and cannot see hard deletes at all. This adds an event log with a
resumable cursor so consumers learn what changed without asking repeatedly.

- events table, appended in the same transaction as the mutation it describes,
  so a rollback cannot orphan an event
- GET /changes, cursor-paged and filterable by kind
- GET /changes/stream, Server-Sent Events; no new dependency, since the cursor
  makes reconnection stateless and removes WebSocket's main advantage
- list_changes MCP tool mirroring the REST endpoint

The cursor is the opaque form <workspace>:<kinds>:<xact_id>:<id>. Three review
rounds went into that shape:

- a bare bigserial id is not safe, because Postgres allocates sequence values
  outside commit order: a lower id committing after a higher one has been read
  is excluded forever
- gating on xact_id while paging on id is also not safe, because the two
  orderings are independent
- a cursor must be bound to the scope that produced it, or carrying one across
  a workspace or kind filter silently skips events

Reads therefore order by (xact_id, id) and gate on
xact_id < pg_snapshot_xmin(pg_current_snapshot()), which makes the visible set
closed downward: nothing can appear beneath a cursor already issued. Scope
mismatch is rejected rather than silently restarted.

Cascading folio deletes emit one event per child via a CTE, so the mutated set
and the recorded set are identical by construction rather than by two
statements agreeing.

Tested against a real Postgres via scripts/test.sh - bare go test skips the
37 DB-backed tests and still reports ok. The concurrency test was verified to
fail against id-only ordering before the fix.
@chicagobuss
chicagobuss merged commit cecbea6 into main Aug 3, 2026
1 check passed
@chicagobuss
chicagobuss deleted the feat/change-feed branch August 3, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant