Order topics match updates, not inserts

[stopachka]

The problem, by example

A chat query: the most recent messages in one conversation.

db.useQuery({
  messages: {
    $: { where: { "conversation.id": convoId },
         order: { createdAt: "desc" },
         limit: 5 },
  },
})

A reactive query subscribes to topics ([idx entity attr value]); a write invalidates the query when its change matches one. The query above includes:

[:vae  #{convoId}  conversations/messages   _]   ; its messages          (scoped to the conversation)
[:ea   #{m1..m5}   {messages/* fields}      _]   ; the 5 matched messages (scoped)
[:ave  _           messages/createdAt       _]   ; ORDER BY createdAt     (APP-WIDE)

Every topic is scoped to this conversation except the order by topic, whose entity and value are both _. So it matches a write to createdAt on any message in any conversation.

What goes wrong

createdAt is written on every new message. So a message created in another conversation matches [:ave _ messages/createdAt _] and re-runs this conversation's query (which recomputes the same 5 messages). With many conversations open, every message insert anywhere re-runs every open chat query. This isn't chat-specific: it's any where ... order by <attr> limit N.

The fix: order topics match updates, not inserts

The order by topic exists to catch a row whose sort position changes — a re-sort, including an out-of-window row reordering into the window. Those are updates. A row entering the window for the first time is an insert, and inserts are already caught by the other topics:

• a new linked row → the where/link topic ([:vae #{convoId} conversations/messages _])
• a new matching row for a scalar/comparator where → the where topic
• a brand-new entity for a where-less query → the id-enumeration topic ([:ea _ id _])

So the page topic doesn't need to fire on inserts — and firing on them is exactly what made it app-wide spam. The fix: value-changed updates emit a :mutated marker, and the page topic subscribes to :mutated instead of :ave/:ea:

before:  [:ave      _  messages/createdAt  _]   ; matches every createdAt write (insert or update)
after:   [:mutated  _  messages/createdAt  _]   ; matches only value-changed updates

• A new message in another conversation (an insert) no longer matches → spam gone.
• A message reordering into the window (an update to its sort key) still matches (entity stays _) → no regression.
• A new message in this conversation still invalidates via the where/link topic.

Why there's no regression

The page topic keeps its app-wide entity, so it still catches a re-sort of any matching row, including one currently outside the limit window. Everything the old topic caught is still caught:

change	caught by
new row enters window (insert)	where / link / id-enumeration topic
in-window row re-sorts (update)	:mutated page topic
out-of-window row reorders in (update)	:mutated page topic (entity is _)
row leaves window (delete)	data-fetch / where topic

The only writes the page topic stops matching are inserts, which are redundant there. (Apps that mutate the sort key across many parents still re-run on those updates — that's inherent and necessary; the common append-only createdAt/serverCreatedAt case has none.)

Dead ends (what we tried first)

Three approaches looked reasonable and were ruled out — recorded so we don't revisit them. All three try to fix the topic by its entity; the actual lever turned out to be the action (insert vs update).

1. Scope the page topic's entity to the visible window

Rewrite [:ave _ createdAt _] → [:ave #{the matched rows} createdAt _]. A write to another conversation's message isn't in the matched set, so the spam stops.

It silently misses a row reordering into the window:

messages where conversation.id = X   order by priority desc   limit 3

   p=90  #a ┐
   p=80  #b ├─ window      page topic: [:ave #{#a,#b,#c} priority _]
   p=70  #c ┘
   p=10  #d   ← off-window

   bump #d to p=100  (an update):   iv = [:ave #{#d} priority {10,100}]
   #{#d} ∩ #{#a,#b,#c} = ∅    →   query never refreshes; #d never appears

Measured on a real order by priority limit 5 link query: an off-window reorder is caught today but not with window-scoping (OLD: true → NEW: false). It would break live reordering for leaderboards, kanban boards, priority lists, etc. (where-less and scalar-where ordered queries already scope their page topic and already have this gap; window-scoping would newly extend it to the common link-where case.)

2. A per-app opt-in flag

Only enable scoping for apps whose sort key is append-only (e.g. Aura's createdAt). Safe, but a band-aid: it doesn't fix the general bug, needs manual curation, and leaves the spam in place for everyone else.

3. Scope to the full where-matching set (not just the window)

[:ave #{every message in X} createdAt _] would catch every in-conversation reorder and drop cross-conversation writes — correct, but the set is wrong on two counts:

[:ave #{ ...every message in conversation X... } createdAt _]
          │
          └─ not known up front (a `limit` query only materializes the window;
             the full match set lives in a pre-limit CTE and is discarded), and
             unbounded (a conversation can have 100k+ messages → a 100k-entity
             topic per open chat). Trades query-spam for topic/memory bloat.

Also dropped

• Order by $serverCreatedAt instead of createdAt: just moves the wildcard to [:ea _ id _] — id is written on every insert too, so the spam is identical.
• Client-side: filter by a denormalized scalar conversation-id field instead of the link (scalar wheres already scope their page topic): works, but relies on that field being backfilled on every row, and only helps the one app.

The insight that unlocked it

The spam is inserts (new messages elsewhere); a reorder is an update. New rows are already caught by the where/link/enumeration topic, so the page topic only needs updates — match :mutated, leave the entity app-wide. No set, no per-app flag, no regression.

What changes

• topics/topics-for-triple-update adds a :mutated marker to value-changed updates (triple.clj/datalog.clj index specs allow it). It's a topic-only marker, not a real index permutation.
• accumulate-results rewrites the page-info topic's index to :mutated.
• Gated behind the skip-order-topic-updates-only toggle (default on) to revert without a deploy.

Tests

• order-topic-matches-updates-not-inserts (instaql_test): an off-window row reordered into the window still invalidates; a new message in another conversation does not (with the toggle off it reverts to the old app-wide behavior, which matches both).
• Regenerated the pagination / pagination-with-checked-fields / $not-with-refs topic snapshots (page topics now key on :mutated).
• Updated invalidator-test/changes-produce-correct-topics for the :mutated marker on updates.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 80bf1107-6a08-4766-ae45-abab864fb26d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch scope-page-info-topics

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

View Vercel preview at instant-www-js-scope-page-info-topics-jsv.vercel.app.