perf(chat): bound the recurring refreshSlot history fetch to the view's own count - #6947
Conversation
UX Review (Fable 5) — ✅ PASSUX-level review of UX-Verdict: PASS Store-only perf change with no rendered surface: no strings, components, or pixels touched, and the diff's guards plus tests preserve the visible transcript, scrollback, and "load earlier" behavior exactly. [UX-REVIEWED] b400821 |
GPT 5.6 Review — ✅ human override acceptedHuman judgment by @iamwhatever overrides the GPT 5.6 finding for This comment is updated in place on each push. The model was not re-run because an authorized human decision supersedes it. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Sound, safety-first bound — but it expires at the 500-row ceiling, so the longest sittings quietly regain the exact per-turn tax this PR removes. Watch
Suggestions
[DESIGN-REVIEWED] b400821 |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of All claims verified against the repo: the server clamp is real ( First-Principles-Verdict: CONCERNS The bound is derived and count-matched correctly, but a shared-cut correctness fix rides along in a perf PR and two unbounded siblings stay unfixed. What this change shipsIntent: stop the recurring chat refresh from re-downloading the whole transcript every turn end and reconnect — a FIX.
Watch
[FIRST-PRINCIPLES-REVIEWED] b400821 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsBased on my independent review of the diff and the shipping logic in No findings. [OPUS-REVIEWED] b400821 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
9f85889 to
c257152
Compare
The mechanism holds and the finding is accepted. Matching the count preserves the row The suggested remedy — restoring the unbounded call — is the only part not taken, since For the overlapping case, Regression coverage in |
c257152 to
740f745
Compare
Accepted and fixed in
One correction beyond the suggested fix, because the server-row count alone does not Coverage, in |
Correct, and my fault: the body was written for the second revision and not reconciled On the second Watch item — the
A short page does not shrink the view, because nothing downstream trusts the ceiling to So the ceiling is an optimization, not the shrink protection: honouring it means the The |
4481d04 to
05ad4f9
Compare
The finding holds and is fixed. Working through it narrowed the mechanism, and the The reachable path is the floor, not the count-match. At
The guard is therefore on the floor: a bounded refresh declines when the floor Durable is defined once. Four tests, appended, no existing assertion touched. Two pin the decline — the Verified on |
05ad4f9 to
dee842e
Compare
The finding holds, and it corrects something I asserted on an earlier head. A prior Counting once on each side is not identity. The disk read is CHAINED, so it spans The fix is the half that was missing. A substitution now also requires the two rows The Three tests appended, no existing assertion touched. A reused id from an older Verified on The client-side span this lane raised seven times ( |
dee842e to
e76a578
Compare
The finding holds, and it is my regression, not reviewer churn. The previous round
The exception is positive evidence, not a relaxation. Each entry in a row's The ordering is what keeps the reused-id fix from being undone: equal Three tests appended, no existing assertion touched. A Verified on Both findings on this span are now closed, and the arc is worth stating plainly: the |
e76a578 to
c07e7fa
Compare
The finding holds and the suggested one-line fix is the right one: Worth being precise about why, because the harm is not the obvious one. An inflated So this is the same loss as the previous client-side round, reached through the count Two tests appended, no existing assertion touched. One stamped permission card leaves Verified on |
c07e7fa to
c9979c4
Compare
The defect is real and is fixed. The suggested remedy is not what shipped, because The diagnosis holds exactly. The proposed fix — "use the frozen disk prefix plus the window snapshot as the So the boundary is found by identity. It runs before the other two helpers and only under the flag. Length is settled Five tests appended, no existing assertion touched. The deleted tail does not Verified on Two notes for whoever merges. This is the third finding on this span, and the |
The finding holds and the recommended remedy was taken: the call and the helper are The finding is correct that the two cases are indistinguishable. A durable injector Why removal rather than a fifth attempt. This span took four blocking findings across Filed as #7526 with all four One cost stated plainly rather than buried: the staleness this work addressed is Verified on |
Accurate, including the part I had not stated: the carve-outs mean a single session can This is put to the maintainer rather than resolved here, because it is the one thing in
I am not asking for a third option, and I am not filing an issue for this: the question Everything the client half can do about it is already in: every guard failure fails |
The concern is real and the suggestion is a good one. Rebutted as disproportional to Three reasons it does not belong here. This PR is three files in one store slice with no What makes it safe to defer is that the fallback is not invisible to a developer who I have added it to #7526 as the observability half rather than filing a separate issue, |
The finding holds and is a genuinely different mechanism from the earlier hits on this The decision now re-reads state after the page arrives: One correction to the finding's causal chain, because it changes what the tests can Four tests appended, no existing assertion touched. Red-before is 1 of 4, and the Verified on |
The mechanism is real and this PR does not dispute it. What is rebutted is the demanded This is a repeat, not a new finding: the same lane raised the identical mechanism on The disagreement is pre-existing on Design Review reached the same place independently and landed on 🟡 CONCERNS — advisory, Three rounds inside the server half are why it is sequenced rather than shipped together: What the client half does own is fully closed, including this lane's round-10 finding on Not resolving this thread as fixed, and not asking this lane to change its verdict: the |
…'s own count `refreshSlot` passed no `limit`, so every recurring refresh -- a WS reconnect, the one fired on `chat_done`, a variant switch -- pulled the whole chained transcript. The cost grows with the transcript and is paid again at the end of every turn. A FIXED bound is not available to this thunk: unlike a pane warm it REPLACES `messages` in place, so a 50-row page would delete scrollback the user had paged back through. That is what the `fetchSlotDetail` comment was guarding. So the bound is COUNT-MATCHED: ask for at least as many rows as the view already holds. The count is the view's SERVER-row span (`meta.mid`-bearing rows, the same notion `serverRowCount` and the reducer's `priorServerRows` are built on), not `messages.length` -- the array also carries client-only rows (a `thinking` block, a `permission` card, a `queued` bubble) and counting those inflates the request past the view's own span. `PANE_HYDRATE_LIMIT` is the FLOOR (a floor cannot truncate), which covers a fresh or near-empty slot. Two carve-outs, both to avoid trading a perf win for a truncation: - Above `REFRESH_LIMIT_CEILING` (500, the ceiling the handler clamps `limit` to) a count-matched request comes back SHORT of what it matched, so a view paged back past it keeps the unbounded shape. - A view with a server span of zero has no count to match against, and that refresh is the client's only read of a transcript it holds nothing of, so it stays unbounded too. Matching the count preserves the row COUNT, not the row IDENTITIES: when the server gained rows while this client was away -- precisely the reconnect this refresh recovers from -- the most-recent-N slice begins NEWER than the view's oldest loaded row, and assigning it wholesale would delete that scrollback. The page is therefore checked before it is fulfilled, and is safe on any one of three counts: it reaches the START of history; it CONTAINS the view's oldest row (a superset loses nothing, which is where the floor's over-request lands); or its own oldest row is IN the view, so `olderHeadAbovePage` can cut a head to keep above it. On none of the three, page and view are FULLY DISJOINT -- refetch unbounded, rather than splice a disjoint page onto the view and publish a transcript with a silent hole in it. For the overlapping case `refreshSlot.fulfilled` keeps that head through the same three shared helpers `switchSlot`/`warmSlotCache` use (`olderHeadAbovePage`, `serverRowCount`, `pagingCursorAfterKeptHead`), which is what that cut exists for -- a third reducer re-deriving it is how the first two diverged. The kept head also shifts the older cursor, so "load earlier" is not a dead click, and `windowComplete` for both reasoning helpers now describes the loaded window rather than the fetch (it defaulted to `true`, a claim a bounded page cannot make). Bounding this fetch also moves it onto the OTHER branch of the slot-detail handler, and the two branches did not agree about which store decides a row's CONTENT. The unbounded branch returns `older + list(slot.messages)`, so the window decides; the bounded branch reads chained disk history, so disk does. That is invisible while the stores agree, and they stop agreeing exactly when a row is rewritten IN PLACE and not yet flushed -- which is what a variant switch is: `chat_regenerate` sets `_pending_rewrite` and broadcasts `chat_variant_switch`, whose client-side handler dispatches this very refresh. So the bound alone would have made selecting a variant paint the PREVIOUS one back over it. `_append_unflushed_tail` does not cover it: it appends rows the disk read is MISSING, and a rewritten row is not missing -- it is present and stale, at the same `meta.mid`. `_overlay_unflushed_edits` closes that: it substitutes each disk row for its live window twin, matched on `meta.mid`, so the bounded branch agrees with the unbounded one about content. A substitution and not an append, so `total`, `has_more` and the slice boundary do not move; rows from older sessions in a chained read match no id and pass through untouched. It runs unconditionally rather than behind `_pending_rewrite`/`_dirty_flag`, so a third flag cannot reopen the gap, and it reuses the `_snapshot_slot_window` pair the tail append already captured on the loop rather than adding a second tearing surface. The `warmSlotCache` half of the issue was already bounded by #3240 and is untouched here. Frontend tests assert the `limit` argument reaching `api.chatSlotDetail`, not just the resulting state -- the argument is the fix, and a state-only assertion would still pass with the bound removed. Backend tests pin the two branches agreeing on content, and 4 of the 5 fail on this branch with the handler change reverted. Fixes #4690
|
/ai-review override gpt b400821: The finding's only remedy is to revert the bounded refresh hunk, which is the entirety of issue #4690, so it is a scope ruling rather than a code defect; the disk-vs-window disagreement pre-exists on main, is disclosed in the PR body §3, and is tracked as its own work in #7526. |
Human judgment recorded@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
Problem / Motivation
refreshSlotcalledfetchSlotDetail(key)with nolimit, so every recurringrefresh pulled the whole chained transcript. It fires on a WS reconnect, on every
chat_done, and on a variant switch — so the cost grows with the transcript and ispaid again at the end of each turn. A user with a long session re-downloads their
entire history on every reply.
The
warmSlotCachehalf of #4690 is already fixed — merged commit8c751bda1(#3240) bounds it to
PANE_HYDRATE_LIMIT. This PR closes the remaining half andleaves the warm path untouched.
Why it matters
The refresh is recurring, not one-shot, so this is the transcript-length tax on
every turn end and every reconnect: a multi-thousand-row session pays it repeatedly
for rows already on screen. It gets worse the longer a session is used, which is the
opposite of what the user experiences elsewhere in the chat.
What changed (motivation → approach → change)
1. The client bound (the fix #4690 asks for)
A fixed bound is not available to this thunk. Unlike a pane warm,
refreshSlotreplaces
messagesin place, so aPANE_HYDRATE_LIMITpage would deletescrollback the user had paged back through — exactly what the
fetchSlotDetailcomment was guarding against. So the bound is count-matched, against the view's
server-row span:
The handler's slice is the most-recent-N, so a request of that size is bounded (it no
longer grows with the transcript) while never returning fewer rows than are on screen.
The count is the server-row span, not
messages.length: the array also carriesclient-only rows (a
thinkingblock, apermissioncard, aqueuedbubble) that theserver transcript does not, and counting those over-requests past the view's own span.
meta.midis the server's per-row stamp — the same notionserverRowCountand thereducer's
priorServerRowsare built on.PANE_HYDRATE_LIMITis the floor, not acap; a floor cannot truncate.
Two carve-outs, both to avoid trading a perf win for a truncation:
REFRESH_LIMIT_CEILING(500). The handler clampslimitto 500(
min(int(limit_raw or "200"), 500)), so a count-matched request above it comes backshort of what it matched. A view paged back past 500 rows keeps the unbounded shape.
client's only read of a transcript the client holds nothing of.
2. The sliding window (matching the count is not matching the rows)
Matching the count preserves the row count, not the row identities. When the
server gained rows while this client was away — precisely the reconnect this refresh
recovers from — the most-recent-N slice begins newer than the view's oldest loaded
row. So the page is checked before it is fulfilled, and is safe on any one of three
counts, each a different relationship between its range and the view's:
!hasMore), so it covers the view whatever theidentities are;
the floor's over-request lands, and a superset can lose nothing;
olderHeadAbovePagecan cut a head to keep above it.On none of the three, page and view are fully disjoint and the fetch is retried
unbounded. I deliberately did not reuse
warmSlotCache's disjoint-and-behind branch(
[...prior, ...pageTail]): that is right for a per-slot cache but wrong for the activetranscript, where it would publish rows 120–299 followed by 320–499 with 300–319
silently missing.
For the overlapping case,
refreshSlot.fulfilledkeeps that head through the same threeshared helpers
switchSlot/warmSlotCachealready use —olderHeadAbovePage,serverRowCount,pagingCursorAfterKeptHead. That cut exists precisely so a reducerconsuming a
fetchSlotDetailpage does not re-derive it; re-deriving is how the firsttwo diverged. The kept head also shifts the older cursor (so "load earlier" is not a
dead click), and
windowCompletefor both reasoning helpers now describes the loadedwindow rather than the fetch — it defaulted to
true, a claim a bounded page cannotmake.
3. The server-side gap this exposes — filed, not fixed here
Bounding this fetch moves it onto the other branch of the slot-detail handler, and
the two branches do not agree about which store decides a row's content:
older + list(slot.messages)That disagreement is pre-existing on
mainand already reachable through thepane-hydrate and warm-cache paths. This PR does not introduce it; by bounding the
recurring refresh it makes it more frequent.
An earlier revision of this PR carried a server-side reconciliation for it. It drew four
blocking findings in one span across four rounds, and the last three each came out of the
previous round's fix — matching rows after the fact always has a case where the match is
wrong. On the fourth, the reviewing lane recommended removing the mechanism rather than
refining it, and that is what happened:
chat_handlers.pyis byte-identical tomainonthis branch.
The gap, all four findings, the round-by-round table, and two directions that remove the
ambiguity by construction are in
#7526. The eleven tests written
across those rounds are preserved in commit
c9979c43dff6c8699735802b64d946a913433fedfor whoever picks it up.
Tests
Frontend —
website/src/store/chatSlice.refreshSlotBound.test.ts, 20 tests. Theyassert the
limitargument reachingapi.chatSlotDetail, not merely the resultingstate: the argument is the fix, and a state-only assertion would still pass with the
bound removed. The mock mirrors the handler — collapse before slice, most-recent-N, and
the 500 clamp.
The bound itself: a recurring refresh sends a
limitand it is not the corpus size; a3-row slot asks for
PANE_HYDRATE_LIMIT; a view paged back to 180 refreshes at 180,not 50;
hasMoreandslotOldestIndexare unchanged for a paged-back view; arefresh covering the corpus reports
hasMore: false; a view above the ceiling staysunbounded; a zero-server-span view stays unbounded; a streaming view is
count-matched and loses no rows; a non-active slot fetches nothing.
A slid window must not drop the head: the server gaining 5 rows keeps the loaded oldest
rows; the kept head shifts the older cursor; a head proving completeness reports
hasMore: false; a gap sliding the page clear of the view retries unbounded; a viewwith no server identity is not bounded at all; a page already at the start of history
does not retry; an id-less legacy prefix below the oldest identified row survives
(all 200 rows,
{hasMore: false, oldest: 0}); a mixed-history page sliding clear retriesunbounded with all 400 rows retained.
The count is the view's server-row span: client-only rows are not counted (a 192-row
view holding 12 reasoning rows requests 180); a window holding reasoning makes
one request, not a bounded one plus an unbounded retry; the floor over-requesting
into a superset does not retry.
Red-before: 7 of the 20 frontend tests fail on the base commit, and the three
head-keeping cases fail with the count-matched bound in place but their own guard
reverted, so each guard is pinned independently rather than by the bound alone.
Also re-ran the neighbouring suites: the
chatSlicefamily (8 files, 528 tests) —including
chatSlice.boundedRefetchShrink.test.tsandchatSlice.warmSlotCacheBound.test.ts,which pin the anti-shrink contract this PR operates inside — and, to confirm this branch
leaves the server untouched, the three
slot_detailbackend suites againstmain's ownhandler (23 tests, all passing). No existing assertion was edited or deleted.
Manual verification
N/A — unit coverage sufficient. The change is exercised directly: the
limitargument onan HTTP call plus the reducer's merge of its response. It has no rendered surface to drive
that the tests do not already cover.
Screenshots / video
Why no screenshot: store-only change — no component, layout, theme, or user-visible
string is touched, so a before/after would be identical pixels.
Related Issues
Fixes #4690