Phase 5: desktop responsiveness + engine watchdog by siracusa5 · Pull Request #105 · ContextCake/context-cake

siracusa5 · 2026-08-07T21:32:16Z

Phase 5 of the Sources Foundation plan: the desktop shell stops blocking on its own
bookkeeping, and an engine that stops answering becomes visible and recoverable instead
of silent.

What was wrong

The shell did disk I/O on the thread that draws. The sidebar resizer wrote a
preference on every pointermove, and the main process answered each one with a
synchronous read, write and rename — 1,200 blocking fs calls for a two-second drag whose
only interesting value is where it stopped.

Typing re-rendered the app. The store was one context memoized over ~25 dependencies,
so query shared an object with concepts, sources and conflicts, and nothing had
React.memo to stop the cascade.

A wedged engine was invisible. An engine exit was already fatal and handled. An
engine that is alive with its HTTP server bound but no longer answering was not measured
anywhere: requests simply never came back, the window kept its last paint, and the app
looked like one that had gone quiet.

Reduce Transparency was read from macOS and never offered, so a user who wanted the
translucent chrome gone had to turn it off system-wide.

A settings-sync pull could drop a source that exists only on this Mac. Inert in 0.5.0
because accounts ship disabled — which is exactly why it is fixed now, before it can eat
a real configuration.

Measured, before → after

	before	after
Preference writes during a 2s / 120Hz drag	241 patches, 1,200 blocking fs calls	1 patch, 1 async write cycle
Sidebar renders per search keystroke	5	0
Engine `GET /api/status` p95 under a 3,000-doc / 90MB index	not measured	10ms (limit 50ms)
Main-process lag under the same index	400ms ceiling only	3ms
A wedged engine	no signal at all	banner at 40s, restart offered at 60s

Then adversarial review found seven confirmed problems, three of them mine

Every one of the three regressions was the fix's own mechanism creating a new silence —
the same shape as the null-heading crash that started this plan:

Search silently stopped filtering the Queue. The context split moved query into
its own context; Triage subscribed to the other two and got memoized. The perf test
asserted only the negative — that the sidebar stopped re-rendering — and nothing
asserted a view still starts. There is now a positive test over every member of
SEARCHABLE_VIEWS, asserted equal to that list so a new searchable view cannot skip it.
Window geometry was lost on ⌘Q. The async write broke an ordering nobody had
written down: before-quit fires before window close, so the synchronous flush ran
on an empty queue. The red-X path still worked, which is why it looked fine.
A failed settings write became silent and was reported as success, because the same
commit removed the try/catch that used to carry the failure to the renderer. A failed
write now keeps its value in memory (that retention is the retry), tells the caller,
and the UI says the setting is in effect but will not survive a restart — rather than
snapping the switch back to a state the app is not in.

Also fixed: the watchdog's stop guard was untested and the wrong shape — a boolean
could not tell the old engine's late answer from the new one's during a relaunch, so it
is an epoch now; and the "blur budget" had removed backdrop-filter from four selectors
that render nowhere while making the one surface with moving content behind it opaque.

One review finding was wrong. It asked that stop() clear announcedUnhealthy; doing
that strands the banner over a working engine, because the relaunch path depends on the
new engine's first healthy answer being news. The mutation is red in the suite.

The latency gate was passing with the bug present

The new gate ran on a 6.6MB fixture where the vault that triggered this plan is 139MB. But
size was the smaller half: the tokenizer's cost tracks merge steps rather than bytes, and
the old corpus was one sentence repeated, so enlarging it alone moved p95 only 4→6ms. The
fixture now mixes dates, paths, identifiers, hex ids, URLs, code and tables into
Zipf-distributed English at 246 ms/MB, matching real notes.

It now catches a 35ms per-request stall (the original proof needed 300ms), and it catches
the files adapter reverting await fsp.readFile to readFileSync — which the old
fixture passed with ISOLATION OK, twice. A fixture-size guard fails loudly below
80MB / 10KB-per-doc so this cannot silently regress again.

Gates

Root npm test — green. Retrieval eval unchanged: recall@1 0.895 / recall@5 1.000 /
mrr 0.947. No ranking was touched.
apps/console — typecheck clean, 317 tests, build green.
apps/desktop — 103 tests, plus smoke, smoke:bootfail, smoke:relaunch,
test:isolation, test:navigation, test:cli-status.

smoke:relaunch is new and proves the recovery actually recovers: new origin, rotated
token, window re-pointed, old engine unreachable rather than orphaned, and trusted IPC
re-validating against the new origin.

Known and deliberately deferred

Deletes do not propagate through settings sync, and are un-doable while two Macs
sync — union-by-name resurrects a remotely-deleted source, and the next push
re-uploads it. Losing a local source is the worse failure, so this is the right trade
for now; a real delete needs a three-way merge against the _sync.shadow base that is
already stored. Inert in 0.5.0.
Object-level merge still treats a remote object as a complete snapshot, so a remote
entry that genuinely omits a key drops it locally. Also inert; also pre-accounts work.
After a failed relaunch the banner takes ~40s to reappear; the dialog carries the
immediate message.

🤖 Generated with Claude Code

… disk I/O on drags The sidebar resizer updated its preference on every `pointermove` and persisted each one, so a two-second trackpad drag sent 240 `ui-state:set` round trips and the main process answered every one with a synchronous read, write and rename on the thread that draws — 1,200 blocking fs calls for a gesture whose only interesting value is where it stopped. Two halves: - The console debounces persistence by 250ms and flushes on pointer-up, on unmount, and never writes back the value it just read at mount. Measured over a 120 Hz two-second drag: 241 patches becomes 1, and it carries the width the drag ended on. - `settings.mjs` writes through `fs.promises` on a serialized queue, keeping the atomic write-then-rename. The queue coalesces, so even 240 unthrottled patches cost one write cycle. `readSettings` stays synchronous — fifteen callers are synchronous by contract and a tiny JSON read is not what stalled anything — but it now prefers the queued state, which is what stops two patches in one tick from each reading the pre-patch file and dropping the other's field. `flushSettings()` lets settings-sync wait for the disk before it reads the same file, and `flushSettingsSync()` lands a pending write on the exit paths that cannot await one. Measured for 240 patches, before → after: 480 readFileSync + 240 mkdirSync + 240 writeFileSync + 240 renameSync → 2 readFileSync and one async mkdir/write/rename. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…er the app The store was one context value memoized over ~25 dependencies, so `query` sat in the same object as `concepts`, `sources` and `conflicts`. One character typed into the toolbar search re-rendered the sidebar, the header and the active view; there was no `React.memo` on any of them to stop it. Split by cadence rather than by subject: - `data` — engine answers and every action. Changes when the cascade changes. - `nav` — view, selection, and whether the Ask panel is open. - `input` — the search box and the chat composer. Every action lives in `data` and every action is now a stable identity — `setView` and `setQuery` read the current view through a ref rather than closing over it, which is what keeps `view` out of the data context's dependency list. `useStore()` still returns all three merged for consumers that read across them. `React.memo` on the view roots, the sidebar and the header, and stable `useCallback` identities for every handler the shell hands them, since a fresh arrow function per render would re-render straight through the memo. Files and Sources are deliberately left un-memoized — their suites drive updates by mutating a module-scoped store mock and re-rendering the same element, and those are the tests holding the navigator's focus guarantee and DOM-order invariant. `css()` now memoizes its parse. It is called inline in JSX, so it re-parsed on every render of every element; the cache is capped and dropped wholesale when full, because plenty of callers interpolate a coordinate. Measured over five keystrokes in the toolbar search, before → after: sidebar renders 5 → 0, header renders 5 → 5 (it owns the field). `render-hygiene.test.tsx` holds both numbers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…where it earns it Reduce transparency was read from macOS and never offered: a user who wanted the translucent chrome gone had to turn it off system-wide, and one who wanted it kept had no say at all. It is now a three-value choice — System, On, Off — where System is the default and the only route back once overridden. That third value is why the preference is stored as `boolean | null` rather than a switch. `reducedTransparency` is what the renderer does, `reducedTransparencyPreference` is what the user chose, and `systemReducedTransparency` is what the Mac says; collapsing them would make "follow this Mac" unreachable after the first click. The choice is device-local — it describes this display, not the user's taste — so it stays out of account sync state alongside the metrics opt-in. The blur budget: `backdrop-filter: blur(18px)` was on the soft panels, the soft cards, the subbar and the Settings sidebar. All four sit over the page background in normal flow — nothing scrolls behind them, nothing shows through but a static gradient — so each was a full-window compositing pass per frame buying a difference nobody can see. The one place it earns its cost is the app sidebar over the window's macOS vibrancy, where real desktop content is behind the glass; that one stays, still gated on the preference. The canvas legend moves to an opaque raised surface for the same reason. `initial` now really carries `systemReducedTransparency` — the argv flag and the preload parse are both new here. The type claimed the field before this commit while preload never produced it, which is the same shape as the null-heading crash that started this work: a type asserting a runtime it does not have. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…able The engine was supervised for exactly one failure: exiting. An engine that stays alive and stops answering produced no signal at all — the process is there, the port is bound, the window keeps its last paint, and every request hangs. Nothing measured it, so the only honest thing the app could say about a wedge was nothing, which is what it said. Reload acknowledgements were the same shape of lie, one layer down. `postWithAck` set a 5s timer that resolved the caller's promise on expiry, so `await service.reload()` returned identically whether the engine had re-read the manifest or had stopped listening. Proved by suppressing the child's ack: the smoke check printed `SMOKE OK` and exited 0 against an engine that never answered. It now resolves `{acked, reason}` and the smoke fails with the reason. The channel deliberately never rejects — the settings-pull path does not await it, and an unhandled rejection on the main process is the fatal handler. Two thresholds, because "slow" and "stuck" are different claims. More than three consecutive missed pings raises a banner naming the likely cause, since a large source genuinely can make the engine slow and that deserves a note rather than an intervention. Sixty seconds unresponsive additionally offers a restart, asked once per outage rather than once per tick. The wedge clock is anchored on the first missed ping, not the one that crossed the threshold: that is the earliest moment there is evidence for, and anchoring later would understate the outage by half a minute. A wedge does not take the fatal-exit path. It re-forks the engine and calls loadURL on the new origin, which the comments here said was impossible — they were written about the crash path and were wrong as a general statement. `npm run smoke:relaunch` now proves the whole thing: new origin, rotated token, window re-pointed, old engine unreachable rather than orphaned, and trusted IPC re-validating the sender against the new origin. Everything after that block in the smoke then runs against the relaunched engine. An unasked-for exit stays fatal, for the real reason: nothing in the main process knows why the child died, so re-forking into it could loop. The isolation test grew the half it never had. It proved the engine cannot freeze the window; isolation means a stall inside the ENGINE's request path now shows up as no main-loop lag whatsoever. A 300ms busy-wait per request measured 2ms of lag and would have passed. It samples the engine's own round trip during the same 2500-document index: measured p50 2ms / p95 3ms / max 15-40ms, held to the plan's p95 50ms / max 250ms SLO, and red with that busy-wait in place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…al sources A pull treated the remote `sources` array as the whole truth: `mergeRemoteValue` mapped over the remote entries and returned only those, so anything present locally and absent from the blob was gone. `applyPulledManifest` then split that list into `layers` and `pendingSources` and wrote it, which means a source added on this Mac — its level, its path, its place in the cascade — was deleted from the manifest because a different Mac pushed a list that had never heard of it. No prompt, no diff, no undo. Accounts ship disabled in 0.5.0, so nothing pulls today and nobody has lost a source to this. That is the only reason it is a landmine rather than a bug report, and the reason to disarm it now: the fix is six lines while the blast radius is hypothetical, and the day it stops being hypothetical is the day someone's vault vanishes. The test went in first and failed for exactly the right reason — `vault` missing from the merged list, both through `mergeSyncedSettings` and through a real `pull()` against a fake row. The merge is now union by name, which is what the push side (`overlayLocalValue`) has always done — the pull path was the asymmetric one. A source in both takes the pulled fields; a source only in the pull arrives; a source only here stays. Entries with no name or id, like a profile's plain string source references, have nothing to union on and stay remote-authoritative. The cost is that a deletion made on another Mac now comes back here instead of propagating; separating "never synced" from "deleted remotely" needs the last-synced blob as a merge base, which `_sync.shadow` already stores and a later three-way merge can use. Resurrecting a source someone deleted is an annoyance; deleting one nobody deleted is data loss, so this is the side to be wrong on. The security half is unchanged and now covered: `pullNow` re-runs the server's blob through `prepareSyncPayload` before merging, so `command`, `args` and `path` become scrub markers no matter what the row contained, and a marker resolves to the local value. Verified against a hostile row carrying `command: "/bin/sh"` and an attacker-chosen path: the planted source arrives with no command at all — `sourceIsRunnable` rejects it, so it lands in `pendingSources`, never a spawned layer — and the existing source keeps its own path. Union only ever appends entries that were already local, so it cannot widen that. The assertion now lives in the pull test. `rendererErrors` was an unbounded array fed by every renderer console error. A renderer in an error loop emits faster than the smoke check reads, so the array was a slow leak with no ceiling; it now holds the newest 200 and counts the overflow, which the smoke reports rather than hiding. Also documented, not fixed: `contextcake mcp` forks a second engine over the same manifest the app is already serving, and `mcp-server.mjs` has no background index — it re-walks every layer root per tool call. Two processes walking the same vault, one foreign MCP child per layer per engine, split cache TTLs, and a live-git pull that skips rather than waits when the other holds the lock. The comment at the spawn site and the gotcha in CLAUDE.md say what contends and why sharing the running app's service needs the bearer handoff designed first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

The context split in 459ed96 made every view root a `React.memo` with no props, so the only thing that can repaint one is a context it subscribes to. `Triage` subscribed to `data` and `nav`, and read the query indirectly — through `store.filtered(tab)`, a `useCallback` with an empty dependency array that reached into `queryRef.current`. Stable identity, no subscription: typing in the toolbar changed `input`, `data` and `nav` did not move, and the memo held. The Queue stopped filtering the day that commit landed. `triage` is in SEARCHABLE_VIEWS, so the box was right there, taking keystrokes and doing nothing with them. Nothing caught it because the only test on this path asserted the negative half of the split — that the sidebar does NOT repaint on a keystroke — while rendering just the sidebar and the header. A component that never re-renders at all satisfies that perfectly. The reproduction is that typing `zzzzznomatchzzzzz` at `#/triage` leaves `container.textContent` byte-identical. `filtered` is gone from the store, replaced by an exported pure `filterSignals(signals, tab, query)`. Subscribing to `useStoreInput()` in Triage alone would have fixed the symptom, but it would have fixed it invisibly: the next reader still could not tell that calling `filtered()` obliged them to subscribe to anything. Taking `query` as an argument puts the dependency in the type — you cannot call it without having the value in hand, and you cannot have the value without subscribing. `route()` keeps reading the ref, which is correct there: it fires from a keyboard shortcut outside the view, and it wants the freshest query rather than a rendered one. It also had its own inline copy of the predicate, which is now the same function. The other four searchable views were audited the same way and all four already subscribed to `useStoreInput` — concepts, files, sources and conflicts were never broken. That is now asserted rather than believed: the new suite drives a real keystroke through the real Header into each view and requires the list to empty, with the case table checked against SEARCHABLE_VIEWS itself, so adding a searchable view without wiring it up fails here. The negative assertion stays beside it, unweakened — the perf win is real and both halves have to hold. Verified red first, and each fix reverted independently: with Triage's subscription removed the new triage case fails with three rows still on screen and the other four still pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…nowhere 68c2745 claimed removing `backdrop-filter` from `.cc-subbar`, `.cc-soft-panel`, `.cc-soft-card` and `.cc-settings-sidebar` deleted "a full-window compositing pass per frame" from each. Grep the console for those four class names and the only file that mentions them is the stylesheet that defines them. Grep the whole repo — site, playground, the design handoff under project/ — and the count is still zero. Settings is built from `.cc-settings-toolbar` / `-nav` / `-content`. Nothing rendered any of the four, so the measured saving was zero, and the sentence explaining the saving was sitting in the file asserting otherwise. The one element the pass touched that does render is the Canvas legend, and it was the one place the blur was doing work. It is absolutely positioned over the pan/zoom viewport: concept nodes and their conflict edges slide underneath it while the user drags. It went opaque under the justification "for the same reason (nothing behind it)", which is precisely backwards — that reasoning holds for a panel in normal flow over a static gradient and fails for the only surface in the app with a moving backdrop. Its translucency is restored, now with the `-webkit-` pair so Safari gets it too on the web build, and a test asserts it: the caption is found in a rendered Canvas and the element must still carry `blur(10px)` over `--cc-header-bg`. Anyone who wants the glass gone has the reduce-transparency preference that same commit shipped, which resolves `--cc-header-bg` to the opaque raised surface and kills `backdrop-filter` app-wide — that, not flattening one card, is the escape hatch. The four dead rule sets are deleted rather than annotated, along with the two media-query overrides that dressed them. `.cc-sidebar` was on the shared rule the blur came off, so it too lost nothing: it is blurred by `:root[data-reduced-transparency="false"] .cc-sidebar` near the end of the file, which wins at (0,3,0) against the unconditional `backdrop-filter: none` two lines above it. "The sidebar keeps vibrancy" was true, but by cascade order and specificity, not by that edit. The comment now says that where the rule is, because the next person to touch it will be reading those two lines, not this message. Canvas.test.ts becomes .tsx for the render. Verified red first: with the legend back on `--cc-raised` the guard fails on an empty `backdrop-filter`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

`relaunch()` reports a refusal by resolving `{ok: false, reason: 'already-restarting'}`, not by rejecting. The banner only had `.catch(() => setRestarting(false))`, so that resolution ran no recovery at all and the button stayed disabled reading "Restarting…" until a `healthy` verdict arrived. This banner only exists because the engine has stopped answering, which is exactly the situation where a `healthy` verdict may never come — so a user who clicked twice, or clicked while a relaunch was already running, was left with a dead control on the only screen offering a way out. A third path had the same shape: a packaged app whose preload predates the `relaunch` channel but not `onStatus` renders the offer, and the optional chain short-circuits the whole expression to `undefined`. No promise, nothing to settle, button disabled forever. Both now clear the flag; only a resolution that actually reports `ok` keeps it held, because that one really does end with the window reloading at the new engine origin. Both cases went in as failing tests first — button still disabled after the click — and the existing "restarts once" assertion is untouched: a relaunch that succeeds still disables the button and still refuses the second click. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…old the user something untrue Six findings from the adversarial pass on the async-settings and watchdog commits. They are one commit because five of them live in main.mjs and one depends on settings.mjs, not because they are one idea. Window geometry was lost on every ⌘Q. Electron fires `before-quit` BEFORE the window's own `close` — verified with a standalone probe and now by the test — so the quit handler cancelled the pending bounds debounce and flushed an empty queue, and only afterwards did `close` compute the frame, onto the write queue that 0223ad9 had made asynchronous. Nothing drained it on the way out. The red X worked only by accident of the reverse order. The frame is now captured in `before-quit`, where the window is still alive and the flush is synchronous, and both paths are driven through the real binary. A failed settings write was discarded and reported as success. `drain()` cleared `unflushed` in a `finally` that could not tell a completed write from a failed one, so the very next `readSettings()` answered with the stale file: the preference gone, nothing left to retry, and `preferences:set` returning a snapshot that looked like agreement. The synchronous write it replaced threw, and that rejection used to reach the renderer. Writes now hand back a `written` promise that resolves `{ok:false}` — it never rejects, because an ignored return must not reach the main process's unhandledRejection, which is the fatal handler — a failure keeps the value so reads stay honest and the next patch carries it, and both IPC setters reject so the console (which already catches them) can stop claiming the change was saved. Nothing that leaves the machine — a metrics report, a settings push — now runs off a choice that is not on disk. A quit arriving inside `relaunchEngine`'s await leaked the engine. It ran `shutdownEngine()` against a null handle and adopted the new one afterwards, so `close()` was never called — and `close()` is what tells the engine to kill the MCP servers it spawned. Teardown now bumps an epoch and `startEngine()` refuses a handle from a superseded generation. `will-navigate` read `service.origin` unguarded. Relaunch is the first path that leaves it null with windows live, and a renderer-initiated navigation in that window threw straight into handleFatal: a wedge recovery could take the app with it. Reverting the guard reproduces `TypeError: Cannot read properties of null (reading 'origin')` and a fatal exit. A relaunch that failed quit the app, which is precisely what its own prompt promises will not happen — and it showed boot-failure copy ("Please reopen ContextCake") after a boot that had plainly succeeded. The app, its windows, its manifest and its session are all still there when the engine fails to come back, so the behaviour changed rather than the promise: say what actually failed, keep everything, and re-arm the banner's Restart Engine button. A device-local change made during a settings pull was reverted with nothing detecting it. `changedDuringPull` compared `_sync.localUpdatedAt`/`dirty`, and `writeLocalSettings` deliberately touches neither — so uiState, window geometry, reduced transparency and the metrics choice were invisible to the check and got written back over. Every write now bumps `_sync.revision`, which is what the check compares. Finally, the union-by-name comment overstated its own safety: a source deleted on another Mac does not merely fail to propagate, it travels back — this Mac's next push re-uploads it and the delete is undone where it was made. The trade is still the right one; the comment now says what actually happens. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…pping back The main process now rejects `preferences:set` when the write fails, which the renderer was handling two different wrong ways. The appearance setters swallowed the rejection entirely — the silent give-up this whole phase exists to remove. The Settings toggles did worse than nothing: they reverted the control to its previous value, while the main process had already applied the change and kept serving it. A user whose disk was full would turn off automatic update checks, watch the switch flip back on, and turn it off again — reading a state the app was not in. A failed write does not mean the choice was ignored. It means the choice will not survive a restart, and that is the only thing the user cannot otherwise see, so that is what gets said. The control holds the new value because the new value is what the app is using; one notice covers the appearance setters and this view's own toggles because they are one file failing for one reason, and it clears on the next write that lands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…wrong shape Replacing `if (stopped) return` with `if (false) return` left all seven watchdog tests green. The suite produced exactly one miss before stopping, and one miss is healthy — `publish()` returned early either way, so the guard was never the reason nothing got published. Every regression test here now drives the miss count past the threshold first, which is what makes the guard's absence observable at all. Made observable, the guard turned out to be insufficient rather than merely untested. `stopped` is a boolean and a relaunch is stop, fork, start — so a ping to the old engine, out for up to its 5s deadline against a swap that takes about a second, lands with the flag already cleared and the dead engine's answer clears the NEW engine's banner. It is an epoch now: stop() bumps it and a check drops any result whose epoch moved. `checkNow()` also armed the watchdog as a side effect, which is what made a stopped watchdog reachable in the first place. `shutdownEngine()` stops it and then closes the service; closing resolves every pending ack with `acked:false`, straight into `noteUnackedEngineMessage`, which calls `checkNow()`. That restarted a deliberately stopped watchdog, pinged an engine being closed, and carried the resulting miss into the next engine's count. It no longer arms. `announcedUnhealthy` deliberately still survives stop(), against the review's reading of it. It records what the WINDOW was last told, not what the engine was doing, and the console's banner clears only on a healthy verdict — so clearing it on stop is precisely what would strand that banner over a healthy engine after a relaunch, which is the one path that stops and starts the same watchdog. There is a test that goes red if someone clears it. Two more, found while in there. The per-ping deadline was the caller's to honor: a ping that never settled left `inFlight` true forever and the watchdog permanently silent — the exact failure it exists to report — so the deadline is enforced here now, and a ping that ignores its signal still counts as a miss on time. And a hung ping held its socket until a deadline nobody was waiting on; on a loop that runs for the life of the app that is a slow leak, so both the deadline and stop() abort it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

The ceiling was being asserted against 2,500 documents of 2.6KB — 6.6MB, against a field vault of 139MB across 3,000 notes averaging 47.6KB. Task 0.2 asked for 3,000 documents at 10-50KB and said why in one line: small docs hide the CPU-per-chunk cost that produces stutter. Size was the smaller half of it. The engine's per-document synchronous work is parse plus `countTokens`, and `countTokens` is an exact o200k BPE encode with no per-piece cache, so its cost tracks merge steps rather than bytes — and the old fixture was one sentence repeated, whose handful of common words are single tokens each. Measured over 8MB: real knowledge-base notes cost ~220 ms/MB, the old fixture 93, and inventing vocabulary to compensate overshoots at ~900. What makes real notes expensive is what real notes are full of, so the generator mixes dates, paths, identifiers, hex ids, URLs, code fences and tables into ordinary Zipf-distributed English at a rate calibrated against that measurement. It lands at 246 ms/MB. The proof that any of this matters is a bug the old gate could not see. Replacing the files adapter's awaited read with `readFileSync` — the one-line simplification that removes the per-document yield, and the exact regression class this file exists to catch — takes p95 to 487ms here and fails. On the old corpus the same regression measured p95 8ms and the gate printed ISOLATION OK, twice. The ceilings themselves did not move. They are the plan's SLO and the product promise, and Task 0.2 asks for generous thresholds because CI machines vary. What moved is the measurement under them, from p95 3ms to p95 11ms — and the sensitivity that buys is now recorded rather than assumed: an injected per-request stall fails the gate from ~35ms upward, where this file previously cited a 300ms busy-wait as its proof. `MIN_ENGINE_PROBES` was 3, which nothing but a dead probe loop could ever trip, and its comment argued for that on arithmetic that does not hold. It is derived now: a run honoring p95 <= 50ms over a 1.2s window of 20ms-spaced probes cannot produce fewer than ~17, so 12 catches a loop that died early while staying impossible for a merely slow engine to reach — and it runs after the latency assertions, so slowness is always reported as slowness rather than as an unsampled window. A healthy run lands at 43. The corpus also checks its own size before anything is measured. A fixture too small to gate the bug is how this gate got here, and it is the same shape as the churn assertion that passed with the bug present earlier in this plan. Costs 1.7s of wall clock (2.6s to 4.3s) and ~90MB of temp space freed at exit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…ing the loop alive CI (Node 22) failed the whole ack-channel file; the same file passes on the Node 24 this was written against. The tests were not wrong about the channel — they were wrong about the event loop. The deadline timer is `unref()`ed deliberately: an unanswered reload must never be the reason the app stays open. In the app that is invisible, because Electron always has a loop to run. In a test that awaits nothing but the deadline, the loop empties first and the timer never fires, so Node reports "Promise resolution is still pending but the event loop has already resolved" and cancels the remaining six tests in the file. Reproduced locally under Node 22 (6 of 7 red), fixed, then re-run green under both 22 and 24 — plus the full desktop suite and every smoke and isolation gate under 22, since any of them could have carried the same assumption. None did. The keepalive lives in the test because that is where the missing lifetime is. Removing the `unref()` would have greened CI just as fast and put a fifteen second stall on the quit path to do it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…uming it CI failed this gate at p95 65ms against a 50ms ceiling, on an engine with nothing wrong with it. The file had guessed: "a GitHub macos-14 runner is roughly half this machine, which puts a healthy p95 there near 20ms". The runner is about six times slower on this workload, and a threshold calibrated by assumption on one machine is a hardware detector everywhere else. So it measures now. Before booting, it times `countTokens` over a sample of the fixture it just wrote — the engine's dominant per-document synchronous cost, and therefore a direct proxy for how fast this machine can index — and scales the ceilings by the ratio to the reference. Floored at 1x, so a machine at least as fast as the reference is still held to the plan's actual SLO; capped at 8x, so a badly contended runner cannot scale the gate into meaninglessness. Two things this got wrong on the way, both worth the comments they left behind. The first calibration concatenated 2MB and timed one call, but `countTokens` encodes only the first 200,000 characters exactly and extrapolates — it measured a tenth of the work and read ten times too fast. It times per document now, which is also how the engine calls it. The second: the probe-count floor was derived from the unscaled ceiling, so a legitimately slower machine fitting fewer probes into the same 1.2s window would have failed with a message about the probe loop being broken. It derives from the scaled ceiling. Local behavior is unchanged — this machine calibrates to 1.02x, so the ceilings stay the 50ms/250ms the sensitivity proofs were measured against. What CI reported is not noise and is now in the plan's debt list: p95 65ms on a modest Mac, with main-process lag fine, means requests really do queue behind the tokenizer there. The gate can tell a regression from slow hardware; it cannot make the SLO true on hardware that misses it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

…ot see The calibrated gate passed CI, and the margin it passed by is the problem. The runner measured only 1.38x slower than the reference, which put its ceiling at 69ms — but its healthy p95 was 65ms on one run and 41ms on the next. A 1.6x spread on identical hardware, from contention no throughput measurement can observe, clearing its ceiling by four milliseconds. That is a gate that passes today and flakes next week, and finding out later costs more than fixing it now. So variance is absorbed in proportion to distance from the reference. A machine calibrating at 1.0 is the quiet dev box these thresholds were measured on and keeps the sharp 50ms ceiling; the further from it, the more likely a shared runner whose noise has to be tolerated. CI's ceiling becomes 98ms — 1.5x above the worse of the two healthy runs rather than 1.06x. This costs sensitivity only where there was little to lose. CI cannot resolve a 35ms stall through 24ms of its own jitter at either ceiling, while the regression the gate exists for — sync file reads back in the adapter — measures 487ms on the reference machine and worse on anything slower. The sharp gate is the local one, and it is untouched: this machine still calibrates to 1.00x. The 8x cap moved onto the final scale. Left where it was, a ratio already capped at 8 would have been multiplied to 18.5x, which is not a gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: John Siracusa <siracusa5@users.noreply.github.com>

siracusa5 and others added 15 commits August 7, 2026 15:36

siracusa5 merged commit bcdfcb5 into main Aug 7, 2026
8 checks passed

siracusa5 mentioned this pull request Aug 8, 2026

perf(console): chat keystrokes stop repainting the view — in both modes #108

Merged

Navigation Menu

Uh oh!

Phase 5: desktop responsiveness + engine watchdog - #105

Phase 5: desktop responsiveness + engine watchdog#105
siracusa5 merged 15 commits into
mainfrom
c/desktop-responsiveness

siracusa5 commented Aug 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

siracusa5 commented Aug 7, 2026

What was wrong

Measured, before → after

Then adversarial review found seven confirmed problems, three of them mine

The latency gate was passing with the bug present

Gates

Known and deliberately deferred

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant