lidge-jun · lidge-jun · Aug 16, 2026 · Aug 16, 2026 · Aug 16, 2026 · Aug 16, 2026
@@ -0,0 +1,92 @@
+# GUI loading resilience + performance campaign — plan
+
+Date: 2026-08-16. Session: 01a00a59-96db-72a0-a12d-c8b9639e5607 (cxc-loop HOTL,
+goalplan slug: opencodex-gui-dashboard-performance-campaign-fix).
+
+## Objective
+
+1. Kill the "infinite loading" wedge: any tab can sit on a skeleton forever until F5,
+   then re-wedge later. Root-caused below with live fault-injection evidence.
+2. Per-tab performance: fewer timers, fewer redundant requests, no unbounded fetches.
+3. Hidden tab ≈ zero cost: every poll/timer in the dashboard pauses while
+   `document.visibilityState === "hidden"` and resumes (with one make-up refresh) on
+   visible. Only explicit opt-outs keep running (restart-reconnect detection).
+
+## Constraints
+
+- Worktree: /Users/jun/.codex/worktrees/f9b0/opencodex (detached at origin/dev tip
+  b81314cd2 when started). Branches: codex/gui-* stack, PRs target dev / parent branch.
+- The user's running ocx (port 10100) is never touched. All live verification runs
+  against a sandboxed instance (OPENCODEX_HOME=/tmp/ocx-gui-perf/home, port 10199)
+  behind a Vite dev server (port 5199, OPENCODEX_PROXY_TARGET).
+- Commits/pushes use --no-verify (user-authorized). Push + PR + admin merge
+  pre-authorized by the user for this campaign. Merges bottom-up via
+  `gh pr merge --merge --match-head-commit <sha>`.
+- Full suite runs on ssh lidge only (bun test --isolate tests); local gates are
+  focused gui tests, `cd gui && bun run build`, `bun run typecheck` at PR boundaries.
+- gui/AGENTS.md: no new hardcoded UI strings without i18n keys; no new dependency for
+  behavior the stack can provide; gui/dist is generated.
+
+## Root cause (evidence: 001_repro_evidence.md)
+
+H1 — client-resource fetch path has no deadline. A hung request leaves
+`refreshing:true` (cold keys also `loading:true`) forever: poll ticks skip while
+`inflight` is set (client-resource.ts:212), the visibility make-up fetch skips too
+(:192), and only manual refresh/unmount aborts (:214, :327). Measured: a stalled
+/api/settings produced ZERO retries over 12s+ while its 5s poll tick fired.
+
+H2 — the 401 re-bootstrap is an abort-proof, page-lifetime, app-wide chokepoint.
+`resolveTokenAfter401` shares one `resolutionInFlight` promise (api.ts:165-189) that
+awaits `reBootstrapSessionToken` → `rawFetch("/opencodex-session")` with no timeout
+and no caller signal (api.ts:108-123). Loopback sessions expire every 5 minutes
+(api.ts:101-107 design comment), so the wedge re-arms periodically. Measured: with the
+bootstrap stalled and the server otherwise healthy, /api/* traffic dropped to ZERO for
+40s+ while /healthz kept polling; cold tabs showed permanent skeletons (screenshot
+archived in 001). Bonus finding: a non-polled store caught mid-wedge (Storage) never
+recovered even after the network healed — nothing ever refires its request.
+
+H5 (secondary) — raw `setInterval` pollers outside client-resource
+(Debug 1s, ProviderSettings 2s, CodexAuth 30s, CodexAccountPickerSetting 30s,
+DefaultModeRequestUserInputSetting 30s, Models loadV2 10s, useCodexAccountPool) have
+no visibility handling, mostly no in-flight guard and no timeout. They keep firing in
+hidden tabs and can stack hung requests.
+
+Performance baseline (measured, CDP Network domain, 31s on Dashboard, idle sandbox):
+48 API requests — 9 endpoints polled at ~5s cadence each plus /healthz x6. Full
+per-tab inventory: 002_polling_inventory.md.
+
+## Work-phase map (dependency-ordered, PHASE-SPLIT-01; stack plan DEV-STACK-01)
+
+| WP | Decade doc | Layer (branch) | Proves on its own |
+|----|-----------|----------------|-------------------|
+| 1 | 010_phase1_resource_deadline.md | codex/gui-resource-deadline (base dev) | bounded fetch + wedge recovery in client-resource; tests |
+| 2 | 020_phase2_auth_unwedge.md | codex/gui-auth-unwedge (base WP1) | 401/re-bootstrap can no longer pin the page; tests |
+| 3 | 030_phase3_hidden_pause.md | codex/gui-hidden-pause (base WP2) | hidden tab ≈ zero fetch/timer activity; tests + measurement |
+| 4 | 040_phase4_poll_consolidation.md | codex/gui-poll-consolidation (base WP3) | shared tick scheduler + re-activation revalidation; request-count delta |
+
+Dependency rationale: WP1 is the data-path foundation every later layer's tests rely
+on; WP2 touches only api.ts but its abort-threading composes with WP1's new signals;
+WP3 builds the visibility ticker on the settled WP1 semantics; WP4 reshapes polling
+on top of WP3's visibility-aware scheduler. Lower layers are mergeable alone: each
+ships its own tests and stands green at its own tip.
+
+Non-goals: server-side endpoint merging, gui/dist edits, release/version changes,
+touching the running instance, redesigning page-level UX beyond error/loading states
+that already exist in data-surface.
+
+## Verifiers (all run at least once before each C>D)
+
+- `cd gui && bun test tests/client-resource-poll.test.tsx tests/<new>.tsx` (focused)
+- `cd gui && bun run build` (tsc -b && vite build — browser/bundler gate)
+- `bun run typecheck` (root, at PR boundary)
+- `cd gui && bun run lint` (oxlint, at PR boundary)
+- Browser: in-app browser against localhost:5199 with CDP fault injection
+  (repeat of 001 scenarios must now recover without reload)
+- Remote: ssh lidge 'cd ~/Developer/opencodex && git fetch && git checkout <merged dev>
+  && bun test --isolate tests' before final DONE claim.
+
+## Expected terminal outcomes
+
+DONE = all four PRs merged into dev, lidge isolated suite green, browser repro
+scenarios recover without reload, hidden-tab request rate ≈ 0. BLOCKED/NEEDS_HUMAN
+reported with evidence if any gate cannot run.
@@ -0,0 +1,91 @@
+# 001 — live reproduction evidence (research, no diffs)
+
+Environment: sandboxed backend `bun run src/cli/index.ts start --port 10199` with
+`OPENCODEX_HOME=/tmp/ocx-gui-perf/home` (isolated from the user's instance on 10100),
+Vite dev server on http://localhost:5199 with `OPENCODEX_PROXY_TARGET=http://127.0.0.1:10199`.
+Browser: in-app browser, raw CDP (Network + Fetch domains) for fault injection.
+
+## E0 — baseline request rate (performance criterion)
+
+All 9 nav tabs visited (~1s each), then 31s dwell on Dashboard, idle sandbox, page
+otherwise untouched. CDP `Network.requestWillBeSent` count:
+
+    48 requests / 31s on Dashboard alone:
+      /api/system/memory x5   /healthz x6   /api/providers x5   /api/v2 x5
+      /api/sidecar-settings x5   /api/shadow-call-settings x5   /api/settings x5
+      /api/injection-model x5   /api/effort-caps x5
+      /api/startup-health x1    /api/diagnostics/project-config x1
+
+i.e. ~1.5 req/s steady-state on one visible tab, ~9 concurrent 5s store pollers.
+
+## E1 — H1: one hung request wedges its store permanently
+
+CDP `Fetch.enable` on `/api/settings` (XHR+Fetch resourceTypes), never continue the
+paused request = a server that accepts but never answers.
+
+- 2 requests paused at mount (StrictMode first + live second), then ZERO further
+  /api/settings attempts measured over 12s and again over 40s — the 5s poll tick
+  skipped the wedged store every time (client-resource.ts:212).
+- Store snapshot stays refreshing:true; a cold key would show its skeleton forever.
+  Exit paths today: F5, manual refresh(), or unmount/remount of every subscriber.
+
+## E2 — H2: stalled 401 re-bootstrap wedges ALL management fetches
+
+Injection: fulfill every /api/* with 401 (real JSON body), stall /opencodex-session
+forever. Observed during the 401 wave: 3 requests fulfilled 401, 1 bootstrap request
+stalled. Only 3 /api requests ever hit the network because every later call joined
+the pending `resolutionInFlight` promise client-side (api.ts:167-169).
+
+Recovery phase: interception narrowed so /api/* flows to the REAL healthy server
+and only /opencodex-session stays stalled. Measured over the next 40s:
+
+    /healthz x2 (the 30s App poll — alive)
+    /api/*   x0 — nothing reaches the network, nothing settles, forever
+
+Cold tab opened in this state (Storage): "Scanning storage…" + 11 skeleton nodes,
+indefinite (screenshot: /tmp/ocx-gui-perf/shots/wedged-storage.png during session;
+regenerable by re-running this scenario). This is the user-reported symptom.
+
+## E3 — post-heal stickiness of non-polled stores
+
+After fully disabling interception (network healthy again), NEW cold tabs recover
+(Usage fetched /api/usage twice and rendered). The Storage store — non-polled,
+cold-mounted during the wedge — NEVER refired: 0 /api/storage requests in the
+following 10s+, skeleton forever. Non-polled stores have no retry path at all once
+their single attempt is lost inside the auth wedge (no poll tick, no visibility
+listener without pollMs — client-resource.ts:175 installs it only while polling).
+
+## E4 — hidden-tab emulation limit
+
+The in-app browser keeps background tabs `visibilityState: "visible"` (verified by
+opening+selecting a second tab). Hidden-tab verification therefore runs as
+happy-dom tests driving visibilityState directly (pattern already established in
+gui/tests/client-resource-poll.test.tsx:64-74), not live emulation.
+
+## Server-side note
+
+`/opencodex-session` is served by a static HTML responder (src/server/gui-static.ts:102)
+and is fast on a healthy loopback server; the defect is that the CLIENT has no
+deadline on this app-wide critical path, so any stall (event-loop stall, proxy
+restart mid-request, remote dashboard over a slow link) wedges the page. Sessions
+expire every 5 minutes (api.ts:101-107), so the exposure re-arms periodically —
+matching "stuck again a while after every refresh."
+
+## E5 — user addendum: "refresh → it loads/gets stuck AGAIN" (recurrence)
+
+User report (2026-08-16, mid-investigation): pressing refresh does not cure it —
+the loading state comes back. Consistent with the two mechanisms above:
+
+1. F5 clears module state and mints a fresh session via meta tags, so the first
+   seconds work; the next 5-minute session expiry re-enters the 401 → re-bootstrap
+   path (H2), and any single hung management route re-wedges its store (H1). Refresh
+   resets the clock, it does not remove the mechanism.
+2. The cold-mount fan-out right after F5 (~15-20 concurrent /api requests across
+   tabs, including endpoints documented as slow: /api/usage?range=30d ~5s cold,
+   Providers.tsx:99-102; live model discovery, Models.tsx:406-408) maximizes the
+   chance that at least one request stalls or 401s immediately, which is why the
+   stuck state can reappear almost immediately after a refresh.
+
+Implication for the fix: recovery must not depend on page lifetime or on the server
+never stalling. Every request needs a deadline that settles the store, and the auth
+resolution must be bounded and abort-aware — both lands in WP1/WP2.
@@ -0,0 +1,59 @@
+# 002 — fetch/poll inventory (research, no diffs)
+
+Source: independent explorer audit of gui/src at origin/dev tip, verified against
+live CDP measurement (001). No EventSource/WebSocket anywhere in gui/src — all
+realtime behavior is interval polling.
+
+## client-resource stores (pauseWhenHidden-capable)
+
+| Surface | File:line | Endpoint(s) | pollMs | Notes |
+|---|---|---|---|---|
+| App shell | App.tsx:128 | /healthz | 30s | non-gating |
+| Sidebar | sidebar-github-row.tsx:63,69 | /api/github/star, /api/update/badge | 300s/600s | |
+| Dashboard | use-dashboard-data.ts:199-285 | /api/startup-health 30s; /healthz+/api/providers 5s; /api/v2 5s; /api/sidecar-settings+/api/shadow-call-settings 5s; /api/settings 5s; /api/injection-model+/api/effort-caps 5s; /api/usage?range=30d; /api/diagnostics/project-config; /api/models | 5s wave | wave-2 gated on overviewReady |
+| Dashboard update job | use-dashboard-data.ts:406 | /api/update/status + /healthz | 1.5s | pauseWhenHidden:false (by design) |
+| Startup | Startup.tsx:206 | /api/startup-health+/api/settings(+/api/windows-tray) | none | session cache seed |
+| Providers | Providers.tsx:103,113 + ProviderWorkspaceShell.tsx:178 | /api/provider-presets; /api/usage?range=30d (shared key, 4 subscribers) | none | cache warm |
+| Models catalog | Models.tsx:366 | /api/models+/api/provider-context-caps+/api/providers | 10s | session cache |
+| Combos | Combos.tsx:197 | /api/combos+/api/config+/api/models | none | active-gated |
+| Compatibility | CompatibilityMatrix.tsx:297 | lab matrix | 60s | |
+| Subagents | Subagents.tsx:133 (loader :117) | /api/subagent-models | none | loader takes NO signal |
+| Logs | Logs.tsx:490 | /api/logs?limit=2000 | 2s (auto-refresh default on) | |
+| Debug | Debug.tsx:42,59 | /api/debug; /api/claude/inbound-debug | 2s | + raw 1s poll below |
+| Usage | Usage.tsx:775 | /api/usage?range&surface | none | |
+| Storage | Storage.tsx:1370,396 | /api/storage; /api/storage/trash | none | E3 victim |
+| Integrations | ApiKeys/ClaudeCode/ClaudeDesktop/Grok/IntegrationsOverview/FileIntegrationPage | 8+ keys | none (ClaudeDesktop status 5s) | active-gated, session-seeded |
+
+## Raw setInterval pollers (NO visibility handling today — H5)
+
+| File:line | Endpoint | Cadence | signal | timeout | in-flight guard |
+|---|---|---|---|---|---|
+| MemoryObservabilityCard.tsx:277 | /api/system/memory | 5s | yes | 10s bounded | yes |
+| MemoryObservabilityCard.tsx:342 | /healthz (reconnect) | 1.5s, gives up 120s | yes | 5s bounded | n/a |
+| ProviderSettings.tsx:162 | /api/provider-request-pacing | 2s | NO | NO | NO |
+| CodexAccountPickerSetting.tsx:42 | /api/settings | 30s | NO | NO | n/a |
+| DefaultModeRequestUserInputSetting.tsx:47 | feature endpoint | 30s | NO | NO | n/a |
+| useCodexAccountPool.ts:342 | account pool load | REFRESH_INTERVAL_MS | NO (:222,:250) | NO | n/a |
+| CodexAuth.tsx:154 | /api/config | 30s | NO (:123) | NO | n/a |
+| Models.tsx:413 | loadV2 | 10s, v2BusyRef-gated | NO (:298) | NO | busy-ref only |
+| Debug.tsx:157 | pollLogs(false) | 1s | NO (:115) | NO | NO — stacks hung requests, refreshing can stick (explorer S6) |
+| use-add-codex-account-oauth.ts:177 | OAuth login-status | 2s, 300s cap | yes | 10s bounded | yes |
+
+## Timeouts today
+
+Only 4 bounded call sites exist: MemoryObservabilityCard (2), ApiKeys mutations
+(15s), stop-proxy (15s), OAuth status (10s). Every client-resource fetcher (~40
+sites) and the session re-bootstrap are unbounded.
+
+## Shared-key hazard
+
+`usage-summary-30d:<base>:all` has 4 independent subscribers (Dashboard, Providers,
+AddProviderModal, ProviderWorkspaceShell). One in-flight serves all — and one hang
+wedges all (H1 applied to a shared key).
+
+## Staleness note (S7)
+
+Non-polled, active-gated pages (Integrations family, Combos) never revalidate on
+tab re-activation: subscribe with cached data does not refetch unless
+seedNeedsRevalidate (client-resource.ts:313-317). Contributes to the "stale until
+F5" feel; addressed in WP4 with staleness-threshold revalidation.