Skip to content

feat(desktop): Settings › Performance shows the local model's speed (#996) - #1120

Open
sheepdogs wants to merge 7 commits into
AtomicBot-ai:rel/xp-buildfrom
sheepdogs:feat/996-desktop-perf-panel
Open

sheepdogs wants to merge 7 commits into
AtomicBot-ai:rel/xp-buildfrom
sheepdogs:feat/996-desktop-perf-panel

Conversation

@sheepdogs

@sheepdogs sheepdogs commented Nov 10, 2026 •

Copy link
Copy Markdown

Closes #996 together with #1119 (agent, against main), which adds the route this pane reads (GET /api/local-model/performance). Against an older agent the pane says so and draws nothing.

What

Settings › Performance, just before Diagnostics. While the pane is on screen it asks the agent every 2 s and shows:

  • Generation: tok/s, newest + average.
  • Prompt: tok/s, newest + average. A fully cached prompt shows "—", not an older speed.
  • Time to first token: newest + average.
  • Workers sharing the GPU: busy / total from the server's slots right now, plus the most requests the agent had open at once in recent requests.
  • Context: newest request's tokens / the slot's context size.
  • Two charts (generation and prompt speed, separate scales), with a tick wherever the model changed and a tooltip per request.

Edge states, each with its own sentence:

  • the agent is too old (404);
  • the agent is not answering (keeps the last numbers under it);
  • the chat is on a cloud model (still draws earlier local samples);
  • no local requests yet.

How

  • renderer/perf-panel.js (new, declares only, like fusion-panel.js):
    • pure summary, formatters, SVG chart and pane HTML, with its own perfEsc for every string (the model name comes from the model server);
    • the poll as a small controller with its clock handed in.
  • Poll rules: it runs only while the pane is on screen, starting and stopping from render().
    • One request out at a time, across leave, re-enter and agent restarts.
    • Answers from an older generation are dropped.
    • A request that never answers is given up on after 15 s (the call itself times out at 5 s).
  • IPC: agent:localPerformance always answers one of {ok:true,data} / {kind:'unsupported'} (404) / {kind:'unreachable',error} — never a throw into the window (main/local-performance.ts).
  • css/perf.css uses the theme's variables only.
  • The route is added to the README's route table.
  • performance is added to the smoke check's pane list in main.ts.

Checks

  • npm run lint: clean.
  • Build (tsc + copy-renderer): clean.
  • npm run test:unit: 1437 / 1437 pass (+44 tests).
  • Mutation checks (each test group shown to fail with its guard removed):
    • escaping the model name;
    • the stale-answer check;
    • single-request-out;
    • the stall limit;
    • finite chart coordinates.
  • The pane was rendered from a real response of the agent route (7 requests against llama.cpp b11028 / Qwen3.6-35B-A3B) with no NaN/Infinity/undefined.
  • Run in the real window on Linux against a llama.cpp server (screenshot in the comments). That run found two display issues, fixed in 661734c.

Left alone

Analytics checks pane names against SETTINGS_PANES, which lists neither performance nor the existing plugins, so views of both fall back the same way. Happy to add both if you want them counted.

AI assistance

Developed with AI assistance (Claude Code built it to a written spec; an independent Codex review ran three rounds before this PR) under my direction. I reviewed the diff, re-ran every check above myself, and ran it against a real llama.cpp server.

sheepdogs and others added 6 commits November 10, 2026 12:25
…e functions

perf-panel.js reads the agent's GET /api/local-model/performance answer
(issue AtomicBot-ai#996): a summary of generation and prompt speed, time to first
token, workers sharing the GPU and context used; two small SVG charts
that break around missing values and mark a model change; the pane's
states; and the 2 s poll, one request at a time, with a generation
counter so an answer from before a leave, re-entry or agent restart
is dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AgentClient.localPerformance() reads GET /api/local-model/performance
with a 5 s timeout and answers the status instead of throwing; main's
agent:localPerformance turns that into ok / unsupported (404, an agent
from before the route) / unreachable, and preload exposes it as
BR.localPerformance (issue AtomicBot-ai#996).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Performance section just before Diagnostics draws perf-panel.js's
pane: generation and prompt speed (last and average), time to first
token, workers sharing the GPU, context used, and a chart of each
speed over the recent requests (issue AtomicBot-ai#996). The poll follows the pane
from render(): it starts when the pane comes on screen, stops when
Settings closes or moves on, and an agent restart makes any answer
already on its way stale. The --smoke settings check lists the new
pane.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… coordinates

Entering, leaving and re-entering the pane, or an agent restart, while a
request was still out sent another one alongside it. A newer generation
now waits for the request already out to settle and then asks; the stale
answer is still dropped. A chart size outside 1..10000 falls back to the
default, so no coordinate can overflow.
Keeping one request out at a time meant a request that never settled
held the pane on "loading" for good, even across leaving and re-entering
it. Past 15 s (the agent call itself times out at 5 s) the poll gives it
up: the pane says the agent is not answering, a fresh request goes out,
and the stuck one's late answer is dropped without freeing the newer
request's turn.
…m a live run

Seen in the running app against a real llama.cpp server:

- Time to first token showed "—" after every turn. The agent's own side
  requests (titles, memory) are not streamed, so they have no first-token
  time, and they land after the reply. The row now shows the newest
  streamed request's time, which is the wait the person had.
- The peak label was drawn inside the SVG, where it scaled with the
  chart's width and sat on the line when the first point was the peak.
  It is now in the caption, in the window's own font.
@sheepdogs

sheepdogs commented Nov 10, 2026 •

Copy link
Copy Markdown
Author

Tried it in the running window (Linux, against a llama.cpp server on another machine on the LAN):

Settings › Performance after three requests

The same pane in the dark theme. The charts draw in the theme's own colours:

Settings › Performance, dark theme

That run turned up two things, fixed in the last commit (661734c):

  • Time to first token showed "—" after every turn. The agent's own side requests, such as titles and memory, aren't streamed, so they have no first-token time, and they land after the reply. The row now shows the newest streamed request, which is the wait the person had.
  • The peak label was drawn inside the SVG. It scaled with the chart and sat on the line when the first point was the peak. It is now in the caption, as "peak 30 tok/s" in the screenshot.

Both have regression tests. npm run test:unit is at 1437 / 1437.

The low newest generation speed in the screenshot (6.3 tok/s) is one of those short side requests. The chart shows it as the last point.

sheepdogs added a commit to sheepdogs/atomic-agent that referenced this pull request Nov 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant