Skip to content

feat(coding-agent): add tokens/sec to the working loader - #1179

Open
samsja wants to merge 1 commit into
mainfrom
feat/token-rate-ui
Open

feat(coding-agent): add tokens/sec to the working loader#1179
samsja wants to merge 1 commit into
mainfrom
feat/token-rate-ui

Conversation

@samsja

@samsja samsja commented Aug 11, 2026

Copy link
Copy Markdown
Member

Summary

Adds tokens/sec to the working loader message so the user can see output throughput while the model is streaming.

Changes

  • agent-activity.ts: Added tokensPerSecond to AgentActivityStatus, tracked via streamingStartedAt and streamingStartTokens fields. The rate is computed as delta_tokens / elapsed_seconds and only reported after 500ms of streaming to avoid jitter.
  • interactive-mode.ts: Updated getWorkingLoaderMessage() to append (N tok/s) after the token count when a rate is available.
  • agent-activity.test.ts: Updated toEqual assertions for the new tokensPerSecond field.

Example

Thinking · 5s · ↓ 1.2k tokens (240 tok/s)

Testing

  • npx vitest run test/agent-activity.test.ts — 13/13 pass
  • npx vitest run test/interactive-mode-status.test.ts — 151/151 pass
  • npm run check — clean

Note

Low Risk
UI-only activity tracking and loader text; no auth, persistence, or protocol changes.

Overview
The interactive working loader now shows output throughput while the model streams, e.g. Thinking · 5s · ↓ 1.2k tokens (240 tok/s).

AgentActivityTracker gains tokensPerSecond on AgentActivityStatus, measured from the first output of the current turn using the same monotonic token count as the existing loader. The rate stays 0 until at least 500ms of streaming so early numbers do not flicker. getWorkingLoaderMessage() appends (N tok/s) only when a positive rate is available; tests and the changelog are updated accordingly.

Reviewed by Cursor Bugbot for commit dfb501a. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add tokens/sec display to the coding agent working loader

  • Adds a tokensPerSecond field to AgentActivityStatus, computed from streaming start time and token count delta in agent-activity.ts.
  • Rate is only reported after ≥500ms of streaming and a positive token delta; returns 0 otherwise.
  • The working loader message in interactive-mode.ts appends the rate in parentheses, e.g. ↓ 1.2k tokens (45 tok/s), when the rate is positive.

Macroscope summarized dfb501a.

Track output token rate in AgentActivityTracker by recording when the
first output token arrives and computing tokens/second from the delta.
The working loader message now shows the rate alongside the token count,
e.g. 'Thinking · 5s · ↓ 1.2k tokens (240 tok/s)'. The rate only appears
after 500ms of streaming to avoid jitter on short responses.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit dfb501a. Configure here.

if (this.streamingStartedAt === undefined && this.currentTokens() > 0) {
this.streamingStartedAt = Date.now();
this.streamingStartTokens = this.completedTokens;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale tokens/sec across stream gaps

Medium Severity

streamingStartedAt is set on the first output tokens but never cleared on message_end, so computeTokensPerSecond keeps dividing by wall time through tool runs and waits. The loader then shows a decaying tok/s while idle or executing, and later streams inherit that inflated elapsed time instead of measuring active output throughput.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit dfb501a. Configure here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably should only consider TokensPerSecond when generations are actually happening. It might be slightly misleading otw

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants