Skip to content

Publish the measured performance numbers in the README - #481

Merged
jonassaa merged 2 commits into
mainfrom
docs/readme-performance-numbers
Sep 18, 2026
Merged

jonassaa merged 2 commits into
mainfrom
docs/readme-performance-numbers

Conversation

@jonassaa

Copy link
Copy Markdown
Owner

Part of #257 — the README half. The site half is still deliberately waiting, and docs/dev/performance.md now says why the two differ.

What

A ## Performance section in the README, between "How it compares" and "Features": one row per benchmark fixture, with the first screen, working-tree status, the first page of history and the ratio against git's own work.

Repository First screen Status First page of history …vs git
torvalds/linux 15.84 s 989 ms 15.95 s 1.7×
deep 253 ms 0.53 ms 249 ms 1.3×
wide 5.42 s 5.42 s 0.25 ms
refs 219 ms 0.55 ms 135 ms 16×

Nothing was re-measured. The block is rendered from the committed docs/dev/benchmark.json, which is the same record docs/dev/performance.md's tables come from.

Why it is generated rather than typed

The README's first line calls this app fast, which makes it the single place where nudging a figure would pay off most — the failure #257 exists to prevent. So scripts/bench-report.mjs gained renderReadme() and a marker pair of its own, pnpm bench now writes three files instead of two, and test/benchmark.test.ts re-renders the README block from the record and fails on any disagreement.

Four new guards, each proven by planting the violation it describes and confirming it failed alone:

guard planted violation caught
re-renders the committed README block changed 253 ms153 ms by hand
a real timing in every README cell swapped a column to an op two fixtures lack
no hand-written figures in the prose wrote "…is 253 ms" above the markers
README.md in the js path filter deleted the filter entry

The third is the one worth calling out: the prose around the block is hand-written, and a measured figure copied into it is exactly what stops moving on the next run. 50,000-commit does not trip it; 253 ms does.

It leads with the bad case

performance.md said publishing 15.8 s for torvalds/linux as a selling point was the wrong response to it. That reasoning is about the marketing site and still holds there — but the README already carries "Where we are behind" and a "Status" list of known gaps, so the honest number belongs on it. The paragraph under the table names it outright, and it is interpolated from the record rather than typed, so fixing the log walk also unwrites the sentence.

docs/dev/performance.md is updated to distinguish the two audiences instead of saying "not published yet".

Verification

  • pnpm tsc --noEmit — clean
  • pnpm test — 4182 tests in 407 files, all passing, run after the last edit
  • Four planted violations, each failing its own test and only its own

No src/ or src-tauri/ change, so e2e is untouched.

🤖 Generated with Claude Code

jonassaa and others added 2 commits September 18, 2026 08:02
The README's first line calls this app fast. #257 turned that adjective into
a benchmark, but the numbers landed only in docs/dev/performance.md, which is
read a thousand times less often than the front page.

A new Performance section prints them: one row per fixture, the first screen,
status and the first page of history, with git's own work beside the last and
the ratio between them.

Why: the block is GENERATED by scripts/bench-report.mjs from the same
docs/dev/benchmark.json the doc's tables come from, and test/benchmark.test.ts
re-renders it and fails on any disagreement. A hand-typed figure on the front
page is the one that stops moving on the next run, and this is the file where
nudging one would pay off most.

It leads with the bad case rather than burying it: torvalds/linux opens in
15.8 s and ten pages into its history cost 157.67 s, interpolated from the
record so that fixing the thing also unwrites the sentence. A further guard
fails the build if a measured figure is written by hand in the prose around
the block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The module header still described the pair it rendered before the README block
joined them, and the README's framing promised a git timing column the summary
table does not carry — it carries the ratio.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jonassaa
jonassaa merged commit 50d43f5 into main Sep 18, 2026
14 checks passed
@jonassaa
jonassaa deleted the docs/readme-performance-numbers branch September 18, 2026 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant