Skip to content

Take the log's order from git: torvalds/linux opens in 999 ms instead of 15.7 s - #484

Merged
jonassaa merged 8 commits into
mainfrom
docs/fast-log-walk-plan
Sep 19, 2026
Merged

jonassaa merged 8 commits into
mainfrom
docs/fast-log-walk-plan

Conversation

@jonassaa

@jonassaa jonassaa commented Sep 18, 2026

Copy link
Copy Markdown
Owner

Implements #483. Opening torvalds/linux goes from 15.74 s to 999 ms.

Spec: docs/superpowers/specs/2026-09-18-fast-log-walk-design.md
Plan: docs/superpowers/plans/2026-09-18-fast-log-walk.md

Measured

pnpm bench --linux, Apple M4 Pro, idle machine, before and after on the same
day and the same fixture:

before after
linux — first screen 15,743 ms 999 ms
linux — first page of history 18,112 ms 222 ms
linux — page ten 15,842 ms 337 ms
deep — first screen 253 ms 59 ms
wide — first screen 5.42 s 5.41 s (it is status, untouched)
refs — first screen 219 ms 224 ms (it is ref enumeration, untouched)

Both acceptance criteria met: the kernel's first screen is under 1.5 s and is
now bounded by status (1.03 s) rather than by history (8.7 ms)
, and the first
page is under 500 ms.

What changed

Stage 1 — paint before history. refreshAll awaited eleven reads behind one
Promise.all and wrote once, so status, branches, tags and HEAD waited fourteen
extra seconds for the log. The page is still started beside the others (the
eleven-way concurrency open_screen measures is unchanged) but lands in its own
write, and refreshAll still resolves only once history has arrived.

Stage 2 — the order comes from git. build_walk_order now calls
git rev-list --date-order (git/log_walk.rs). Only oids cross over — commit
metadata still comes from libgit2 — so log_cache, FrontierBuilder, cursors
and ref decorations are untouched. Any failure (git missing, non-zero exit,
unparseable output) falls back to today's libgit2 revwalk: a slow page, never a
failed one. git/commit_graph.rs keeps a --split commit-graph warm, written
in the background after open, honouring core.commitGraph.

The correction worth reviewing carefully

#473 and #476 both propose --topo-order. It is the wrong ordering.
Measured byte-for-byte on the kernel's first 2,000 oids:

libgit2 sorting vs --date-order vs --topo-order
TIME | TOPOLOGICAL (what log_page uses) IDENTICAL differ at line 6; 1,627/2,000 shared

--topo-order does not reorder the same commits — it returns different ones, so
it would have silently changed which commits the first page shows.
tests/log_walk_ordering.rs pins it, with a fixture whose branches interleave on
purpose; the violation was planted and confirmed red, then reverted. The
benchmark's baselines are corrected too.

Two things found on the way

  • Oid::from_str accepts abbreviated hex and zero-pads it, so abc123
    parses into a valid-looking oid naming nothing. Truncated rev-list output
    would have become a plausible order with one wrong entry instead of an obvious
    failure. The parser requires full-length hex; a test records why.
  • The fixtures now carry a commit-graph, written by bench-fixtures.mjs,
    because the app writes one on open and a fixture without one measures a state
    the app does not leave a repository in. The git baselines get the same
    file
    , which makes several ratios look worse than when we withheld it. That is
    the correct comparison.

Deliberately not here

Verification

  • cargo test1399 passed, 0 failed (102 binaries)
  • pnpm test4184 passed (408 files)
  • pnpm tsc --noEmit — clean
  • pnpm bench --linux on an idle machine; all three published artifacts
    regenerated together and test/benchmark.test.ts re-renders them
  • e2e in Docker, snapshot rebuilt — 8 specs passed: smoke,
    history-diff, open-persisted-screen, history-ops, commit,
    status-stage, file-history, repo-tabs

🤖 Generated with Claude Code

jonassaa and others added 8 commits September 18, 2026 13:16
Opening torvalds/linux still costs 15.7s on current main, and the whole
of it is one libgit2 call: a sorted revwalk materialises the complete
ordered list before it yields a single oid, and libgit2 never reads the
commit-graph that makes git's answer 188ms.

Why: #479 fixed the per-page repayment (page ten went 157.67s -> 112ms)
but deliberately left the first walk, which is now the entire large-repo
problem. Measured fresh on a quiet machine, against 4c6b5ed.

Also records a correction both #473 and #476 need: they propose shelling
out to --topo-order, and that is the wrong ordering. libgit2's
TIME|TOPOLOGICAL is byte-for-byte --date-order; --topo-order shares only
1,627 of the first 2,000 kernel oids, so adopting it would silently
change which commits the first page shows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
refreshAll awaited eleven reads behind one Promise.all and wrote once, so
status, branches, tags and HEAD — a second's work on torvalds/linux —
waited fourteen more seconds for the log page. The log now lands in its
own write.

Why: the other ten reads are collectively fast on every fixture the
benchmark covers, so joining them to the slowest read is what turns a
one-second screen into a 15.7-second one. The page is still STARTED
beside the others, so the eleven-way concurrency open_screen measures is
unchanged, and refreshAll still resolves only once history has landed —
callers refresh and then read commits.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
torvalds/linux first screen 15.74s -> 999ms, first page 18.11s -> 222ms,
deep 253ms -> 59ms. wide and refs are unmoved, which is right: their cost
is status and ref enumeration, neither of which this touched.

The fixtures now carry a --split commit-graph, written by
bench-fixtures.mjs, because the app writes one on open and a fixture
without one measures a state the app does not leave a repository in. The
git baselines get the same file, so several ratios look worse than when
we withheld it -- that is the correct comparison.

The generated headline no longer asserts a cause it cannot see: it names
the slowest remaining operation from the data, which is now
file_history at 16.9s rather than the log.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jonassaa jonassaa changed the title Spec and plan for the first log walk on a million-commit repository Take the log's order from git: torvalds/linux opens in 999 ms instead of 15.7 s Sep 18, 2026
@jonassaa
jonassaa merged commit 8e94a47 into main Sep 19, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant