diff --git a/CLAUDE.md b/CLAUDE.md index 17eb69b5..d67a6aff 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,7 +30,9 @@ to what you are reading. A new section here needs a reason a pointer cannot serv - `docs/dev/performance.md` — the large-repo benchmark: the fixtures, what each number does and does not include, the baseline it is compared against, and the published results. Read it before touching the log walk, `status`, or - the refresh path, and re-run `pnpm bench` when you do. + the refresh path, and re-run `pnpm bench` when you do — it regenerates + `README.md`'s Performance block as well as the two under `docs/dev/`, and + `test/benchmark.test.ts` fails if any of the three is hand-edited. - `docs/dev/releasing.md` — what a version number means and when to bump which part, the cut-a-release runbook (changelog lands on `main` FIRST), and the prerelease-promotion traps. Read it before tagging anything. diff --git a/README.md b/README.md index 2362d1b8..6648f416 100644 --- a/README.md +++ b/README.md @@ -93,6 +93,7 @@ Every route, per platform, with the Gatekeeper and update notes spelled out: - **Free, all of it.** GPL-3.0, no license fee, no per-seat pricing, no "pro" tier gating rebase or conflict resolution. - **No account, no telemetry.** Nothing to sign in to and no analytics SDK anywhere in the tree. Your repositories and history never leave your machine. Forge tokens for the optional pull-request integration are yours, supplied by you and stored by your own git credential helper. - **Native, not a bundled browser.** A small Tauri binary with real OS windows on all three platforms. +- **Fast, and measured rather than asserted.** A reproducible benchmark over generated repositories that isolate deep history, a huge working tree and thousands of refs — plus a real clone of the Linux kernel — with `git`'s own timings beside every row, the bad ones published along with the good: [Performance](#performance). - **Keyboard-first and dense.** A Rider-style default keymap, a command palette, type-to-jump lists, hunk navigation and staging without touching the mouse — a dev-first TortoiseGit alternative that assumes you know git. ## How it compares @@ -132,6 +133,38 @@ images shown as "binary" rather than a preview — the full list is under Mercurial, and no Finder/Explorer shell integration, which is the thing TortoiseGit exists for. +## Performance + +"Slow on big repositories" is the most consistent complaint about every +established git GUI, so "fast" is a claim this project has to back with a number +somebody else can check. `pnpm bench` builds three deterministic fixtures — +deep history, a huge working tree, thousands of refs — optionally clones +`torvalds/linux` on top, drives the real git backend through all of them, and +records `git`'s own timing on the same question beside every row. + +The block below is generated from that run, and `test/benchmark.test.ts` +re-renders it from the committed record and fails the build when the two +disagree — so no figure here can be nudged by hand. + + + +Measured on Apple M4 Pro (14 cores, 48 GB, macos/aarch64) with git version 2.50.1 (Apple Git-155), on 2026-09-17 — medians over repeat calls against the real backend. `pnpm bench` reproduces the generated fixtures in about a minute; the kernel clone is opt-in. + +| Repository | First screen | Status | First page of history | …vs `git` | +| --- | --- | --- | --- | --- | +| **torvalds/linux**
1,482,923 commits · 96,034 files · 946 tags · 13 changed | 15.84 s | 989 ms | 15.95 s | 1.7× | +| **deep**
50,000 commits · 16 files | 253 ms | 0.53 ms | 249 ms | 1.3× | +| **wide**
1 commit · 50,000 files · 55,000 changed | 5.42 s | 5.42 s | 0.25 ms | — | +| **refs**
2,000 commits · 32 files · 5,001 branches · 2,000 tags | 219 ms | 0.55 ms | 135 ms | 16× | + +**First screen** is the eleven reads the app issues when it opens a repository, issued at once — a composite, because the failure worth catching is one slow read blocking the other ten. **Status** returns per-file added and removed counts, so its baseline is `git status --porcelain` plus both `--numstat` diffs rather than a bare `git status`. Ratios are against git's *work*, with process start-up subtracted — deliberately the comparison that flatters us least — and a dash is a baseline too small to divide by. No figure here includes the UI: the benchmark drives the git backend directly, with no webview in it. + +**torvalds/linux is the bad case, and publishing it is the point.** The first screen costs 15.84 s there, and reaching ten pages into its history costs 157.67 s: a sorted libgit2 revwalk pre-walks all 1,482,923 commits before it yields one, and the next page pays for that again. The developer who opens a repository that size and waits is the one this was written for, so the number belongs here rather than in a backlog. + +Every operation on every fixture, the `git` command behind each baseline, and what the numbers were read to mean: [`docs/dev/performance.md`](./docs/dev/performance.md). + + + ## Features - **Start anywhere** — open a repository, clone one (submodules included, with progress), or init a new one, and keep the ones you use in the recent list; reveal any file in Finder or Explorer, or open the repository in your terminal, straight from the context menu. diff --git a/docs/dev/performance.md b/docs/dev/performance.md index 9537178b..bb5e3722 100644 --- a/docs/dev/performance.md +++ b/docs/dev/performance.md @@ -122,11 +122,21 @@ disputed with evidence rather than in the abstract. | `$PGBENCH_HOME/results/.json` | no | raw, every sample — what makes a result checkable | | `docs/dev/benchmark.json` | yes | the published record, summary statistics only | | the table block below | yes | the same numbers, as a document | - -Both committed artifacts are **generated and not hand-editable**. -`test/benchmark.test.ts` re-renders the markdown from the JSON and fails when -they disagree. That guard is the point of the whole exercise: the way a measured -number turns back into an adjective is somebody nudging it in a hurry. +| `README.md`'s Performance section | yes | the same run, one row per fixture, for a reader who will never open this file | + +All three committed artifacts are **generated and not hand-editable**. +`test/benchmark.test.ts` re-renders both markdown blocks from the JSON and fails +when any of them disagree. That guard is the point of the whole exercise: the +way a measured number turns back into an adjective is somebody nudging it in a +hurry. + +The README block is the same `renderReadme` output the guard re-renders, so +`pnpm bench` rewrites three files and they are committed together. It prints +three operations rather than twelve, and those three are `REQUIRED_OPS` minus +`open` — the ops every fixture is already forced to publish. Widening it to an +op a fixture may legitimately lack is how the front page starts printing a blank +cell for the thing that regressed; `test/benchmark.test.ts` asserts every timing +cell there carries a digit. The record sits beside this file rather than under `site/`, and both are covered by the `docs/dev/` entry already in the `js` path filter in @@ -134,15 +144,28 @@ by the `docs/dev/` entry already in the `js` path filter in skippable by exactly the change it polices — the failure mode #210 already shipped once. -### The marketing site does not print these yet - -#257 asks for a measured figure on the site in place of an adjective, and the -block to do it is written. It is **deliberately not shipped yet**: the honest -headline today is that opening `torvalds/linux` takes 15.8 seconds, and the -right response to that is to fix it rather than to publish it as a selling -point. It ships once the log-walk work in the findings below lands — at which -point the record moves to `site/src/data/` beside `comparison.json`, which is -where this repository keeps published records the site reads. +### The README publishes them; the marketing site still does not + +The two are not the same audience and the numbers read differently to each. + +The **README** prints them, and leads with the bad case. That file already +carries a "Where we are behind" paragraph and a "Status" section of known gaps, +so a table whose worst row is `torvalds/linux` at 15.8 seconds is in keeping +with it rather than at odds with it — and the sentence under the table says so +in as many words. It is also where the word "fast" appears in the first line, +which makes it the single most valuable place for the adjective to be replaced +by something a reader can check. The block is generated and guarded exactly like +the one below; the prose around it is hand-written, and +`test/benchmark.test.ts` fails if a figure is copied into it, because a +hand-typed number stops moving on the next run. + +The **marketing site** is still waiting. #257 asks for a measured figure there +in place of an adjective and the block to do it is written, but a landing page +sells, and 15.8 seconds as a selling point is a different claim from 15.8 +seconds as a disclosed limitation. It ships once the log-walk work in the +findings below lands — at which point the record moves to `site/src/data/` +beside `comparison.json`, which is where this repository keeps published records +the site reads. Until then nothing under `site/**` is touched by a re-measurement, which also means `pnpm bench` cannot redeploy the website by accident. diff --git a/scripts/bench-report.mjs b/scripts/bench-report.mjs index dde5e57c..3e0c7375 100755 --- a/scripts/bench-report.mjs +++ b/scripts/bench-report.mjs @@ -1,5 +1,5 @@ #!/usr/bin/env node -// Render benchmark runs into the two things that get committed (issue 257). +// Render benchmark runs into the three things that get committed (issue 257). // // `src-tauri/benches/repo_bench.rs` writes one JSON document per fixture, full // of raw samples. This turns the set of them into: @@ -8,12 +8,16 @@ // read by `test/benchmark.test.ts`; // * the table block inside `docs/dev/performance.md`, between its generated // markers — the developer-facing record, with every operation, the `git` -// baseline beside it and the ratio between them. +// baseline beside it and the ratio between them; +// * the summary block inside `README.md`, between markers of its own — the +// same run, cut down to one row per fixture, because the README's first +// line calls this app fast and that is the sentence the numbers are for. // -// **Both are generated, and neither is hand-editable.** That is the whole point -// of the exercise: the site is supposed to print a number somebody measured, -// and the way a measured number turns back into an adjective is somebody -// nudging it in a hurry. +// **All three are generated, and none of them is hand-editable.** That is the +// whole point of the exercise: what gets published is supposed to be a number +// somebody measured, and the way a measured number turns back into an adjective +// is somebody nudging it in a hurry. `pnpm bench` writes all three; they are +// committed together, and `test/benchmark.test.ts` fails on a partial commit. // // The markdown is rendered from the PUBLISHED record rather than from the raw // runs, and that indirection is the guard test's whole leverage: the test @@ -128,6 +132,13 @@ export const BEGIN = ""; export const END = ""; +/** The README's own pair. Spelled differently from the doc's on purpose: the + * two blocks are different renderings of one run, and a reader who greps for + * one marker should not land in the other file's block. */ +export const README_BEGIN = + ""; +export const README_END = ""; + /** Milliseconds, at a precision the measurement can actually support. * * Three significant figures at most, and seconds once past a thousand @@ -171,6 +182,25 @@ export function describeRepo(r) { return parts.join(" · "); } +/** "1 commit", not "1 commits". `tidyScale` fixes this where the harness hands + * us free text; here we are building the phrase ourselves, so we can just get + * it right. */ +function count(n, singular, plural = `${singular}s`) { + return `${thousands(n)} ${Number(n) === 1 ? singular : plural}`; +} + +/** The same shape line as `describeRepo`, for a table CELL rather than a + * caption: shorter words, and the singular handled, because this one renders + * into the README where `wide`'s single commit would otherwise read + * "1 commits" on the project's front page. */ +export function describeRepoShort(r) { + const parts = [count(r.commits, "commit"), count(r.trackedFiles, "file")]; + if (r.branches > 1) parts.push(count(r.branches, "branch", "branches")); + if (r.tags > 0) parts.push(count(r.tags, "tag")); + if (r.dirtyEntries > 0) parts.push(`${thousands(r.dirtyEntries)} changed`); + return parts.join(" · "); +} + /** * What `git` spent on the WORK, with process start-up taken back out. * @@ -306,6 +336,99 @@ export function renderMarkdown(data) { return out.join("\n").trimEnd(); } +/** + * The README's summary block: one row per fixture, three operations wide. + * + * Generated for the same reason `performance.md`'s tables are, and re-rendered + * by the same guard test. If anything it matters MORE here: the README's first + * line calls this app fast, so this is the file where a measured number most + * wants to drift back into an adjective. + * + * What is new here is the SELECTION. A README cannot carry four twelve-row + * tables, so it carries three columns — and those three are exactly the ones + * `REQUIRED_OPS` already forces every fixture to publish, minus `open`, which + * is a fresh handle and free everywhere. Picking columns on editorial grounds + * instead is how this block quietly starts printing a blank cell for the one + * operation that regressed. + */ +export function renderReadme(data) { + const m = data.machine; + const out = []; + out.push( + `Measured on ${m.cpu} (${m.cores} cores, ${m.memoryGb} GB, ${m.os}) with ` + + `${m.gitVersion}, on ${data.measuredOn} — medians over repeat calls ` + + "against the real backend. `pnpm bench` reproduces the generated " + + "fixtures in about a minute; the kernel clone is opt-in.", + ); + out.push(""); + + const head = [ + "Repository", + "First screen", + "Status", + "First page of history", + "…vs `git`", + ]; + const at = (fixture, op) => fixture.operations.find((o) => o.op === op); + const cell = (fixture, op) => { + const found = at(fixture, op); + return found ? fmtMs(found.repeatMedianMs) : "—"; + }; + const rows = data.fixtures.map((f) => [ + `**${f.title}**
${describeRepoShort(f.repository)}`, + cell(f, "open_screen"), + cell(f, "status"), + cell(f, "log_first_page"), + fmtRatio(at(f, "log_first_page")?.ratioToGit ?? null), + ]); + const line = (cells) => `| ${cells.join(" | ")} |`; + out.push( + [line(head), line(head.map(() => "---")), ...rows.map(line)].join("\n"), + ); + out.push(""); + out.push( + "**First screen** is the eleven reads the app issues when it opens a " + + "repository, issued at once — a composite, because the failure worth " + + "catching is one slow read blocking the other ten. **Status** returns " + + "per-file added and removed counts, so its baseline is `git status " + + "--porcelain` plus both `--numstat` diffs rather than a bare `git " + + "status`. Ratios are against git's *work*, with process start-up " + + "subtracted — deliberately the comparison that flatters us least — and a " + + "dash is a baseline too small to divide by. No figure here includes the " + + "UI: the benchmark drives the git backend directly, with no webview in " + + "it.", + ); + + // The honest headline, interpolated rather than written, so that fixing the + // thing it describes also unwrites the sentence. Guarded because the kernel + // clone is opt-in: a run without it publishes the fixtures it measured and + // says nothing about the one it did not. + const real = data.fixtures.find((f) => f.kind === "real"); + const screen = real && at(real, "open_screen"); + const tenth = real && at(real, "log_page_deep"); + if (real && screen && tenth) { + out.push(""); + out.push( + `**${real.title} is the bad case, and publishing it is the point.** The ` + + `first screen costs ${fmtMs(screen.repeatMedianMs)} there, and ` + + `reaching ten pages into its history costs ` + + `${fmtMs(tenth.repeatMedianMs)}: a sorted libgit2 revwalk pre-walks ` + + `all ${thousands(real.repository.commits)} commits before it yields ` + + "one, and the next page pays for that again. The developer who opens " + + "a repository that size and waits is the one this was written for, so " + + "the number belongs here rather than in a backlog.", + ); + } + + out.push(""); + out.push( + "Every operation on every fixture, the `git` command behind each baseline, " + + "and what the numbers were read to mean: " + + "[`docs/dev/performance.md`](./docs/dev/performance.md).", + ); + return out.join("\n").trimEnd(); +} + /** The published record. Summary statistics only — see the header note on why * the samples stay out of the repository. * @@ -363,13 +486,19 @@ export function buildPublishedRecord(runs) { }; } -export function splice(doc, block) { - const a = doc.indexOf(BEGIN); - const b = doc.indexOf(END); +export function splice( + doc, + block, + begin = BEGIN, + end = END, + what = "docs/dev/performance.md", +) { + const a = doc.indexOf(begin); + const b = doc.indexOf(end); if (a === -1 || b === -1) { - throw new Error("docs/dev/performance.md is missing its generated markers"); + throw new Error(`${what} is missing its generated markers`); } - return doc.slice(0, a + BEGIN.length) + "\n\n" + block + "\n\n" + doc.slice(b); + return doc.slice(0, a + begin.length) + "\n\n" + block + "\n\n" + doc.slice(b); } function arg(argv, name, fallback) { @@ -405,8 +534,11 @@ function main() { const data = buildPublishedRecord(runs); const markdown = renderMarkdown(data); + const summary = renderReadme(data); if (print) { console.log(markdown); + console.log(`\n${"=".repeat(72)}\nREADME summary block:\n`); + console.log(summary); return; } @@ -417,6 +549,19 @@ function main() { const docPath = join(root, "docs/dev/performance.md"); writeFileSync(docPath, splice(readFileSync(docPath, "utf8"), markdown)); console.log(` wrote ${docPath}`); + + const readmePath = join(root, "README.md"); + writeFileSync( + readmePath, + splice( + readFileSync(readmePath, "utf8"), + summary, + README_BEGIN, + README_END, + "README.md", + ), + ); + console.log(` wrote ${readmePath}`); } if (import.meta.url === `file://${process.argv[1]}`) { diff --git a/scripts/bench.sh b/scripts/bench.sh index 12fec0e0..13d87d69 100755 --- a/scripts/bench.sh +++ b/scripts/bench.sh @@ -16,8 +16,10 @@ # JSON document per fixture. It is behind `--features bench` so the Rust CI # gate never builds it. # 3. `scripts/bench-report.mjs` renders those documents into -# `docs/dev/performance.md` and `docs/dev/benchmark.json`, which is -# what `test/benchmark.test.ts` reads. +# `docs/dev/benchmark.json`, the table block in `docs/dev/performance.md` +# and the summary block in `README.md` — all three generated from one run, +# all three re-rendered and compared by `test/benchmark.test.ts`. Commit +# them together; a partial commit fails that guard by design. # # Publishing is the default because a benchmark nobody publishes is a benchmark # nobody runs twice. `--no-publish` is for the case you are iterating on the diff --git a/test/benchmark.test.ts b/test/benchmark.test.ts index a1b760dc..1ea6e4f2 100644 --- a/test/benchmark.test.ts +++ b/test/benchmark.test.ts @@ -12,7 +12,10 @@ // * `docs/dev/benchmark.json` is the published record, written by // `scripts/bench-report.mjs`; // * the table block in `docs/dev/performance.md` is RENDERED from that JSON by -// the same module. +// the same module; +// * so is the summary block in `README.md`, which is the one that matters +// most: the README's first line calls this app fast, and it is read a +// thousand times for every reading of `performance.md`. // // Re-rendering here and comparing byte for byte means the two can only agree if // both came out of one `pnpm bench`. Edit either by hand and this fails, naming @@ -36,7 +39,10 @@ import { BEGIN, END, OP_ORDER, + README_BEGIN, + README_END, renderMarkdown, + renderReadme, // @ts-expect-error — plain .mjs with JSDoc types, no .d.ts } from "../scripts/bench-report.mjs"; @@ -45,6 +51,7 @@ const read = (rel: string) => readFileSync(root(rel), "utf8"); const DATA_PATH = "docs/dev/benchmark.json"; const DOC_PATH = "docs/dev/performance.md"; +const README_PATH = "README.md"; type Operation = { op: string; @@ -95,6 +102,14 @@ type Published = { const data: Published = JSON.parse(read(DATA_PATH)); const doc = read(DOC_PATH); +const readme = read(README_PATH); + +/** The contents of a generated block, or `null` when a marker is missing. */ +function block(text: string, begin: string, end: string): string | null { + const a = text.indexOf(begin); + const b = text.indexOf(end); + return a === -1 || b <= a ? null : text.slice(a + begin.length, b).trim(); +} /** The operations every fixture must carry. Not the whole of `OP_ORDER`: two * entries are conditional by design — `diff_workdir_file` needs a dirty tree @@ -118,6 +133,67 @@ describe("the published benchmark numbers", () => { ).toBe(renderMarkdown(data).trim()); }); + it("re-renders exactly the summary block committed in the README", () => { + const committed = block(readme, README_BEGIN, README_END); + expect( + committed, + `${README_PATH} is missing its BENCHMARK SUMMARY markers`, + ).not.toBeNull(); + expect( + committed, + `${README_PATH} and ${DATA_PATH} disagree. The README block is generated ` + + "from the same record as the doc — re-run `pnpm bench` (or " + + "`pnpm bench --linux`) and commit all three files together.", + ).toBe(renderReadme(data).trim()); + }); + + it("prints a real timing in every cell of the README table", () => { + // The README carries three operations instead of twelve, and they are ones + // `REQUIRED_OPS` already forces every fixture to publish. So a dash in a + // timing column there never means "not applicable" — it means a fixture + // stopped measuring something and the project's front page went quiet + // about it. (The ratio column is exempt: a `git` baseline at the start-up + // floor prints no ratio on purpose.) + const rows = renderReadme(data) + .split("\n") + .filter((line: string) => line.startsWith("|")) + .filter((line: string) => !/^\|[\s|:-]+\|$/.test(line)) + .slice(1); + expect(rows).toHaveLength(data.fixtures.length); + for (const row of rows) { + const [name, ...rest] = row + .split("|") + .slice(1, -1) + .map((c: string) => c.trim()); + for (const cell of rest.slice(0, 3)) { + expect(cell, `${name} publishes no timing in the README table`).toMatch( + /\d/, + ); + } + } + }); + + it("keeps hand-written numbers out of the README's Performance section", () => { + // The prose around the block is written by a person, and a measured figure + // copied into it is the exact regression this whole apparatus exists to + // stop: the generated block moves on the next run and the sentence above it + // does not. Numbers belong inside the markers, where re-rendering moves + // them. + const start = readme.indexOf("## Performance"); + expect(start, 'README has no "## Performance" section').toBeGreaterThan(-1); + const rest = readme.slice(start + 1); + const end = rest.indexOf("\n## "); + const section = end === -1 ? rest : rest.slice(0, end); + const prose = section.slice(0, section.indexOf(README_BEGIN)); + + const figures = prose.match(/\d[\d.,]*\s*(ms|s|×)\b/g) ?? []; + expect( + figures, + "A measured figure is written by hand above the generated block. Put it " + + "inside the markers, or say it without a number.", + ).toEqual([]); + }); + it("names the machine and the day it was measured", () => { // A performance number with no machine beside it is not a measurement, it // is a boast, and it is the first thing a reader checks. @@ -246,4 +322,16 @@ describe("CI runs this guard when its inputs change", () => { "commit now skips the guard that holds its numbers together.", ).toContain("docs/dev/"); }); + + it("has README.md in the js path filter", () => { + // Same story, and #210 was literally a README-only PR that ran no suite at + // all. The README is now an input of this file too, so the entry it already + // has for `comparison.test.ts` is load-bearing twice over. + const workflow = read(".github/workflows/tests.yml"); + expect( + workflow, + "`README.md` left the `js` filter in tests.yml, so editing the published " + + "performance figures by hand now skips the guard that catches it.", + ).toMatch(/README\\?\.md\$/); + }); });