Skip to content

perf(workspace): cut Git process count in checkout verification - #261

Merged
kumanday merged 4 commits into
mainfrom
perf/workspace-checkout-verification
Nov 9, 2026
Merged

kumanday merged 4 commits into
mainfrom
perf/workspace-checkout-verification

Conversation

@kumanday

@kumanday kumanday commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Summary

Checkout verification asked Git one question per process. workspace_manager::parent_execution_root_reuses_three_repositories_and_preserves_children started 5,211 Git processes. It took 85 s on macOS and about 10 s on Linux CI. This PR keeps every check and makes fewer, cheaper Git calls.

  • Batched rev-parse in verify_git_checkout. One call now answers all the repository-layout questions: work-tree status, top level, git dir, common dir, objects path, grafts path, and shallow state. A second call returns HEAD and --symbolic-full-name HEAD. Before, these took nine processes. A detached HEAD prints HEAD, so the branch comes out empty, as it does with git branch --show-current. A new helper, rev_parse_lines, requires exactly one line per answer; any other output fails verification.

  • Batched rev-parse in verify_parent_integration_checkout. One call now returns the top level, common dir, and HEAD.

  • Verified state is passed to callers. Parent preparation and validate_parent_retained_checkouts now get the branch, HEAD, and cleanliness that verification already observed. Before, they re-ran status, branch --show-current, and rev-parse HEAD after it. The checks they make are unchanged.

  • The real Git binary on macOS. When git on PATH is the /usr/bin/git xcrun trampoline, git_program() resolves the real binary once with xcrun --find git, which is the same lookup the trampoline performs. That removes about 10 ms from every Git process. The workspace manager and the memory code indexer both use it. On other platforms and other PATH layouts, Git is still spawned through PATH.

  • CI fix for DuckDB linking. Before saving, rust-cache prunes the library under target/duckdb-download, so the cache-directories entry added in test: cut slow Rust test wall time #260 never carried it. The libduckdb-sys build-script output is still cached, and on a cache hit it points the linker at the missing library, so the first runs of this PR failed to link against -lduckdb. The rust-test job now removes that build-script output after restoring the cache, and the build script downloads the library again. Only opensymphony depends on the duckdb crate, and CI rebuilds opensymphony anyway, so the extra rebuild covers just libduckdb-sys and duckdb.

Evidence

Local runs of the parent test on macOS (Apple Git 2.50.1):

Git processes started Test time
main 5,211 85.0 s
batched rev-parse and reused facts 3,711 58.5 s
plus the macOS Git resolver 3,559 22.8–26.1 s

Git processes were counted from GIT_TRACE2_EVENT start events.

CI (rust-test, Linux, where there is no trampoline), comparing main run 42529164239 with this PR's run 42533630669:

main this PR
parent execution root test 11.0 s 8.6 s
nextest wall time 89.3 s 83.4 s
summed test time 329.7 s 310.0 s

A single Git call costs 16.6 ms through /usr/bin/git and 6.8 ms through the resolved binary (average of 50 rev-parse calls).

Checks:

  • cargo nextest run --test workspace_manager --lib -E 'binary(workspace_manager) | test(/opensymphony_workspace::/)': 86 passed.
  • cargo fmt --all --check and cargo clippy-dev are clean.

The DuckDB costs in memory code indexing are handled in a separate PR.

Checkout verification spawned one Git process per question. A parent
execution root test started 5,211 Git processes and took 85 s on macOS.

- Batch the repository-layout, shallow-state, HEAD, and branch queries in
  verify_git_checkout into two rev-parse calls, and the integration
  checkout root, common dir, and HEAD into one.
- Return the verified Git facts to parent preparation and retained-child
  validation so they no longer re-run status, branch, and rev-parse HEAD.
- Resolve the real Git binary once on macOS when git on PATH is the
  /usr/bin/git xcrun trampoline, removing about 10 ms of developer-dir
  lookup from every workspace and code-index Git process.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-08T22:41:27.414204Z 3cfe5a6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

rust-cache prunes target/duckdb-download before saving, so the
cache-directories entry never reached the cache and restored builds
failed to link against -lduckdb. Remove the libduckdb-sys build-script
output when the download directory is missing so the build script
downloads the library again.
The restored cache keeps target/duckdb-download but not libduckdb.so, so
the directory check never triggered the rebuild.
A restored cache never holds the downloaded library, so the check only
risked missing a stale version directory.
@kumanday
kumanday merged commit d3046a7 into main Nov 9, 2026
7 checks passed
@kumanday
kumanday deleted the perf/workspace-checkout-verification branch November 9, 2026 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant