Repository navigation
perf(workspace): cut Git process count in checkout verification - #261
Merged
Merged
Conversation
Checkout verification spawned one Git process per question. A parent execution root test started 5,211 Git processes and took 85 s on macOS. - Batch the repository-layout, shallow-state, HEAD, and branch queries in verify_git_checkout into two rev-parse calls, and the integration checkout root, common dir, and HEAD into one. - Return the verified Git facts to parent preparation and retained-child validation so they no longer re-run status, branch, and rev-parse HEAD. - Resolve the real Git binary once on macOS when git on PATH is the /usr/bin/git xcrun trampoline, removing about 10 ms of developer-dir lookup from every workspace and code-index Git process.
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
rust-cache prunes target/duckdb-download before saving, so the cache-directories entry never reached the cache and restored builds failed to link against -lduckdb. Remove the libduckdb-sys build-script output when the download directory is missing so the build script downloads the library again.
The restored cache keeps target/duckdb-download but not libduckdb.so, so the directory check never triggered the rebuild.
A restored cache never holds the downloaded library, so the check only risked missing a stale version directory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Checkout verification asked Git one question per process.
workspace_manager::parent_execution_root_reuses_three_repositories_and_preserves_childrenstarted 5,211 Git processes. It took 85 s on macOS and about 10 s on Linux CI. This PR keeps every check and makes fewer, cheaper Git calls.Batched
rev-parseinverify_git_checkout. One call now answers all the repository-layout questions: work-tree status, top level, git dir, common dir, objects path, grafts path, and shallow state. A second call returnsHEADand--symbolic-full-name HEAD. Before, these took nine processes. A detached HEAD printsHEAD, so the branch comes out empty, as it does withgit branch --show-current. A new helper,rev_parse_lines, requires exactly one line per answer; any other output fails verification.Batched
rev-parseinverify_parent_integration_checkout. One call now returns the top level, common dir, andHEAD.Verified state is passed to callers. Parent preparation and
validate_parent_retained_checkoutsnow get the branch, HEAD, and cleanliness that verification already observed. Before, they re-ranstatus,branch --show-current, andrev-parse HEADafter it. The checks they make are unchanged.The real Git binary on macOS. When
gitonPATHis the/usr/bin/gitxcruntrampoline,git_program()resolves the real binary once withxcrun --find git, which is the same lookup the trampoline performs. That removes about 10 ms from every Git process. The workspace manager and the memory code indexer both use it. On other platforms and otherPATHlayouts, Git is still spawned throughPATH.CI fix for DuckDB linking. Before saving, rust-cache prunes the library under
target/duckdb-download, so thecache-directoriesentry added in test: cut slow Rust test wall time #260 never carried it. Thelibduckdb-sysbuild-script output is still cached, and on a cache hit it points the linker at the missing library, so the first runs of this PR failed to link against-lduckdb. Therust-testjob now removes that build-script output after restoring the cache, and the build script downloads the library again. Onlyopensymphonydepends on theduckdbcrate, and CI rebuildsopensymphonyanyway, so the extra rebuild covers justlibduckdb-sysandduckdb.Evidence
Local runs of the parent test on macOS (Apple Git 2.50.1):
mainrev-parseand reused factsGit processes were counted from
GIT_TRACE2_EVENTstartevents.CI (
rust-test, Linux, where there is no trampoline), comparingmainrun 42529164239 with this PR's run 42533630669:mainA single Git call costs 16.6 ms through
/usr/bin/gitand 6.8 ms through the resolved binary (average of 50rev-parsecalls).Checks:
cargo nextest run --test workspace_manager --lib -E 'binary(workspace_manager) | test(/opensymphony_workspace::/)': 86 passed.cargo fmt --all --checkandcargo clippy-devare clean.The DuckDB costs in memory code indexing are handled in a separate PR.