Repository navigation
Conversation
…nchmark UC-03 for NumPy. Two revisions are built with `spin`, each in a git worktree of its own, and one of NumPy's own asv benchmarks (bench_function_base.Mean.time_mean, unchanged) is run on each, alternated round by round, then compared with `bx compare --run --profile revisions`. The sides are selected by a per-side PYTHONPATH, one `bx bench` call at a time, and compare.sh checks that each side imports its own build: the asv adapter does not put the target on the benchmark's sys.path, and benchx does not record PYTHONPATH. The README says so. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…onf for asv and pyperf compare.sh no longer makes a git worktree per revision. Each revision is checked out (with its submodules) in the NumPy checkout and built there by `spin build -C`: the checked-out code in `build`, any other commit in `build-<short sha>`. The checkout is restored when the script ends, and the script refuses a tree with uncommitted changes. It takes `[-t TEST]... [BASELINE [CONTENDER]]` as `spin bench --compare` does, with no default benchmark, and prints which numpy each call imports. The comparison ends with `bx compare --tables`: a table of each commit and a table of the changes, as `asv continuous` prints them. The script passes `--adapter asv --suite benchmarks/benchmarks`, and benchx.conf.json has `adapter.name: pyperf` and only the timing, so the same file serves a pyperf suite. This needs a benchx newer than 0.1.0: `--tables`, and a pyperf conf serving `--adapter asv`, are not in the PyPI release. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The sample output is regenerated from the same real v2.4.0 vs v2.4.3 run, with the per-benchmark tables indented by two spaces as `bx compare --tables` now prints them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cross-ref benchx-dev/benchx#117