Repository navigation
Conversation
…ables The slicing performance-over-time plot pooled all five slice sizes into each point, mixing runtimes that differ several-fold and making the values incomparable with the method-ranking heatmap. It now uses the largest slice. generate_figures also writes performance_over_<benchmark>_changes.csv tables comparing each method and modality between consecutive environment timepoints, with absolute, percent, and fold changes in runtime plus the corresponding downloaded bytes and web packets from the network tracking runs. Rows are sorted by the size of the change, with sub-10 ms cached reads placed last. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Panel titles showed raw facet variables (e.g., "is_preloaded = False | modality = Ecephys") and the legend title showed the internal column name "benchmark_name_clean". Panels are now titled by modality, with "first read" or "repeated read" for the slicing plot, and the legend is titled "Method". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
performance_over_time_remote_slicing.pdf) previously pooled all five slice sizes into each point. This mixed runtimes that differ several-fold, widened the error bars, and made the values incomparable with the method-ranking heatmap, which uses the largest slice. The plot and its summary table now use only the largest slice, and the figure caption says so.generate_figuresnow also writesperformance_over_time_remote_file_reading_changes.csvandperformance_over_time_remote_slicing_changes.csvto the summary-tables directory. Each row compares one method and modality between two consecutive environment timepoints, with:These tables back the software-version analysis in the NWB benchmarks paper.
Testing
nwb_benchmarks generate_figures --output-dir figures_benchmark --summary-tables-dir figures_benchmark_summary_tables. Apart from the slicing-over-time summary, all existing summary CSVs regenerated byte-identical.pytest tests/test_incremental_visualization.pypasses, and so do the pre-commit hooks.🤖 Generated with Claude Code