Skip to content

Included the re-hashed Python 3.13 environment in the 2026-10-07 timepoint - #225

Merged
CodyCBakerPhD merged 2 commits into
mainfrom
merge_equivalent_environments
Nov 12, 2026
Merged

CodyCBakerPhD merged 2 commits into
mainfrom
merge_equivalent_environments

Conversation

@oruebel

@oruebel oruebel commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Motivation

The three incremental slicing runs added in NeurodataWithoutBorders/nwb-benchmarks-results#14 were missing from the figures. The 2026-10-07 summary tables still reported n_runs=2.

The runs were recorded under a new environment ID, e833ef39…, while the 2026-10-07 timepoint is mapped only to 42bfe711…. The new ID comes from the runtime section that #212 added to environment files. The software did not change. The conda package list in both environment files is identical, and the runtime-reported versions match it.

Changes

  • Added EQUIVALENT_ENVIRONMENT_IDS in database/_processing.py. It maps an alias environment ID to its canonical ID in ENVIRONMENT_TIMEPOINTS.
  • _preprocess_results relabels alias IDs with the canonical ID before any environment filtering. ENVIRONMENT_TIMEPOINTS and the plotting code are unchanged.
  • Added tests/test_equivalent_environments.py.

Verification

  • pytest tests/test_equivalent_environments.py tests/test_excluded_samples.py passes.
  • nwb_benchmarks generate_figures now reports n_runs=5 for every method in incremental_slicing_overview_2026-10-07.csv.

🤖 Generated with Claude Code

The runtime section added to environment files in #212 changed the
environment ID of the LBL Mac environment from 42bfe711 to e833ef39
without any change to the installed packages. Relabel results recorded
under the new ID with the canonical ID so they are included in the
2026-10-07 timepoint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CodyCBakerPhD

Copy link
Copy Markdown
Collaborator

Good enough for now; in the long-term I do think an overhaul of the underlying data models will be worth it

That'll be a fun database migration when that day comes

@CodyCBakerPhD
CodyCBakerPhD merged commit a02d3e8 into main Nov 12, 2026
5 checks passed
@CodyCBakerPhD
CodyCBakerPhD deleted the merge_equivalent_environments branch November 12, 2026 23:46
CodyCBakerPhD pushed a commit to NeurodataWithoutBorders/nwb-benchmarks-dashboard that referenced this pull request Nov 12, 2026
Mirrors nwb-benchmarks' EQUIVALENT_ENVIRONMENT_IDS (NeurodataWithoutBorders/nwb-benchmarks#225):
results recorded under e833ef39 are read as 42bfe711's, so the manuscript view's 2026-10-07 figures
include them. The parity fixture gains one such run, and its expected values are regenerated
against nwb-benchmarks' current main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T
CodyCBakerPhD added a commit to NeurodataWithoutBorders/nwb-benchmarks-dashboard that referenced this pull request Nov 13, 2026
* Set up the NWB Benchmarks Dashboard from the web-app template

Draw the NWB Benchmarks paper's figures (method rankings, traffic vs
runtime, performance across versions, download vs streaming, incremental
slicing) in the browser from the bundle nwb-benchmarks-results publishes,
with every run's metadata as a filter and the paper's selections as
presets. The figure code ports nwb-benchmarks' Python analysis, checked
number by number against it on a fixture by tests/unit/parity.test.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Serve from the default GitHub Pages address instead of a custom domain

Drop public/CNAME and the preview workflow's hard-coded pages-base-url,
so deploys and PR previews use <owner>.github.io/<repo>, which follows
the repository wherever it is transferred.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Align the live-test table after the link change

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Open on every result, with a manuscript view at ?manuscript

The dashboard now starts from all results, for the reader to narrow.
?manuscript (or the "Manuscript figures" button) draws each figure from
the selection nwb-benchmarks' generate_figures uses for the manuscript:
the reference environment for Figures 3, 4a and 5, the dated environments
for Figure 4b and incremental slicing. It replaces the two paper presets
and the per-card "Show as in the paper" buttons.

Environments the paper does not date are labeled by their first run's
date, so a selection across many environments stays readable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Drop the manual user-test checklist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Use the NWB Benchmarks logo for the header, favicon and README

The logo comes from nwb-benchmarks' docs under its BSD-3-Clause-LBNL
license, annotated in REUSE.toml with the license text in LICENSES/. In
dark mode it sits on a light plate so its navy lettering stays legible.
Also pluralizes the data summary's counts ("1 machine").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Drop the extrapolation figure in favor of incremental slicing

The paper replaces its extrapolated Figure 4a with the incremental
slicing figure, so the dashboard drops the extrapolation card, its code,
stories, tests and parity values. The download times the incremental
figure needs move into its module, and the mock bundle loses the local
runs only the extrapolation read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Keep the zero-traffic inset inside its plot, and move the manuscript view to 2026-10-07

The inset's frame now holds its title and tick labels within the panel's plot area instead of
spilling over the x axis, the manuscript view's single-environment figures use the 2026-10-07
environment, and the incremental card no longer refers to the paper's Figure 4a.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Drop the BBQS mark and the footer's What's New, bug and feature links

The page keeps only the CON mark and the version stamp in its footer; the header logo now links
to nwb-benchmarks instead of brain-bbqs.org.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Count the re-hashed Python 3.13 environment as the 2026-10-07 one

Mirrors nwb-benchmarks' EQUIVALENT_ENVIRONMENT_IDS (NeurodataWithoutBorders/nwb-benchmarks#225):
results recorded under e833ef39 are read as 42bfe711's, so the manuscript view's 2026-10-07 figures
include them. The parity fixture gains one such run, and its expected values are regenerated
against nwb-benchmarks' current main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Rework the dashboard around Plotly figures, a key:value search and a summary table

- Figures are drawn with Plotly, configured as dandi/usage-page configures its plots (PNG and SVG
  downloads and a source-data link in the modebar; Plotly Cloud sharing off), each card with a
  Plot | Table view.
- The facet checklists give way to a free-text and key:value search with suggestions, after
  CodyCBakerPhD/website's record search, kept in the URL as ?q=.
- The summary is a table of the selection's totals, with the latest results' date and a refresh
  button that downloads the published results again in place; local bundles are no longer read.
- The manuscript view moves to a header toggle next to the theme's, and only it letters panels;
  figure leads no longer cite the paper's figure numbers.
- The header logo is half as tall again, and the subtitle reads "The NWB Benchmarks results for all
  machines, environments, and time ranges".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Set the subtitle under the title in the header

The subtitle moves into the header's title block, so it sits just under the title instead of below
the whole row the taller logo opened up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Read every record, show the totals bare, and restyle the search

- Results files of every format are read, including those before format 3.0.0: their machine and
  environment come from the older hash fields or the file's name, their positional test cases are
  read by position, and their modality comes from a named case on the same file or the NWB
  object's type. Their slice sizes are ranked apart from the current suite's.
- The totals stand on their own, as on dandi/usage-page, with "Records" in place of "Results
  files", and the last-updated date and the refresh button beside each other under them.
- The search drops its status line: what it could not use, and examples from the data, show in
  its dropdown. The box is a pill after CodyCBakerPhD/website's, quiet in both themes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Explain the manuscript button in a tooltip

Hovering or focusing the header's manuscript button shows what it does and which selections it
draws from, and, while it is on, how to leave it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Tuck each figure's options behind a settings gear, and centre its Plot | Table switch

As on dandi/usage-page, a figure's options (color scale, network metric, benchmark and so on) sit in
a popup under a gear at the card's right, which a click elsewhere or Escape closes. The color
scale's fixed option reads just "Fixed", and the rankings' lead reads "for each modality".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Star and outline the fastest method of each modality, and stand the color scales upright

In the method rankings, the fastest cell of each modality is outlined in the accent color with a
star badge in its corner (on a phone, a star before its value), in place of an asterisk, and each
heatmap's color scale stands vertically beside it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Keep the last results bundle between visits

A return visit draws the dashboard from the copy kept in Cache Storage at
once, then has the browser revalidate its HTTP-cached copy against the
bundle's ETag (raw.githubusercontent.com exposes neither the ETag nor a
preflight for If-None-Match) and redraws only if the bytes' digest changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Stand the traffic legend beside its panels and pad under the modebar

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Draw a kept copy without flashing the loading card

The page now starts without the loading card and shows it only before a
download. The dashboard (and Plotly with it) loads through a dynamic
import, so the small main script runs at once: a return visit draws the
kept copy with no card, and a first visit shows the card immediately and
starts the results download alongside Plotly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Snug each figure's settings gear to the right of its Plot | Table switch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Match each settings gear to its Plot | Table switch's height

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Title the cards as statements, and draw slice-by-slice streaming on a log axis

The incremental slicing figure now defaults to (and lists first) its
logarithmic time axis, labelled at decades only like the traffic figure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Cut the manuscript button's hover text to its first sentence

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

* Check the manuscript tooltip's sentence after leaving the view

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DQrzw3EzjrfYqHpgtbET2T

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants