Skip to content

Publish each run's metrics to the DST dashboard - #457

Merged
radiken merged 3 commits into
masterfrom
alan/dashboard-publish
Nov 17, 2026
Merged

radiken merged 3 commits into
masterfrom
alan/dashboard-publish

Conversation

@radiken

@radiken radiken commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

After the post-run scrape, nimlibp2p and logos-messaging runs post the same metrics and window to the DST dashboard when DST_DASHBOARD_TOKEN is set. python -m src.analysis.publish_run <run folders> publishes existing runs.

radiken and others added 2 commits November 6, 2026 14:46
After the post-run scrape, post the same metrics and window to the dashboard,
which keeps them past the lab's six-month retention. A dataset that comes back
empty fails the publish and the experiment is removed again. publish_run
publishes existing run folders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every run records the repo, commit and branch in metadata.json, and whether
the code was clean and pushed. Runs are never refused. Only runs from clean,
pushed code go to the dashboard; the others are skipped with the reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@radiken radiken self-assigned this Nov 10, 2026
@radiken radiken added the enhancement New feature or request label Nov 10, 2026
@radiken radiken linked an issue Nov 10, 2026 that may be closed by this pull request
@radiken
radiken marked this pull request as ready for review November 10, 2026 21:01

@AlbertoSoutullo AlbertoSoutullo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment. If creation succeeds but the datasets request fails, the experiment remains without being verified. On retry, the 409 is treated as success, so we never check whether its datasets are complete. Could we verify the existing experiment on 409, or remove it when verification fails so a retry can recreate it?

A 409 used to count as published, so a run whose row check never ran
(the check failed after the create) stayed unverified. Now a 409 finds
the experiment by title and runs the same check; an incomplete one is
removed so the next retry recreates it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@radiken

radiken commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Minor comment. If creation succeeds but the datasets request fails, the experiment remains without being verified. On retry, the 409 is treated as success, so we never check whether its datasets are complete. Could we verify the existing experiment on 409, or remove it when verification fails so a retry can recreate it?

True, thanks! A 409 now looks the experiment up by title and runs the same row check, so a retry either confirms it or removes it if a dataset is empty.

@radiken
radiken merged commit ed2f272 into master Nov 17, 2026
2 checks passed
@radiken
radiken deleted the alan/dashboard-publish branch November 17, 2026 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

General tooling (recurring)

2 participants