Repository navigation
Publish each run's metrics to the DST dashboard - #457
Conversation
After the post-run scrape, post the same metrics and window to the dashboard, which keeps them past the lab's six-month retention. A dataset that comes back empty fails the publish and the experiment is removed again. publish_run publishes existing run folders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every run records the repo, commit and branch in metadata.json, and whether the code was clean and pushed. Runs are never refused. Only runs from clean, pushed code go to the dashboard; the others are skipped with the reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AlbertoSoutullo
left a comment
There was a problem hiding this comment.
Minor comment. If creation succeeds but the datasets request fails, the experiment remains without being verified. On retry, the 409 is treated as success, so we never check whether its datasets are complete. Could we verify the existing experiment on 409, or remove it when verification fails so a retry can recreate it?
A 409 used to count as published, so a run whose row check never ran (the check failed after the create) stayed unverified. Now a 409 finds the experiment by title and runs the same check; an incomplete one is removed so the next retry recreates it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
True, thanks! A 409 now looks the experiment up by title and runs the same row check, so a retry either confirms it or removes it if a dataset is empty. |
After the post-run scrape, nimlibp2p and logos-messaging runs post the same metrics and window to the DST dashboard when
DST_DASHBOARD_TOKENis set.python -m src.analysis.publish_run <run folders>publishes existing runs.