Withdraw the determinism claim about the chorus collapses, and record why they collapse - #668
Merged
Merged
Conversation
… why they collapse The page said 111 of chorus_norm's inner fits collapse in both draws and read that as the failure being deterministic given configuration, seed and fold pair. It is not evidence of that. The two draws share their configurations, seeds and fold pairs and differ only in recordings, so the per-configuration collapse rates alone predict an overlap of 112.6 against the 111 observed, and 40.4 against 38 for chorus_gain_norm. Computed here from the chorus-collapse branch's own collapse_table.json. Round 2 of that branch's murderboard found the same misreading in the replicate report. In its place, the diagnosis: at a learning rate of 0.03 the head is never woken, 292 of 396 inner fits at that rate against 7 of 468 below it, every collapsed fit with a silent head layer and no working fit with one, and the same fit training when replayed behind a warm-up. Its source PR is marked for what it is — landed with four open murderboard findings and round-2 repairs that were never blind-verified. The merge-gap rerun's line drops "needs no retraining", which this page cannot source, and says where the rerun is instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hQX6iESxBsQk7Jan875e3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The goal page merged in #664 said 111 of
chorus_norm's inner fits collapse in both draws, and read that as the failure being "deterministic given configuration, seed and fold pair, not noise." Round 2 of the chorus-collapse murderboard (#667) found the same misreading in the replicate report. It is onmainin the goal page too, so it is withdrawn here.Why it is wrong. The two draws share their configurations, seeds and fold pairs and differ only in recordings. So the per-configuration collapse rates predict an overlap on their own, with no reproducibility needed:
chorus_normchorus_gain_normComputed from
docs/learned/chorus_collapse/collapse_table.jsonon thechorus-collapsebranch, pooling each configuration's rate across both draws. The observed overlap is what chance gives. Whether a given collapse is reproducible is a questiondiagnose_chorus_collapse.py replaycan answer, and nobody has asked it.In its place, the diagnosis. At a learning rate of 0.03 the head is never woken: 292 of 396 inner fits at that rate collapse against 7 of 468 below it; every collapsed inner fit has a head layer passing nothing that varies (153 of 153, 75 of 75) and no working fit does; and the same fit, replayed bit-exactly, trains at 0.01 or 0.003 or behind a 200-step warm-up. Tuning picked no lr-0.03 configuration for
chorus_normunder either selection, so goal 2 paid in effective grid size rather than in a wrong answer —chorus_gain_normbeing the exception, at 6 of 8 folds on F1 alone.The source is marked for what it is: #667 landed at Tony's call with four open murderboard findings and round-2 repairs that were never blind-verified. Its readout and todo arrive with it, so those two links are noted as pending rather than presented as live.
One more line changed. The merge-gap rerun's entry dropped "and it needs no retraining" — this page cannot source that, and 064 is reporting GPU contention on the rerun — and now says where the rerun is instead.
Gates: sapper clear,
check_quotesclear,check_pipelinesclear. Docs only, one file.🤖 Generated with Claude Code
https://claude.ai/code/session_017hQX6iESxBsQk7Jan875e3
Generated by Claude Code