Why a third of chorus_norm's tuning fits do not train: the head is never woken at lr 0.03 (landed with murderboard flags) - #667
Merged
Conversation
…first steps at lr 0.03 The replicate report found about a third of chorus_norm's inner fits collapse to one call per recording, most of them the same fit in both draws. tools/diagnose_chorus_collapse.py answers why from the runs' own fits: a collapse table joined to configuration, a census of every second-draw fit's head, and an exact replay of any inner fit (bit-identical to its checkpoint) that can change one thing. - lr 0.03: 292 of 396 inner fits collapse; below it, 7 of 468. - 152 of 153 collapsed chorus_norm fits have a dead head layer, against 1 of 279 working ones. The output's spread separates the groups without overlap. - The collapsed fit trains at lr 0.01 or 0.003, or at 0.03 with a 200-step warm-up; a 200-step warm-up rescued 7 of 8 other collapsed fits. - Not PR #596's "starts deaf" failure. Tuning never chose a lr-0.03 configuration, so goal 2's cost is a smaller effective grid. The repair is left as a decision (todo filed). The readout and its data live in docs/learned/chorus_collapse/; a test rebuilds the page from them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…wn, the mechanism measured by what passes Eleven reviewers found no wrong number and several wrong inferences. The page now: - opens on the problem: a collapsed fit's flat output against a working fit's, on a recording held out from both, with calls and planted events in the lane above; - counts a head layer silent when no unit's output varies (142 of 153 collapsed chorus_norm fits, 0 of 279 working), replacing a sign test that GELU's negative dip defeats; - separates this failure from PR #596's with a test that can fail: the head's input still separates events in collapsed fits (median d 1.84) while their output does not (0.13); - shows the head goes silent after the loss has stalled, so silence marks a collapse rather than starting it; - compares the between-draw overlap with chance (111 vs 111.5 expected), tests seed and fold pair inside configurations, counts one warm-up run once (6 of 7 train, 1.4 expected by chance), and excludes the untuned default's refits from what tuning chose; - scopes the learning-rate story to chorus_norm (chorus_gain_norm's encoder shape matters as much); - cites the known failure class and remedy, and names Adam. The todo is renamed and corrected, the INDEX row narrowed, glossary entries added, and the eleven role reports archived verbatim in docs/reviews/chorus-collapse_2026-09-19-roles/. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…st that can fail, a matched pair A blind round of all eleven roles found deeper faults than round 1: - the "not PR #596's failure" test was scale-free, and untrained plain chorus passed it (role 4); - zero padding made constant layers "vary" at a recording's ends (role 6); - Figure 1's two fits differed in fold pair, not only in seed (roles 1, 3, 4, 6). Repaired: - the census runs on a fresh recording, and every variation measure skips 400 frames at each end. Every collapsed inner fit now has a silent head layer (153 of 153, 75 of 75) and no working fit does; - #596's own test (vote change for one onset), with untrained controls: most collapsed chorus_norm fits' votes still respond, while 4 of 153 are as deaf as plain chorus (19 of 75 in chorus_gain_norm); - Figure 1 is now a pair that differs only in seed, and all 14 replays were rerun: every head starts mostly silent, and the working fit's wakes by step 50 while the collapsed fit's never does. This reverses round 1's claim that silence follows the stall; - what tuning chose is split by selection rule (chorus_gain_norm picked lr 0.03 only on F1 alone), plus the seed test per draw and across draws, the null relabelled, the fold structure and configuration definitions corrected, citations corrected, and the order reworked. Blocking findings went 3 -> 4, so per the process the run stops and is escalated; the round-2 repairs are not yet blind-verified. Run record: docs/reviews/chorus-collapse_2026-09-19.md. The branch also merges main, so the fair-comparison report the page links to is present. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…its flags Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…se page rebuilds identically on Python 3.11 random.shuffle and random.sample draw different sequences on 3.11 and 3.12+ from the same seed, so CI's 3.11 leg rebuilt p = 0.138 where the committed page said 0.153. numpy's legacy RandomState is stable across versions. Conclusions unchanged; the page, the darkroom copy and the record's artifact hash are updated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…uilds identically on Python 3.11 Python 3.12 changed sum() over floats to compensated summation, so near-ties in the permutation tests resolved differently on 3.11 and CI's rebuild moved p values in the last digit. The clustering statistic and the carry-over correlation now sum integers (the same ranking, scaled), and the float sums use math.fsum. Verified: the page rebuilds byte for byte under 3.11.16 and 3.14. The seed test's p reads 0.002 where float ties had given 0.001; nothing else moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
syncytium2
pushed a commit
that referenced
this pull request
Sep 19, 2026
…s which The replicate's selections are still computing; a page of one draw is honest if it says so, and a page that cannot build until both are in is a page nobody sees tonight. Also: the goal page's chorus-collapse readout and todo are on main now, not arriving with #667. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
syncytium2
added a commit
that referenced
this pull request
Sep 20, 2026
…and the check bounds both sides (#671) * The fair comparison's readout: no net is ahead of tuned CoactDetect, and SCE's lead is the merge-gap artifact Goal 2's run on WSMIP064 (2026-09-18 16:14 to 2026-09-19 05:58, 1,963 jobs, 0 errors). Its summaries are committed beside a report written for a reader new to the project, with eight inline-SVG figures that explain the problem, the bench, the four nets, nested cross-validation, the fold defect on this run's own draws, the two selections and the second draw before any result. The coded search had the context rule but not goal 1's crowded veto, so tools/crowded_check_fair_comparison.py applies it afterwards with goal 1's own scoring (search_all_settings._job over TAIL). CoactDetect's choices pass in all 8 cases; binned SCE's 30 s merge gap fails in 7 of 8, so its first place does not stand. locust has no admissible gated result: the budget refused every candidate in all 4 folds. WSMIP065 raised all three checks. Draft before the murderboard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Evidence behind the fair comparison's report, regenerable: fold draws, the merge-gap ablation, the replicate's numbers tools/fair_comparison_evidence.py, three subcommands, each writing into the run's summary folder: - fold-draws: Figure 4's data from the run's own fit records and, for "before the fix", from bugarach.learn.train.fold_maker. It reproduces the committed fold_draws.json exactly; three murderboard roles found that no committed code had produced it. - merge-gap: the coded detectors tuned their merge gap and the nets' was fixed at 2 s. This rescores every coded choice, and re-decodes every chosen net refit from its saved model, with ONLY the merge gap changed, after first reproducing the run's own scores at each side's own gap. WSMIP065 will run the same code on the second draw. - replicate: the replicate's headline numbers, with source and commit recorded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The fair comparison's report after round 1 of the murderboard: the merge gap reverses the headline All eleven roles ran against the first build. Their reports are archived verbatim in docs/reviews/fair-comparison-2026-09-19-roles/ (paths replaced; originals in the darkroom). The finding that changes the result is role 4's: the coded detectors tuned their merge gap and the nets' was fixed at 2 s. merge_gap.json measures both directions with only that setting changed, after reproducing the run's own scores (coded exactly, nets within 0.0015 F1). At the run's unequal gaps CoactDetect leads the best net by 0.007 F1; with the gap matched, at 2 s or at 8 s, chorus_norm leads. The page now says this run does not settle which side is better, and that a rerun tuning the nets' merge gap needs no retraining. Also from the review: the earlier comparison ran on a different, retired simulator; binned SCE's lead is its 30 s merge (it falls below CoactDetect at 8 s); one SCE choice passes the crowded check; the nets exceeded the budget on held-out data in 25 of 80 refits, now reported beside the coded side's misses; the coded side is scored on pooled training folds, not the inner rotation; the probe rate, the backgrounds (25th and 75th percentiles), the hit rule (spans), the tube description, the distractor construction and the coded detectors' origins are corrected; the contamination limit cites its record and the open participation item; t values carry the Nadeau-Bengio factor with its scope; tables are numbered; figures redrawn; the hand-drawn architecture figure is gone (draughtsman owns it). Glossary: merge gap, crowded-recording check, shared false-alarm budget, admissible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rebuild the fair comparison's report from the clean tree at 2ca264e Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Round 2 of the fair comparison's review, and the evidence it asked for All eleven round-2 reports, archived as they arrived (paths scrubbed; the originals are in the darkroom). Three pieces of evidence they asked for: - The crowded-recording check now also scores goal 1's own reference, the shipped operating points. No verdict changes, and the page can now say so from a file rather than a sentence. - A breakdown of held-out recall by participation level and background. The pooled F1 hides where the two sides differ: the faintest events, 10% of the ROIs, on the busy background go to the nets, and on the quiet one mostly to CoactDetect. - The replicate's refits below 0.2 F1, read by the same rule as this run's. Two failed fits in one replicate fold carry that draw's whole margin against the nets on F1 alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The fair comparison's report after round 2: under the budget CoactDetect leads; on F1 alone the sign is not settled Round 2 of the murderboard found the round-1 headline half right. The merge gap does reverse the order, but only for choices on F1 alone: under the shared false-alarm budget CoactDetect stays ahead of both chorus nets at every matched gap. And the replicate's "agreement" was two refits that failed to train; set aside by one rule in both draws, the replicate's best net leads on F1 alone in 4 of 4 folds while this run's trails in 4 of 4. The page now says that, and a sentence about the data is printed only while it is true: the builder's claim() stops the build otherwise. Also: - the argument reordered: terms first, the method with its rules (the context window, the crowded check, "admissible" as the glossary defines it) before any number, and the merge gap in its own section after the results it qualifies; - new figures: the whole bench recording, how a call is scored (one call, one event), binned against sliding counting, both draws per fold with failed refits drawn twice, matched gaps for both selections, where the two sides differ (the faintest events, by background), and tuning; - attributions corrected: the nets are the Deep Sets shape, binned SCE is not the 2003 rule, CICADA cited by its DOI, SPIKE-synch's root added; - the tube refit's mechanism corrected (its threshold came from the inner fits), the corrected t factor stated exactly, reproduction bounds stated as measured; - a sapper feedback note for SAP004's blindness to backslashes, and the first author of the CICADA framework paper corrected in the README. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rebuild the fair comparison's report from the clean tree at 69480a7 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The fair comparison's report after round 3: every refit counted first, and the headline held to its own uncertainty Round 3 of the murderboard (eleven roles, blind) found no wrong number in the tables and a page that still said more than its evidence. Fixed here: - The answer leads with every refit counted: CoactDetect ahead on average in both draws and both selections. The best net leads only at a matched merge gap, or fold by fold in the replicate once failed refits are set aside, and that set-aside is now called what it is, a check made after seeing the held-out scores. - Under the budget, the matched-gap lead over chorus_norm is as weak as the F1-alone reversal (t -1.5 to -2.1), and the page now says so; the budget is named as CoactDetect's own rates times a declared 1.6. - The confound narrows: untuned chorus_norm is already within 0.02 of the tuned CoactDetect in both draws, so the earlier lead was not lost to tuning the nets. - What was not made equal gets its own section, and the limits add how the nets were trained (the probe and the distractors labeled negative), the three merge rules, normalization over the whole recording, and the tolerance measured for one side only. - The merge-gap concept moves to section 2 as its own figure; the gap curves get a logarithmic axis and a key; the margins figure drops the connecting lines that read as confidence intervals; new figures for the participation breakdown at all three levels and the crowded check; the failed refits of both draws become a table. - Attributions: CoactDetect's null credited to whole-train and per-cell shifts (Pipa 2008; Bocchio 2020; Dard 2022), Grün part II, the guard to Rohling 1983, SPIKE-synch's window to event synchronization and its detection step to Cecchini 2021, "designed here" read as "not found elsewhere yet". - Glossary: call, firing, background, empty recording, refit, the failed-training signature; merge gap and the budget corrected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rebuild the fair comparison's report from the clean tree after round 3 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The fair comparison's report after the final checks: binned SCE's admissible wins under the budget, and the last overreach removed A finding-driven pass over every round-3 finding, and role 10 re-run on the final render, both archived in docs/reviews/fair-comparison-2026-09-19-followup/. The pass caught one error the round-3 fixes introduced: section 9 said the crowded-recording check was never run on the replicate. It was. Read from the replicate's own crowded_check.json and selections, binned SCE under the budget is admissible and ahead of CoactDetect in 3 of 4 replicate folds (+0.011 to +0.017 F1), against 1 of 4 in this run. The page and its lede now say so, behind a claim() guard. Also: the contrast between the two selections is stated as consistency (under the budget every net is behind in every fold of both draws, guarded) rather than margin size, which the replicate contradicted; the scorer-resolution margins are computed; the lede's tuning and faint-event sentences are scoped; stale figure numbers are gone from the SVG labels; the ringed dot means one thing on every figure; Figure 8 says where its no-result marks sit; the axis-break bridge in Figure 11 no longer borrows LoCo's style. And a todo: goal 1's published crowded numbers compare two seed sets. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rebuild the fair comparison's report from the clean tree after the final checks Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The fair comparison's murderboard record: three blind rounds, a finding-driven check and role 10 on the final build Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rebuild the fair comparison's report from the clean tree, and name the shipped build in its record Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The fair comparison's record: the replicate's crowded check was run, and says so Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The report builder parses on Python 3.11 again: no backslash inside a replacement field Figure 3's caption escaped its quotation marks inside an f-string expression, which 3.12 allows and 3.11, the declared floor, does not. tests/test_syntax_floor.py caught it on every leg; the caption now uses typographic quotes and says the same thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rebuild the fair comparison's report after the 3.11 fix, and record the new hash Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The syntax-floor scan sees inside nested f-strings, and the second offence it missed is fixed Figure 10's caption escaped quotation marks inside an f-string that was itself inside another's replacement field. The scan exempted all f-string literal text, including a nested one's, so it passed on 3.13 and 3.14 and the 3.11 leg alone failed. On 3.11 that text is expression, and a backslash there is a SyntaxError. The scan now flags a backslash in a nested f-string's text and a nested f-string reusing the enclosing quote; both new cases are proved to be 3.11 errors (checked on a real 3.11) and proved caught. Every tool and test in the tree parses on 3.11. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rebuild the fair comparison's report after the second 3.11 fix, and record the new hash Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * WIP: the nets' merge gap tuned like any other setting, from the saved fits tools/tune_net_merge_gap.py re-decodes every saved fit of the fair comparison at eight merge gaps (0 to 30 s), re-picks each fit's threshold at each gap by its own rule, and re-selects on the inner fits only: F1 alone and under the shared budget, reported separately, with goal 1's move rule (0.002 inner F1) and its crowded-recording check inside the search. At 2 s it reproduces the run exactly: every picked threshold, row and empty-recording count, and both selections' inner and held-out F1. Pushed mid-run so the work is visible off this machine. The run's crowded pass is finishing; the selections, the replicate and the page (tools/build_net_merge_gap_page.py, not yet built) follow. The tests read committed outputs that are not here yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * tune_net_merge_gap: score only the (gap, threshold) pairs a selection reaches, where the full grid is impractical Four inner fits of one chorus_norm configuration ran more than three hours each on the full crowded grid (36 thresholds x 8 gaps x 24 three-hour recordings) without finishing, while the other 1,934 fits took under a minute each, and the re-chosen walk needs that configuration. select now lists the exact pairs it could not read, crowded --pairs scores only those, and a partial file refuses any cell it does not hold. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * tune_net_merge_gap: close a partial crowded file before replacing it np.load held the old file open, and Windows refuses to replace an open file, so every partial update failed with Access is denied. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The merge-gap page builder: guarded sentences, fold labels as the report's, and the outer-refit crowded check stated Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * This run's draw: the nets' merge gap tuned, both selections All 1,938 fits reproduce the run at 2 s. With the configuration kept, on F1 alone chorus_norm goes from -0.007 to +0.004 against CoactDetect (3 of 4 folds; inside the 0.010 noise scale). Under the budget every net stays behind except chorus_norm in one fold, by +0.004; chorus_gain_norm closes 45% of its shortfall. The nets settle at 5 to 8 s; the crowded check refused every wider choice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The merge-gap page builds from whichever draws are committed, and says which The replicate's selections are still computing; a page of one draw is honest if it says so, and a page that cannot build until both are in is a page nobody sees tonight. Also: the goal page's chorus-collapse readout and todo are on main now, not arriving with #667. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Goal 2's page: the merge-gap rerun is a result, and the crowded check is the finding The Next entry becomes what the rerun found, from this draw's net_merge_gap.json: under the budget CoactDetect still wins 15 of 16 net-folds, tuning closes about 45% of the chorus nets' shortfall, and on F1 alone the +0.004 lead is inside the 0.010 draw-to-draw scale. The no-retraining claim is back, sourced to the tool that re-decodes the saved fits. Stated as its own finding: the crowded check refused a wider gap in all 32 choices, 30 s for every net, so long-gap inflation is not specific to the coded side; and one choice fails that check on the outer refits. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The replicate's draw too, and the page shows both treatments of the failed trainings The replicate's 1,943 fits re-decode and reproduce its run at 2 s. Under the budget CoactDetect is ahead of every net on average in both draws. On F1 alone the sign depends on the chorus nets' collapsed trainings: counting every refit the nets trail in the replicate, and with the refits under 0.2 F1 set aside both chorus nets lead there by more than the draw-to-draw scale. The page reports both, and Figures 1 and 2 scale to the folds that trained rather than to a collapsed one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The merge-gap page, built Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The merge-gap page, rebuilt after the murderboard found four ways it read better than the data Eleven roles, one round, reports archived verbatim. What changed, each because the first draft let a reader conclude more than the numbers support: - The set-aside was applied to the tuned choice and not to the baseline, so the gap was credited with a lead that predates it. Both sides now carry it: in the replicate chorus_norm is already +0.005 ahead at 2 s, and the gap adds 0.010, one noise unit. - Held-out F1 at every gap is now drawn (Figure 1). It peaks at the top of the grid in all 64 folds, so every chosen gap is a boundary the crowded check imposed, not an optimum -- and CoactDetect's 8 s is the top of its own grid. The two landing together is not agreement. - The crowded check's cost to the settings it ACCEPTED is reported beside its refusals: a median 0.012 F1, the same size as the gain, with two choices failing it outright on the outer refits. - The noise scale is named for what its source measures, two draws on two machines, and the page now says that source also measures a systematic +0.015 F1 shift in the replicate's favour -- the size of the lead there. Also: the budget never bound CoactDetect while 44 of 160 net refits break it on held-out data; the threshold is unchanged in 26 of 32 gated choices, so the gain is not the lower-threshold mechanism the draft claimed; failed trainings and no-call refits are counted separately, from the run's own flags; Figure 2 counts refusals instead of stacking 70 crosses; the figures keep an 11 px floor, a readable grey and a gutter for off-scale folds; Table 1 marks within-noise means and failed checks. The tool records the machine, torch and both trees' commits, asserts the threshold grid it indexes, and refuses a move whose crowded score is not finite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Round 2 of the murderboard: five blind roles, and the four things they overturned The crowded cost the page reported is each net against its own 2 s, and the head-to-head on those same recordings runs the other way: the tuned chorus nets beat CoactDetect there by 0.016 to 0.056 F1 while tube loses 0.14. Figure 5 now draws both, and the text no longer lets 'the gain and the cost are the same size' stand in for a comparison it is not. CoactDetect's grid stops at 8 s because goal 1's crowded check refused its 16 s -- so the check bound both sides, at different times, and the page said it had never bound CoactDetect at all. Held-out F1 is largest at the widest gap but not monotonically: it dips in 18 of 64 fold-rows, so 'a wider merge can only delete duplicate calls' was too strong, and the glossary already said why. 'The nets that end ahead were already ahead' was false -- chorus_norm crosses zero in this draw -- and the lede's range covered the chorus nets while saying 'each net'; tube gains a fifth as much. Also: the set-aside cannot be even-handed and says so (CoactDetect has no refits); low refits are counted as refits, not decodings, and split by the run's own two failure kinds; the reproduction is described for what it covers, at 2 s, on one machine for both draws; the empty band below the cut is computed (0.388, not a rounded 0.39); the budget's over-budget count carries its as-run baseline; the grid's coarseness says which way it cuts; panels A and B are drawn at one scale; the table's footnote marker is no longer a degree sign after a number, the crowded-check flag marks every column that uses the flagged gap, and each figure is now referenced by number and name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The run record, the role archives, and the gate that could not read them Two blind rounds, eleven roles then five, every report archived verbatim. The record leads with what the review changed about the result and is delivered UNCONVERGED: blocking findings went 11 then 10, and the process says a flat count means a structural problem patching will not retire, so it escalates instead of running a third round. The roster gate refused a complete archive because reports_decl() strips underscores as markdown emphasis, rewriting net_merge_gap_...-roles/ into a path nobody wrote. The archives are renamed with hyphens and the defect is filed for upstream: a gate whose job is to catch a cited-but-missing archive reported a present one as missing, and the quickest way to green would have been to delete the line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The page, rebuilt so its provenance names the review record that exists Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In both draws of goal 2, about a third of chorus_norm's tuning fits made one call per recording. This PR adds the diagnosis of why: a readout page, the tool that builds it, and the data the page is rebuilt from.
Landed with open murderboard flags, at Tony's call. Two review rounds ran, each with all 11 roles. Blocking findings went 3 → 4, so the process stopped and escalated. The second round's repairs are applied but have not been blind-reviewed.
What it finds
What is in it
tools/diagnose_chorus_collapse.py, with subcommandstable,census,trace,replayandpage.replayre-runs a fit bit-exactly against its checkpoint and can change one thing at a time.docs/learned/chorus_collapse/: the page, plus the JSON it is rebuilt from.tests/test_diagnose_chorus_collapse.py: rebuilds the page byte for byte, checks figure numbering, and checks for personal paths.docs/todo/2026-09-19-chorus-norm-does-not-train-at-lr-0.03.md, which holds the repair decision.docs/reviews/chorus-collapse_2026-09-19.mdand both rounds' role reports.main, so the fair-comparison report the page links to is present.Open flags
train()with hooks and sharing the collapse rule with the replicate report's builder.docs/goals/is untouched (#664 owns it).🤖 Generated with Claude Code