Repository navigation
A lit shelf for the training-failure literature, with both papers filed - #681
Merged
Merged
Conversation
#679 withdrew the chorus-collapse page's novelty claim on the strength of Kosson et al. 2024 and Lu et al. 2020. Both were read and cited; neither was filed. The shelf's own README says a PDF with no entry is indistinguishable from one someone downloaded and forgot, and these two decided something that is live on main. New subfolder rather than a stretched one: DL/ is learned detectors and set-structured inputs, ml/ is the unlabelled detector's claims, and neither covers why training fails early at a large step size. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lit/optimization/ now holds Kosson et al. 2024 and Lu et al. 2020 with a README entry each, in the shelf's own form: author, year, where it came from, and which decision it bears on. Both PDFs were fetched by hand from arXiv and verified by extracting the first page after the copy, not by file size. The entries carry the distinction that made #679's ruling, so a later session re-reading them does not have to redo it. Kosson's units are alive and the update kills them, counted at the end of training, and his remedy is a leaky rectified-linear unit - which is itself the evidence that the mechanism is ReLU's exact zero region. Lu's born-dead networks are dead before training and his theorem sends them to a constant function, and his depth-up / width-down ordering matches Table 1 in all four comparisons at lr 0.03. Neither transfers to a GELU, which has no zero region, so the shelf says plainly that the ordering match is a match in direction and not evidence that his mechanism operates here. The README also records what these papers do NOT settle, including the one search nobody has run: whether a learned event detector collapsing to a single call covering a whole recording has been reported in the imaging literature. That is the remaining way the page's narrowed claim could still turn out to be old. One correction carried forward: role 2's archived report cites Lu's constant-function result as Theorem 3.4; in the text it is 3.7. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Files the two papers that #679 used to withdraw the chorus-collapse page's novelty
claim. They were read, acted on and cited publicly — and filed nowhere, which is the
exact failure
lit/README.mdexists to prevent:New shelf rather than a stretched one.
DL/is learned detectors and set-structuredinputs;
ml/is the unlabelled detector's claims. Neither covers why training fails inits first few hundred steps, which is what these two are about and what goal 2 will keep
running into.
<darkroom>/bugarach/lit/optimization/now holds:kosson_2024_warmup_update_size.pdflu_2020_dying_relu_initialization.pdfREADME.mdPlus one row in
lit/README.md's subfolder table. Nothing else underlit/touched —the shelf is new, so it cannot collide with the ACTIVE
lit/radar/claim.The entries carry the distinction, not just the citation, so a later session re-reading
them does not have to redo #679's ruling: Kosson's units are alive and the update kills them
(counted at end of training, remedy a leaky ReLU — itself the evidence the mechanism is
ReLU's exact zero region); Lu's are dead before training and his theorem sends them to a
constant function, with a depth-up / width-down ordering that matches Table 1 in all four
comparisons at lr 0.03. Neither transfers to a GELU, so the README says plainly that the
ordering match is a match in direction, not evidence that his mechanism operates here.
It also records what the papers do not settle — including the one search nobody has run:
whether a learned event detector collapsing to a single call covering a whole recording has
been reported in the imaging literature. That is the remaining way the page's narrowed claim
could turn out to be old.
Verification: both PDFs fetched by hand from arXiv (
fetch_paper.pyis deliberately notvendored — SAP004), and each checked after the copy by extracting its first page, not by file
size. Venues confirmed independently: Kosson from the arXiv comment field ("Accepted to
NeurIPS 2024"), Lu from its DOI.
One correction carried forward: role 2's archived report cites Lu's constant-function result
as Theorem 3.4; in the text it is 3.7. Nothing shipped wrong — the page carries no theorem
number — but the archive does.
🤖 Generated with Claude Code