Skip to content

A lit shelf for the training-failure literature, with both papers filed - #681

Merged
syncytium2 merged 2 commits into
mainfrom
lit-optimization
Sep 21, 2026
Merged

syncytium2 merged 2 commits into
mainfrom
lit-optimization

Conversation

@syncytium2

Copy link
Copy Markdown
Owner

Files the two papers that #679 used to withdraw the chorus-collapse page's novelty
claim. They were read, acted on and cited publicly — and filed nowhere, which is the
exact failure lit/README.md exists to prevent:

A PDF with no entry is indistinguishable from a PDF someone downloaded and forgot.

New shelf rather than a stretched one. DL/ is learned detectors and set-structured
inputs; ml/ is the unlabelled detector's claims. Neither covers why training fails in
its first few hundred steps
, which is what these two are about and what goal 2 will keep
running into.

<darkroom>/bugarach/lit/optimization/ now holds:

file
kosson_2024_warmup_update_size.pdf Kosson, Messmer, Jaggi — NeurIPS 2024, arXiv:2410.23922, 22 pp
lu_2020_dying_relu_initialization.pdf Lu, Shin, Su, Karniadakis — Commun. Comput. Phys. 2020, doi:10.4208/cicp.OA-2020-0165, 34 pp
README.md both entries, in the shelf's author / year / provenance / which-decision form

Plus one row in lit/README.md's subfolder table. Nothing else under lit/ touched —
the shelf is new, so it cannot collide with the ACTIVE lit/radar/ claim.

The entries carry the distinction, not just the citation, so a later session re-reading
them does not have to redo #679's ruling: Kosson's units are alive and the update kills them
(counted at end of training, remedy a leaky ReLU — itself the evidence the mechanism is
ReLU's exact zero region); Lu's are dead before training and his theorem sends them to a
constant function, with a depth-up / width-down ordering that matches Table 1 in all four
comparisons at lr 0.03. Neither transfers to a GELU, so the README says plainly that the
ordering match is a match in direction, not evidence that his mechanism operates here.

It also records what the papers do not settle — including the one search nobody has run:
whether a learned event detector collapsing to a single call covering a whole recording has
been reported in the imaging literature. That is the remaining way the page's narrowed claim
could turn out to be old.

Verification: both PDFs fetched by hand from arXiv (fetch_paper.py is deliberately not
vendored — SAP004), and each checked after the copy by extracting its first page, not by file
size. Venues confirmed independently: Kosson from the arXiv comment field ("Accepted to
NeurIPS 2024"), Lu from its DOI.

One correction carried forward: role 2's archived report cites Lu's constant-function result
as Theorem 3.4; in the text it is 3.7. Nothing shipped wrong — the page carries no theorem
number — but the archive does.

🤖 Generated with Claude Code

defazio2 and others added 2 commits September 21, 2026 08:10
#679 withdrew the chorus-collapse page's novelty claim on the strength of Kosson
et al. 2024 and Lu et al. 2020. Both were read and cited; neither was filed. The
shelf's own README says a PDF with no entry is indistinguishable from one someone
downloaded and forgot, and these two decided something that is live on main.

New subfolder rather than a stretched one: DL/ is learned detectors and
set-structured inputs, ml/ is the unlabelled detector's claims, and neither
covers why training fails early at a large step size.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lit/optimization/ now holds Kosson et al. 2024 and Lu et al. 2020 with a README
entry each, in the shelf's own form: author, year, where it came from, and which
decision it bears on. Both PDFs were fetched by hand from arXiv and verified by
extracting the first page after the copy, not by file size.

The entries carry the distinction that made #679's ruling, so a later session
re-reading them does not have to redo it. Kosson's units are alive and the update
kills them, counted at the end of training, and his remedy is a leaky
rectified-linear unit - which is itself the evidence that the mechanism is ReLU's
exact zero region. Lu's born-dead networks are dead before training and his
theorem sends them to a constant function, and his depth-up / width-down ordering
matches Table 1 in all four comparisons at lr 0.03. Neither transfers to a GELU,
which has no zero region, so the shelf says plainly that the ordering match is a
match in direction and not evidence that his mechanism operates here.

The README also records what these papers do NOT settle, including the one search
nobody has run: whether a learned event detector collapsing to a single call
covering a whole recording has been reported in the imaging literature. That is
the remaining way the page's narrowed claim could still turn out to be old.

One correction carried forward: role 2's archived report cites Lu's
constant-function result as Theorem 3.4; in the text it is 3.7.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@syncytium2
syncytium2 merged commit 921b27f into main Sep 21, 2026
3 checks passed
@syncytium2
syncytium2 deleted the lit-optimization branch September 21, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants