Add non-record 10min/16MB submission: Wavelet-Lite PR549 Parallel Muon (1.1483)

[bro4all]

Summary

This PR adds a non-record track_10min_16mb submission under:

• records/track_10min_16mb/2026-03-24_WaveletLite_PR549_ParallelMuon/

The submission is a PR #549-derived Parallel Muon stack with one architectural change: a tiny causal wavelet-lite mixer inside each residual block.

Final result

• Exact saved-artifact roundtrip: val_bpb=1.14825550
• Total submission size: 15,859,711 bytes
• Margin under 16,000,000-byte cap: 140,289 bytes
• 8xH100 training pace: 90.24 ms/step
• Post-EMA diagnostic at train-time wallclock cap: val_bpb=1.1400

Why submit as non-record

This does not beat the current SOTA, so this is intentionally submitted as a non-record run under the standard 10min/16MB track.

Why it is not duplicate work

Closest prior work is PR #549, but this submission adds a new in-block causal wavelet mixer and removes TTT from the final run while trimming the bigram table to fit the byte budget.

Additional nearby prior work addressed in the README:

• PR Add WaveletWeightedWidenet submission directory with README and metadata #211 WaveletWeightedWidenet
• PR non-record:10Layer + BigramHash+ SWA + Attention-Residuals #632 Attention-Residuals
• PR Record: 1.1558 BPB — 11L U-Net + Catalytic + SwiGLU + SW64 #507 11L U-Net + Catalytic + SwiGLU + SW64
• PR Non-record: Basis Block Interpolation (novel negative result) + Hyperparameter Sweep (MATRIX_LR=0.03 improves SOTA by 0.059 bpb) #530 Basis Block Interpolation

Included files

Per the repo submission rules, this PR only adds a new folder with:

• README.md
• submission.json
• train_gpt.py
• final_model.int6.ptz
• training log
• exact roundtrip eval log
• local results.tsv snapshot

Notes

• The final artifact and exact roundtrip eval were recovered from a persisted full-precision checkpoint after the training pod exited, and both logs are included in the submission folder.
• The README documents the full Runpod recipe, nearest prior PRs, exact structural differences, and failed intermediate attempts.

[MatoTeziTanka]

Community Review — Add non-record 10min/16MB submission: Wavelet-Lite PR549 Parallel Muon (1.1483)

BPB: 1.1483 | Compliance: LOOKS CLEAN — score-first-per-chunk TTT (legal #1416/#1423 pattern)

What I found in the code (head SHA c7ff71b9def0, file records/track_10min_16mb/2026-03-24_WaveletLite_PR549_ParallelMuon/train_gpt.py):

The TTT path at line 1106 implements the score-first-per-chunk pattern: each chunk is scored under torch.no_grad() / inference_mode() before the base_model.train() + SGD adaptation runs on that same chunk, with an is_last_chunk guard so the final chunk gets no adaptation pass. This is the structural shape the legal frontier uses (PRs #1416 erichroepke, #1423 aryanbhosale).

Per Issue #402 and Issue #677, TTT is legal when each token is scored before the adapter updates on it, and that's what the code does here — chunk ci is scored under weights adapted only on chunks 0..ci-1. No prequant_ttt_adapt_adamw(val_tokens, ...) multi-epoch fine-tune, no scored-region SLOT, no target-in-key n-gram cache.

CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.05s, dim=512, layers=11, vocab=1024, code=91471 B, SMOKE_TEST_PASS

Verdict: LOOKS CLEAN.

Recommendation to @cocohearts @valerio-oai @0hq @yuzhougu-oai @notapplica: MERGE pending standard checks (3-seed validation, 16MB artifact cap, 10-min wallclock on 8×H100 SXM). The compliance picture matches the legal reference frontier and no flags were raised by the classification pass.

Auto-classification caveat: this review was drafted by the AST-based classifier against a template derived from manually-reviewed cluster PRs (#1420, #1450, #1487, #1541, #1529, #1533, #1518). If I've misread a subtlety in your eval path — e.g., multi-epoch TTT that I mistook for single-pass, or a target-in-key lookup I missed in a helper function — please flag it and I'll re-run the audit manually.

---

Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.05s, dim=512, layers=11, vocab=1024, code=91471 B, SMOKE_TEST_PASS. Classification via deterministic AST-based classify_prs.py (pattern bank derived from ~65 manually-reviewed PRs earlier in the 2026-04-11 sweep). This review was auto-drafted from a template and spot-checked before posting — if the template misread your code, please call it out so I can iterate the classifier.