Repository navigation
Learned SNARC: research architecture and training PRD - #6
Conversation
…trained result) The bridge note said "a simple detector at the right choice boundary produced +0.165". The source (dev-SAGE laya FINDINGS section 15, turn_detector.py) says +0.165 is the headroom under PERFECT turn detection; the trained detector scored 0.461 against the 0.596 repeat floor (-0.136). In SAGE, also fix the relative link to forum/claude/SAGE/SNARC_SPECIFICATION.md (one level too shallow). Seat: legion-claude Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
dp-web4
left a comment
There was a problem hiding this comment.
Disclosure (for dp, not a fix):
- This repo is public, and the bridge note carries private dev-SAGE research results:
- Thor's Laya measurements, the 30k-vs-421M probe, and the lived-role retrain;
- ft09 game behavior;
- links into dev-SAGE.
- dev-SAGE's CLAUDE.md says "Don't reference dev-SAGE from public SAGE commits or docs". SAGE's CLAUDE.md lists game-specific results and metrics as content that must not leak.
- An open public PR already shows its diff. If the disclosure is intended, ignore this. If not, drop the bridge note (and the snarc PRD's §16) from the public PRs and link to "private research" instead.
Fixed on this branch (seat commit, at dp's request): the bridge note claimed "a simple detector at the right choice boundary produced +0.165". The source (Laya FINDINGS §15, turn_detector.py) says the opposite. +0.165 is the gain with a perfect (oracle) turn detector. The trained detector scored 0.461 against the 0.596 repeat floor (−0.136), worse than not splitting. E2 depended on that sentence.
Not changed, flagged:
- E0's motivating case may not need learning. ft09's loop was already reporting
exhausted: Trueat step 15, and a one-line rule that reads that flag would have stopped it. So the deterministic detector is the arm the learned SNARC has to beat. On that trace it may simply win, which would be a useful result. - The 30k vs 421M comparison needs context. Both scores (0.299 and 0.285) are below the 0.590 "repeat the last action" floor. The source also marks the 421M figure provisional (3 of 6 epochs, 1 of 4 arms). The note drops both.
- The same bridge note is in three repos, word for word. Copies like that drift. I'd keep one canonical copy and link to it from the others.
- Cross-repo links point at the
exploration/learned-snarc-2026-10-10branch. They break once it is deleted after merge. I'd point them atmain.
What I checked and found correct:
- Every other figure matches its dev-SAGE source: 31.8 ms, 859 ms CPU, 86–88 ms for 5 questions, 0.7660 on 400 cases, ECE 0.2132 vs 0.1747, 245,404 rows / 9,408 steps, 0.8490 vs 0.8654, and step 15 / 135 actions.
- All six SAGE file paths exist, and
snarc_scorer.pyreally is annn.Module. - The snarc README and crossfeed audit cited exist.
- R6 matches Web4's phrasing.
- The plan's discipline is right: advisory only with Hestia holding authority, no circular or future labels, shadow mode before live use, and "done" meaning runnable code.
Verdict: hold for dp's disclosure call; otherwise a sound research PRD.
— legion-claude
|
Legion's review is valid. Dennis accepts the current disclosure as an exception and directs us not to repeat it. Future public changes require checking the confidentiality instructions first. Thank you for the review. |
Adds standalone exploration and full PRD. Audits existing heuristic + SAGE neural SNARC, defines event/decision interfaces, outcome labels, safety boundaries, evaluation arms, staged training, shadow rollout and acceptance gates.