Goal 2: nested-CV tuning of the nets against every-knob coded detectors, on the GPU and the bench - #642
Draft
syncytium2 wants to merge 213 commits into
Draft
Goal 2: nested-CV tuning of the nets against every-knob coded detectors, on the GPU and the bench#642syncytium2 wants to merge 213 commits into
syncytium2 wants to merge 213 commits into
Conversation
…oints Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…see it, and a tool retunes all six MAX_FALSE_POSITIVES_PER_HOUR lived in tests/test_bench.py as a regression budget at the shipped setting, the same defect MAX_PROBE_PER_MIN had until 2026-08-22: a test cannot fail a calibration. The first retune proposed binned SCE at a setting reporting 32 calls an hour on the empty recording against a budget of 6, and found the budget only by failing it. tools/retune_operating_points.py sweeps every detector's knob on widened grids over both backgrounds and the empty recording, keeps values under both budgets, takes the best mean F1, and moves a stored value only where the gain's 95% bootstrap interval excludes zero. It writes retune.json and Figure 1 to the darkroom. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he other three were already best tools/retune_operating_points.py, 48 bench recordings per point on both backgrounds and on the empty recording. A stored value moves only where the best setting under both false-alarm budgets beats it with a 95% bootstrap gain interval that excludes zero: binned SCE 99 -> 98 mean F1 0.490 -> 0.525 (+0.022 to +0.048) LoCo 99.9 -> 99.5 mean F1 0.669 -> 0.686 (+0.005 to +0.029) rate+context 5.0 -> 4.5 mean F1 0.606 -> 0.630 (+0.012 to +0.037) CoactDetect, SPIKE-synch: already best. locust: 99.99 is +0.010, interval includes zero, so 99.999 stays. Binned SCE's F1 optimum, 75 (0.665), reports 42 calls an hour on the empty recording against a budget of 6; that trade is filed for Tony rather than taken. Grids widen to the searched ranges; the browser's defaults and its rate and SPIKE-synch grids follow. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… tests say so with a tie margin CI caught the retune reordering the fitted-field ranking: LoCo at 99.5 edges CoactDetect by 0.003 at one background rate, so 'one raw winner' flipped. At twelve recordings that is noise. Restated: the fitted field has a detector within 0.01 F1 of the top everywhere (CoactDetect, largest deficit 0.003) and the flat field has none (best 0.033). Largest rank change on the fitted field is now three - rate+context at 4.5 Hz, inside a 0.49-0.53 cluster at 40 mHz - recorded rather than hidden. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Our best parameters: binned SCE 98, LoCo 99.5, rate+context 4.5 Hz
Setup step 3 was the one decision the handoff waited on. Tony chose WSL2: Ubuntu 26.04, Python 3.14.4, torch 2.14.0+cpu, 24 physical and 48 logical CPUs, 96 GB. The status line now records that, the clone's actual path, and the venv's one gap (pyspike, which nothing under src/ imports and CI also runs without). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
main moved to #597 after the handoff was written. It widens the calibration grids of CoactDetect (alpha, 8 to 11 values), LoCo (percentile, 6 to 8) and binned SCE, moves LoCo's shipped percentile 99.9 to 99.5, and limits calibration to values under both false-alarm budgets. The Mac's Gate 1 references predate it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ed false-alarm budget, overnight unattended The workstation session stopped before Gate 1 with three questions, and a second handoff (detector optimization, on detector-review-doc) turned out to claim the same hand-written parameters under the opposite selection rule. Tony decided on 2026-09-16, and the plan now says so in the sections those decisions change: - Scripted, not in the app: the 2026-08-28 ruling rested on an assumption that the project was further along than it was. - Gate 1's float check runs at 7fc052d, before #593 and #594, and all six hand-written detectors must match the Mac exactly, with no exemptions. - The hand-written grids are the tip's, after #597, written out value by value: 99 configurations for CoactDetect, 72 for LoCo, LoCo shipped at 99.5. - Option D: every fit is scored once and read twice. Primary, ungated, as the table under test was built. Secondary, gated by one budget for every model: 1.6 times shipped CoactDetect's busy-window and quiet-field false-alarm rates, measured per outer fold on training recordings only. On the learned side the threshold joins the candidate, because a hand-written configuration already contains its threshold. - Comparison only: tuned CoactDetect and LoCo never become OPERATING_POINTS. - Unattended overnight: a systemd unit, a tested check that WSL keeps running with nothing attached, no sleep or restart, progress.json, a priority queue, and outputs under ~/runs rather than /tmp. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…d detector setting Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…misses by 0.0012 F1 At 7fc052d, every per-fold result field of all six hand-written detectors equals the Mac's, so the recordings, detectors, scorer and null twins are the same on both machines. tube's largest per-fold gap is 0.0212 F1 against a tolerance of 0.02. It is deterministic here (a rerun matched field for field), and the trained weights differ, not only the threshold. The gap is a third of the Mac's own seed-to-seed spread. The handoff stops on any mismatch, so steps 2 and 3 have not run, and the record asks for the ruling. Fits here take about 1.4 times the Mac's, which moves the overnight estimate to roughly 3.5 hours of training. tools/compare_bakeoff_runs.py is the check, reusable for steps 2 and 3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tools/search_all_settings.py asks whether the settings the one-knob retune never touched are worth anything. Coordinate rounds from the shipped point, full two-setting grids for the pairs that interact (binned SCE threshold x bin width, LoCo threshold x context window, locust percentile x synchronous frames), optionally every combination for one detector, and then held-out confirmation: every candidate is re-scored on recordings 49-96, which nothing was chosen on, with a bootstrap interval on its gain over the shipped point. Both false-alarm budgets gate every choice. It changes no operating point. The search logic is tested on made-up curves with known answers, which caught grid widening that reset each round and doubled a monotone setting 13 times. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… needs tomorrow Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s one stop Tony's ruling on step 1: the tube miss (0.0212 F1 against 0.02, on one fold) is a training difference between machines, not a defect. The hard stop belongs to the six hand-written detectors, which matched exactly; the step's first wording ran the two together. For learned models a miss is recorded and every comparison with the Mac's numbers says so. Step 3 stops only if a fold lies further from both Mac seeds than the Mac's own largest seed-to-seed gap for that model, or on the failed-training signature (F1 near 0.125 at threshold 0.0001). meta.json must carry Gate 1's three reproduction facts, and everything the run writes carries the torch version. The status line records all of it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…d rather than drawn Binned, a sub-second shift of a real recording kept 64% of LoCo's calls and 70% of CoactDetect's (2026-09-07 probe). Two causes: counts in bins laid from the start of the recording, and nulls drawn from one random stream consumed bin by bin, so one changed candidate re-randomized every later bin. window_mode='sliding' (detectors/sliding.py) counts distinct ROIs in a trailing window as an exact step function, and computes each ROI's probability of landing in the window under a uniform circular shift in closed form, so the null count is an exact Poisson-binomial: mean and sd for CoactDetect's z-test, quantile for LoCo's bar. No random numbers. On the same three real TTX baselines both keep 100% of calls at every shift from 0.1 to 0.9 s, matched within 0.05 s. Cost: 2-4x the binned version, 0.06-2.2 s per recording. Shipped settings switch to sliding. Binned stays as the MATLAB port and its parity fixtures pass unchanged. The threshold settings were tuned binned; the regime-shift precision budget now fails for both at those values, which is why it moves into bench.py where a search can respect it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ion-swing budget thr_step_sec leaves the search space (no anchors when the window slides), and a candidate must now keep precision within MAX_PRECISION_DROP between the two backgrounds, the third budget bench.py holds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tony: "write a hand off for 3 and 4. prepare to end session". Filed in docs/handoffs/ and NOT at the root. A root HANDOFF-*.md means work is in flight, and nothing is: every commit is on main, no branch waits and nothing is half-done. A root file here would be the false positive that docs/handoffs/README.md was created to stop. The open items stay in the todo; this is the briefing a session needs before working them. It recommends step 4 (the residuals) BEFORE step 3 (the blind round), because several residuals change numbers the report quotes, and a blind round spent on text about to change is eleven agents wasted. That reorders the todo, so the recommendation is stated with its reason and the call is left open. Every residual carries file and line on main (967dfd4), checked before writing: the aggregate-leak bank at initialisation values, oracle_threshold missing pick_threshold's edge-of-grid guard, the shared-offset control never shown able to fail on lab fast, line's vote cap bounding height and not integral, numbers quoted that no file stores, the unrun seed axis, and the untested numpy rigid shift. Plus the blind-round procedure and nine traps this session walked into — the most expensive being a pipe that swallowed a failing check's exit code and let a broken commit through. The todo and the goal page now point at it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ain_norm stops step 3 Step 2, at the tip: tube is identical to step 1, and CoactDetect, LoCo and SPIKE-synch still match the Mac exactly through #597's wider grids. Locust, rate+context and binned SCE differ, each for a known change. Step 3: chorus_norm is inside the stop. chorus_gain_norm fires it on folds 0 and 1, further from both Mac seeds than the Mac's seeds are from each other (0.0316 F1). It picked the same thresholds as the Mac's seed 0 on every fold, the committed code on its path is unchanged, and it is not the failed-training signature; whether it is deterministic here, or one unlucky draw, is not known. line_length did not run. This machine's fits are 1.25 to 1.4 times the Mac's. The provisional training estimate is 65 CPU hours, about 3 hours at 22 jobs, and it is a floor. Every output carries the torch version in environment.json. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… and what landing it needs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…-darkroom Release the darkroom claim on field-size-candidates
…eps-3-4 Handoff for the rigid-shift report's last two steps
The prototype spent its time computing catch probabilities one ROI at a time. An index keyed by (ROI, time) finds every ROI's slice of a context in one search, and the null distribution comes from one characteristic-function FFT instead of a loop. Sliding now runs in 0.25-0.75x the binned time on bench and real recordings, and still keeps 100% of calls under every shift. The guard works in sliding mode as in binned: CoactDetect excises a band and compacts or keeps the circle (guard_norm), LoCo pulls its halves away. Both are checked against the burst the guard exists to keep out of its own null; the fast index and the FFT are checked against the obvious forms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first sliding run's winners were a 240 s context for LoCo and CoactDetect and a 12.8 s minimum distance for locust: the settings most likely to be fitting the bench's 120 s spacing of planted events. The crowded tail recordings, fitted to the most crowded real ones, are now a column beside every held-out candidate - a check, never a selection input. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… that changes for a caller Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ded check, and the other session's run Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…id about each winner Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hat order (#636) * Three goals had no page, and two searches of main concluded they had no work Asked where four goals stood, two independent read-only searches of `main` reported that the detector review document did not exist and that detector optimization had nothing in flight. Both were reading the tree correctly. The review document is two built pages with three blind murderboard rounds behind them; the optimization goal had a search over every declared setting running at the time. All of it, and every summary of it, sat on unmerged branches. That is the failure `docs/goals/` was created on 2026-09-14 to end, reproduced twice in one afternoon by fresh readers. A goal whose only account of itself lives on the branch it describes is invisible to every session that starts from `main` — which is every session — and disappears entirely when that branch lands. Three pages, following the shape `docs/goals/README.md` sets out and the precedent `unsupervised-learning.md` set: - `coded-detector-optimization.md` (`opt/`) — what the 2026-09-16 retune moved and on what rule, that only one knob per detector has ever been swept, the two false-alarm budgets, the long-context result that wins held-out and loses on crowded recordings, and the thresholds waiting on a ruling. - `learned-model-family.md` (`nets/`) — the folder-is-the-registry family and which members are on which branch, what each scores against CoactDetect, the three architectures that fail to train and the one that turned out to be a fixable defect rather than a verdict on its shape, and the gate the tuning run is halted at. Explicitly not the label-free goal. - `detector-review-document.md` (`review/`) — both built pages, the review that changed findings rather than wording, the sources it draws on and how current each is, and the darkroom claim that reads ACTIVE on `main` because its release is on the branch too. A page may cite a file on an unmerged branch; it may not do so silently. Every such source carries a marker, and taking a marker off is part of landing the branch it names. Also: four index rows so the pages are reachable by the words a session would grep for, and the root handoff now points at the folder rather than at the one goal that had a page when it was written. Link check green (299 index pointers), sapper clear, check_quotes clear. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hQX6iESxBsQk7Jan875e3 * Resolve the merge properly: conflict markers were committed and pushed The earlier merge staged an unresolved file with `git add -A` and the markers reached origin. Only the branch was affected -- nothing merged, nothing on main. All three files now take main's version: this branch's copies landed in #601 and main has evolved them under the 2026-09-17 program, so there is nothing here to preserve. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hQX6iESxBsQk7Jan875e3 * Mirror interface2 to GitHub before anything is curated, and say why that order Tony asked whether to transfer the whole GitLab repository now and pick it apart for the sub-goals later. The handoff implements yes, and argues the order: a mirror is additive and reversible, curation is subtractive and lossy, and doing the second without the first is a one-way door. It also retires the risk that prompted the goal — losing GitLab access — today rather than at the end of a curation project. Written for a session on WSMIP065, executable as commands, with the checks that separate a copy from a verified copy: - Branch and tag counts compared both sides, not the push's exit code. - The 35 stranded tools — committed on a branch, never merged, and the part a non-mirror transfer loses silently — derived from armory's manifest and confirmed present. 35 of the estate's 44 are in interface2. - `push --mirror` deletes refs the source lacks, so the target is created empty and the handoff says why that is not a formality. - What a mirror does not carry at all: issues, merge requests, the wiki, release notes, CI semantics, LFS objects. The gate is private on creation and public as a separate ruling, on three known hazards: commit prose that reads as a second laboratory (#414), personal paths in the backslash shape SAP004 cannot see, and a paper under review whose venue may forbid prior disclosure. Publishing here is not undoable, so the order matters more than the speed. The remote URL is read at run time rather than written down: it carries a personal namespace and this repository is public. One decision is left to Tony and named as his — which remote is authoritative once both exist, since two writable copies drift, and drift is the failure armory was built to name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017hQX6iESxBsQk7Jan875e3 --------- Co-authored-by: Claude <noreply@anthropic.com>
backdate() raced git's own cleanup, and parallel CI made it land
Cache pip's built wheels, and name the slowest tests in the log
…ci.yml The suite has run `-n auto --dist loadfile` since 71950dd, and the three consequences a session has to know were documented only in comments inside .github/workflows/ci.yml — which is not a file anyone reads before writing a test. Both rules were learned from red runs: a wall-clock budget needs the serial marker, and so does anything touching the built site/. Also records what the change taught about reading CI: every skip now prints its reason, and a race the suite always had (backdate() vs git's own cleanup) began landing once the timing changed, so a new red test in a parallel run deserves reading as a timing window first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The parallel CI's rules live where a session reads them, not only in ci.yml
* Claim the interface2 mirror on the board before the first push A new private GitHub repository is a shared output, so it is claimed before it exists. The block records that the mirror is refreshed after the Mac session still pushing to GitLab ends, and that it stays private. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Record the interface2 mirror as verified: 197 refs identical, 35 of 35 stranded tools GitHub was compared against the mirror clone ref for ref, name and SHA, and armory's stranded tools were checked by content rather than path, because six of the branches armory cites were deleted from GitLab after being merged or contained elsewhere. The claim stays active until a refresh push after the last Mac session lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t a running one My own concurrency block, 71950dd. It keyed main on the ref and said every commit landing on main keeps a finished result. It did not. `cancel-in-progress: false` protects a run that is RUNNING; a run still WAITING for its group is cancelled when a newer request arrives. So 66fa654's run sat pending behind 9185431's, be2c25d's merge arrived seven minutes later, and GitHub cancelled 66fa654's to make room — a merge nothing ever tested. With three merges in an afternoon that was the normal case, not bad luck. Keyed on github.sha for a push, two main runs are never in the same group, so neither can displace the other. Pull requests keep the ref, which is the whole point of the block: a new push there still cancels the run it invalidated. The claim in the comment is now the thing the key actually guarantees, and the incident is recorded beside it so the next reader does not re-derive it from a cancelled run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A merge could land with no CI run at all, because a waiting run is not a running one
…'s final sync (#641) Tony decided GitHub is the only remote. Four more repositories were mirrored into private GitHub repos and verified ref for ref, and their checkouts re-pointed. Two GitLab projects, R and ggplot-tuner, turned out to be gone from GitLab already, so the local checkouts were the only copies: ggplot-tuner was mirrored from its checkout, and R's 21 commits that fireflies lacked went into fireflies as archive/R-* tags. The R checkouts were kept, not deleted, because two of them hold data that git ignores. interface2 waits for the Mac session to finish before its final sync. Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e machine's disk Before GitLab is locked down and the work lives only on GitHub, nothing the handoff cites should exist only on WSMIP064. Two things did. - The shakedown's summary (declaration, results, selections, configs; under 1 MB) is now in docs/learned/tuned_vs_coact/shakedown_home_spec/, so the numbers the handoff quotes trace to files. Its README says what it is: a rehearsal on the retired home spec, not a result. The one personal path, the output folder in the recorded command line, is written as %USERPROFILE%. The 0.4 GB of per-fit files stay local and the README says so. - The launcher was a file in %USERPROFILE%\runs. It is now tools/launch_tuning_run_windows.cmd, parameterised by run name and tool arguments, with the schtasks lines that register it in its header. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…d to end, and .cmd files check out CRLF SHIFT moves %0 as well as the arguments, so %~dp0 read after the argument loop named the first argument, not the script, and the launcher could not find the worktree's .venv. It now captures its folder before any shift. Run for real: a bench --quick run under a name, 20 of 20 jobs, then the same name again, resuming with 0 jobs. .gitattributes pins *.cmd to CRLF, as it already pins *.sh to LF: cmd.exe finds a goto label by scanning the file, and LF-only endings can make it miss one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ft becomes a todo (#643) * interface2 is off GitLab: the handoff leaves the root, and what is left becomes a todo The mirror handoff asked for a copy that stopped there. Tony took it further the same day - remove every dependency on GitLab, then, with no machine working in it, kill it - so interface2 was cut over after a final ref-for-ref parity check and GitHub is now its only remote. The handoff moves to docs/handoffs/ with a header saying which of its open questions were settled; the board claim is released; and the steps no one on this machine can take - the Mac's re-point, archiving the GitLab projects, the branch triage on GitHub, and text that still names GitLab as live - are a todo rather than a handoff body. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The Mac is re-pointed too, and GitLab has received nothing since the cutover Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Going through the GitLab UI turned up four CASCADE projects. Each held a one-line README and nothing else, and Tony deleted them. The actual work - three unpushed commits and 168 untracked files, including CASCADE models trained on this lab's data and its ground-truth recordings - sat in a local checkout of the Helmchen lab's repository with no remote holding any of it. It is now syncytium2/cascade-td, private, verified by SHA per branch and by content per file. interface2's GitLab project is archived. Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e kept, then deleted (#645) Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
… is triaged (#646) Co-authored-by: richard defazio <defazio@umich.edu> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…to reach the tuning tool Three conflicts, each resolved to keep both sides. train.py: main's logit mode and custom grid for pick_threshold now run through this branch's device-aware probabilities(), with THRESHOLD_GRID as the default grid. line.py: main's bound_vote beside this branch's extra_pools and top_m; line_length builds neither, so its forward pass is unchanged. fair_bakeoff.py: both lists of registered models. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…cordings Tony's four conditions for goal 2's weekend run, 2026-09-18. The fold defect, in this tool: train fits on a contiguous run of the training list, so in fold order the run sat inside the first training folds and outer folds sharing them trained the same model. The shakedown shows it: at seed 0 the held-out folds trained on (0,1,2) and (0,1,3) both fitted recordings 1000-1009. At twelve seeds per fold it would have been three outer folds of four. Seeds are now dealt round-robin across the training folds, starting after the held-out one, so every fit spans all its training folds and the threshold pair rotates with the held-out fold. Plan.fold_check replays train's and pick_threshold's own draws for every seed; the run refuses to start unless every outer fold has its own fitting set and threshold pair and every inner pair its own fitting set, and meta.json records the check. The coded side slides. OPERATING_POINTS still ships CoactDetect and LoCo binned (goal 1 held the switch for the viewer's sake, 8525be3), and run_detector fills every unsearched knob from there, so the per-fold search would have walked binned detectors goal 1 already measured as inferior. CODED_BASE carries goal 1's chosen sliding values (6fe09ab) under every coded setting scored, as the search's start and as the budget's reference; meta.json records the base, its source and each coded detector's window mode. Twelve seeds per fold: each fold's held-out F1 pools 24 recordings, not 12. progress.json is rewritten at least once a minute, job or no job, and --mirror copies it to a claimed darkroom folder, so the run stays readable after the session that launched it ends. The estimate's GPU fit times are today's, measured through this tool's own fit path: one process gives chorus_norm 309 fits an hour, two to six give 264-274. One test is marked xfail: bugarach detect cannot read back a setting the every-knob grids choose (todo filed on main). The run never reads it back. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ing coded side and twelve seeds per fold Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… the primary checkout Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s 2000-2047, and the launcher's comma bug is owed a fix Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t one slot WSMIP064 and WSMIP065 are each writing a report on goal 2's fair comparison, and both briefs asked for a figure of what the four architectures do differently. Tony: "They should definitely use draughtsman otherwise they'll likely be completely incomparable." So chorus_norm, chorus_gain_norm and line_length get specs beside a comparison sibling of the tube's, all at report.css's 912px column with a 9.5px type floor, and every one breaks its rows at the same place: the first row is the stages that still hold one row per ROI, the second starts where that axis collapses, the third is the head. The first row's length is the comparison: two boxes for tube, three for line_length, four for both chorus. TRACED THROUGH THE REGISTRY. build_chorus_norm() called bare returns plain chorus — the registered norm=True never reaches it — and draughtsman calls its target with no arguments, so the first trace was of the failed control and check passed on it. bugarach.learn.registered exposes ARCHITECTURES[name].make as a zero-argument target. tube and line_length are safe only because their builders' defaults equal their registered config. Named row breaks needed a draughtsman change (layout.breaks, draughtsman PR #1 at 18b4453); re-vendored here. The front-page specs change only their stamp, which the freshness family requires to agree, and architecture.svg and architecture-phone.svg are byte-identical. tools/make_comparison_figure.py stacks the four committed figures at their own extents, one unit to one pixel, and refuses a set whose specs disagree about the slot. It writes to the darkroom; the page is not committed, the four figures it is built from are, and the staleness test regenerates them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
WSMIP064's fair-comparison report ships in the public repo and points at the architecture drawings by path, which works for the darkroom page only on a machine with the mount. docs/learned/comparison/comparison.svg is the four committed panels composed at one scale, rewritten by --commit and byte-compared against a fresh composition by the test, so it cannot outlive its panels. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e swapped, and the gated leader was chorus_gain_norm WSMIP065 checked the line against the shakedown's own results.json. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The module-scoped run fixture shells out to the tool with a 600 s timeout and four jobs. Under CI's four xdist workers that budget measured the other workers: 3.11 passed while 3.13 and 3.14 reported 13 setup errors each, all TimeoutExpired at 600 s, in one run (35444799882). The module is now marked serial, so it runs after the parallel pass with the runner to itself. The timeout is unchanged. The jobs stay at four rather than one: the quick run took 276 s at four jobs and 559 s at one on this workstation, and 559 against 600 leaves no room on a slower runner. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… 600 s Comment only. Measured on run 35453034315, the first with the module serial: one job would have timed out on every leg, and four leave 55 s of headroom on the slowest. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The fair comparison's four architectures, drawn through draughtsman at one slot
The vendored draughtsman stamp names the commit that landed, 0898967
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft, and not set to auto-merge. The base is #596's branch, whose merge is Tony's call (it registers the learned models in the lab server and the model picker). This PR exists so the work is findable in one place; it carries
tune-learned-vs-coactin full.What it holds. The tool and plan for goal 2 of the program Tony set on 2026-09-17 (
docs/goals/README.md): a fair comparison of the coded detectors against the nets.tools/tune_learned_vs_coact.py: nested cross-validation, with nothing about a configuration chosen on the fold it is scored on. Two selections: F1 alone, and F1 under one false-alarm budget shared by every model. Three training seeds per configuration; resumable, atomic result files.docs/goals/learned-model-family.md):bench.OPERATING_POINTS.search_all_settings.choose_settings(option A, agreed session to session). Also: a context window wider than the planted spacing is refused, because it contaminates its own null.train(device=...), which leaves the CPU default byte-identical and turns on deterministic CUDA;--deviceand--gpu-jobson the tool, with GPU fits in their own worker pool.tools/launch_tuning_run_windows.cmd, the proven Task Scheduler launch path.Status: the full run is blocked by its own guard until goal 1 lands its every-knob sliding CoactDetect values in
OPERATING_POINTS. The whole account is the status line ofHANDOFF-workstation-tuning.md.19 tests in
tests/test_tune_learned_vs_coact.pyand 3 intests/test_learn_train_device.pypass on WSMIP064. The GPU tests skip where there is no CUDA device.🤖 Generated with Claude Code