This repository was archived by the owner on Jul 17, 2026. It is now read-only.
feat(recon): depth-phased routing wave (stacked on #977)#979
Open
claude-spd1 wants to merge 3 commits into
Open
feat(recon): depth-phased routing wave (stacked on #977)#979claude-spd1 wants to merge 3 commits into
claude-spd1 wants to merge 3 commits into
Conversation
depth_phased_probability routing: per-site Bernoulli(p_layer(step)) where each layer runs the p template compressed onto [t_layer, total_steps], start times staggered linearly over wave_span_frac of training. Backward (default) starts the DEEPEST layer at step 0 — Oli's intuition that later layers, with shorter causal paths to the logits, are easier to learn first — sweeping the wave backwards through the network; forward is the mirrored control. Layer indices parse from the h.<i>. site-name prefix (asserts, so toy targets refuse loudly). wave_span_frac=0 degenerates to scheduled_probability. R1 independence unchanged. Crew-Address: task/run-some-random-architecture-ideas Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…2'/S24) PersistentPGDReconLossConfig gains a routing field (default all — byte- identical for every existing config); build_loss_terms uses it in place of the hardcoded AllRoutingConfig literal. The step already honored routes for persistent entries, so this only unpins the plan. Warmup ascents stay route-all (S24 torch parity). An unrouted position's persistent source gets zero gradient that step (frozen path, S14 grad reuse) and resumes when re-routed. S12' already admits subset-routed adversarial terms. Requested by Oli (slack, 2026-07-13) to include PPGD routing in the subset-routing investigation. Crew-Address: task/run-some-random-architecture-ideas Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…aw (S24 amended) Per Oli's decision (slack 2026-07-13): commit entirely to the routing being used — a persistent term draws its routing ONCE per step and the SAME draw covers all warmup ascents and the final loss forward, so the adversary trains and is scored under identical routing (matching fresh-PGD's single-draw semantics). Supersedes the S24 warmup route-all torch-parity pin. Byte-identical for route-all configs: the final-forward routing key derivation is unchanged, and route-all warmup routes are None exactly as before (stacked-parity suite green). Crew-Address: task/run-some-random-architecture-ideas Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
depth_phased_probabilityrouting for the subset recon terms: per-siteBernoulli(p_layer(step))where each layer runs theptemplate (aScheduleConfig) compressed onto[t_layer, total_steps], with layer start times staggered linearly overwave_span_fracof training.direction="backward"(default) starts the DEEPEST layer at step 0 — a routing wave sweeping back through the network;"forward"is the mirrored control. Layer indices parse from theh.<i>.site-name prefix (asserted, so non-layer targets refuse loudly).wave_span_frac=0degenerates toscheduled_probability. SPEC S11 amended additively.Stacked on #977 (needs its RoutingSampler step-arg plumbing); base is
feature/routing-schedules-fixed-k— retarget tofeature/jaxafter #977 merges.Motivation and Context
Oli's ask on the board task
run-some-random-architecture-ideas: later layers have shorter causal paths to the output so should be easier to learn initially — ramp their routing probability up earlier, a wave moving backwards. Runs alongside the champion-vs-baseline seed sweep.How Has This Been Tested?
3 new tests in
test_routing.py: backward wave orders layer means correctly mid-training and converges to the endpoint at end; forward mirrors backward layer-wise; non-h.<i>.site names refused. All 11 routing tests green;basedpyrightclean; integration smoke through real jitted train steps (both directions, finite losses).Does this PR introduce a breaking change?
No — purely additive config variant; existing samplers untouched.
Crew-Address: task/run-some-random-architecture-ideas
🤖 Generated with Claude Code