This repository was archived by the owner on Jul 17, 2026. It is now read-only.
feat(recon): per-sample assignment for MergedStochasticPGDReconLoss (+ 4L 400k arm configs)#980
Open
claude-spd1 wants to merge 8 commits into
Open
Conversation
…nLoss assignment: per_sample draws the adversarial-vs-stochastic Bernoulli once per batch element instead of per (batch, position) — the (B, 1) draw broadcasts over the position axes, so a sequence's masked forward sees a single mask family and per-position losses are never scored against a mixed-family attention context. Same S10' feasible set, a coarser joint sampler; per_position (the default) byte-preserves the existing behaviour. Requested by Braun on task recon-pass-consolidation-results. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Crew-Address: task/recon-pass-consolidation-results
…readable Path.exists() raises PermissionError (not False) when a parent dir denies stat; the t-9d2b8f02 cache lives under a group-restricted dir, so pytest collection crashed for users outside that group instead of skipping. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Crew-Address: task/recon-pass-consolidation-results
…), cw-east Identical to the merged arm except assignment: per_sample; seed-1 variant guards the single-seed read. data_files points at the crew user's staging of the same HF dataset (the canonical /mnt/data copy is group-locked). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Crew-Address: task/recon-pass-consolidation-results
…quet glob Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Crew-Address: task/recon-pass-consolidation-results
…e CI + Lp imp-min) Lucius's catch: every round-1 arm (control included) used a ~8x-smaller CI fn (d1024x4 vs d2048x8) and SmoothL0 instead of Lp imp-min, landing well below the paper baseline (klCI 0.39-0.45 vs 0.337, PGD ~0.9-1.3 vs 0.64). Round 2 changes only the recon-term structure between arms: control pair / merged per-position / merged per-sample, all byte-matching pile_ppgd_bsc.yaml otherwise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Crew-Address: task/recon-pass-consolidation-results
…ty changes vs round 1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Crew-Address: task/recon-pass-consolidation-results
…ended, additive)
Two new routing configs for the subset recon terms:
- fixed_k_subset: per position, a uniform subset of exactly k live sites
(vs uniform_k's k ~ U{1..n}).
- scheduled_probability: static_probability with p evaluated per step from
a ScheduleConfig (increasing ramps use final_val_frac > 1; both endpoints
validated as probabilities).
SAMPLE_ROUTING gains the traced step: RoutingSampler is now
(key, leading_shape, step_f32) -> draws; only scheduled samplers read the
step (all others ignore it), and build_loss_terms/make_plan thread
total_steps down so the scheduled sampler can evaluate its schedule with
scheduled_value_traced. SPEC S11 + the SAMPLE_ROUTING binding-seam row
amended additively; R1 independence unchanged (draws stay independent
across sites, positions, forwards, steps — the schedule only moves p).
Motivation (board task run-some-random-architecture-ideas): test whether
the subset recon term wants more base-model supervision early — low
routing p ramped up over training — vs the static uniform-k baseline.
Crew-Address: task/run-some-random-architecture-ideas
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-chunk rescue arms
Braun's overnight directive: reproduce the 8-chunk plan's quality with a single
all-sites pass. No chunk-count quality A/B exists at any LM scale (survey), so this
stages one on the ab2 big-CI + SmoothL0 harness: ChunkwiseSubsetReconLoss refs
(sites_per_chunk 3 and 6) vs 1-chunk arms carrying the deep-linear-identity rescue
levers — fixed_k routing k in {3,6,12} (absolute routed-site budget) and the
routep-sched-1 champion ramp (scheduled_probability 0.1->1.0). Includes the
cherry-picked fixed-k/scheduled routing commit (5a18ff7, S11 amended) and the
merged-term case updated for make_plan's total_steps.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Crew-Address: task/recon-pass-consolidation-results
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the PPGD-merge work, requested by @danbraunai on bridge task
recon-pass-consolidation-results: draw the adversarial-vs-stochastic assignment once per batch element (assignment: per_sample) instead of per (batch, position), so a sequence's masked forward sees a single mask family — no mixed-family attention context within a sample.MergedStochasticPGDReconLossConfig.assignment: per_position | per_sample(defaultper_position, byte-preserves existing behaviour). The(B, 1)Bernoulli draw broadcasts over the position axes; mask/delta/routing composition is unchanged.test_merged_train_step_end_to_endparametrized over both assignment modes.PermissionError, not skip) for users outside the goodfire group because thet-9d2b8f02skipif probes a group-locked dir.data_filespoints at the crew user's staging of the same HF dataset (danbraunai/pile-uncopyrighted-tok-shuffled) because the canonical/mnt/data/.../pile_neox_tok_512copy is group-locked.Runs launched on cw-east (W&B group
ppgd-merge-ab-400k):p-188aefae(seed 0, job 174349),p-22f655d8(seed 1, job 174350), both--cuda_extra cuda13(Blackwell TMEM-fixed cuBLAS).🤖 Generated with Claude Code
Crew-Address: task/recon-pass-consolidation-results