This repository was archived by the owner on Jul 17, 2026. It is now read-only.
config: canonical-seat policy, CI parse gate, sweep-debris cleanup#983
Open
claude-spd1 wants to merge 1 commit into
Open
config: canonical-seat policy, CI parse gate, sweep-debris cleanup#983claude-spd1 wants to merge 1 commit into
claude-spd1 wants to merge 1 commit into
Conversation
claude-spd1
pushed a commit
that referenced
this pull request
Jul 13, 2026
….md / #983) Sweep one-offs don't get committed — provenance for the launched runs lives in the run-dir pinned launch_config.yaml, the refs/runs/snapshot/* refs, and wandb. No canonical muon seat yet: muon stays config-gated experimental until it's promoted to a default recipe, at which point a seat row lands in CONFIGS.md. The experiment configs remain on the bridge/task-* experiment branches for history. Crew-Address: slack/C08T7UV4449/1783730484.936959 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
claude-spd1
force-pushed
the
bridge/task-consumer-config-schema-drift-configs
branch
from
July 13, 2026 21:01
4359384 to
e941c0f
Compare
…ep debris The repo carried 30 LM run yamls of which ~26 were launched one-offs (PROFILE/SAVESMOKE/scaling-matrix permutations) — every one a permanent migration obligation for every schema PR, while the actual archetype configs (jose, ss-2L, pile-4L) rotted unparseable. Provenance for launched runs never depended on these files: the run dir pins launch_config.yaml, submit snapshots the tree to refs/runs/snapshot/<id>, and wandb stores the config. - CONFIGS.md: the policy — a registry of <10 canonical LM seats; sweeps are not committed; schema PRs migrate every repo yaml in the same PR (executed in-repo migration, the #966 pattern, never a comment-attached script — the #939 lesson); stored-run pins are never rewritten in place. - param_decomp_lab/tests/test_repo_configs_parse.py: the CI gate — every LM yaml in the tree must schema-parse and pass the canonical assert; the three broken archetype seats are quarantined in a shrink-only KNOWN_BROKEN list. - Delete 26 one-off yamls from param_decomp/configs/ and the 5 dead non-seat yamls from param_decomp_lab/experiments/lm/. Kept seats: llama8b_l18_C49k_200k, llama8b_full32L_seq512_b128_dp128, llama8b_full32L_HSDP_b32_dp32_SAVESMOKE, pile_ppgd_bsc (+ the three broken-awaiting-migration lab seats). Crew-Address: task/consumer-config-schema-drift
claude-spd1
force-pushed
the
bridge/task-consumer-config-schema-drift-configs
branch
from
July 13, 2026 21:15
e941c0f to
95963a8
Compare
claude-spd1
pushed a commit
that referenced
this pull request
Jul 14, 2026
The launch config is a one-off run recipe, not a canonical seat — it rides the task thread (task:ingest-qwen-3-8b), and a launched run pins its own copy in the run dir. Crew-Address: task/ingest-qwen-3-8b Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ocg-goodfire
pushed a commit
that referenced
this pull request
Jul 15, 2026
) * feat(targets): ingest Qwen/Qwen3-8B-Base as a decomposition target Qwen3-8B-Base is structurally Llama-3.1-8B plus one delta — per-head RMSNorm on q/k after projection, before RoPE (HF Qwen3Attention q_norm/k_norm) — so it rides the existing llama8b target rather than a fork: - `LlamaConfig.qk_norm` (vendored_jax) + optional `FrozenAttn.q_norm/k_norm` (+ static `eps`), applied in `core` after the head reshape, exactly as HF does. Llama paths are untouched (fields default None; SPEC unchanged — q/k sites still decompose before RoPE/SDPA, the masked site output now feeds the norm first). - `qwen3_8b_config()` (36L, d 4096, 32h/8kv, inter 12288, vocab 151936, theta 1e6, eps 1e-6, plain RoPE, untied embed) + `hf_model_config()` model-name allowlist; HF loader reads the q/k_norm keys. - lab dispatch: `_resolve_target` accepts transformers.Qwen3ForCausalLM, d_resid/build_target/run_metadata resolve arch by model name; "Llama"/ "Qwen3" path schemas registered (raw-HF `layers.{i}.self_attn.*` grammar). - attn-patterns eval recipe takes the q-site name and applies the layer's QK-norm (per-layer weights — one shared closure was Llama-only). - tests: qwen-tiny parametrizations of the target suite + a QK-norm load-bearing test; NEW direct JAX-vs-HF parity suite (`tests/qwen3_hf_parity/`): tiny-random Qwen3ForCausalLM golden at fp32 tolerance (passes at rtol 2e-4) and a slow real-weights check (bf16; final-logits KL < 5e-3 + tie-aware argmax; passes against the cluster snapshot). Goldens regenerate via the torch-env `gen_hf_fixtures.py` (torch 2.13.0+cpu, transformers 5.13.1). - launchable config `qwen3_8b_full36L_HSDP_b32_dp32.yaml` mirroring the llama flagship (252 sites, 63 sites/chunk -> 4 chunks). Needs a Qwen3-tokenized prestaged dataset (fineweb_qwen3_tok_512) before launch. Crew-Address: task/ingest-qwen-3-8b Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(targets): split model families out of a shared glu_transformer target (review) Per PR #987 review: no switch-behavior inside llama8b, no cartesian-product configs, no inline model-family dispatch in evals. - targets/glu_transformer.py (+ _sharding): the SHARED HF GLU-transformer machinery — site grammar, FrozenAttn/GLULayer/GLUDecomposedModel (renamed from Llama*), scan/masked-forward engine, HF loading parameterized by the family's attn loader + inv_freq. FrozenAttn gains a `_prep_qk` pre-RoPE hook (identity) and a target-owned `pattern` recipe; GLUDecomposedModel gains `attn_pattern(q_site, q_flat, k_flat)`. - targets/llama8b.py: the Llama family file — llama31_8b_config (vendored LlamaConfig, llama3 rope) + load_decomposed_llama_from_hf. - targets/qwen3_8b.py: the Qwen3 family file — Qwen3FrozenAttn with REQUIRED q_norm/k_norm/eps fields overriding _prep_qk (QK-norm keyed by attn class, never a flag; invalid states unrepresentable), qwen3_8b_config (GLUConfig), load_decomposed_qwen3_from_hf. vendored_jax/llama.py reverted untouched. - attn_patterns_eval inverted: the recipe is target-owned behind a runtime_checkable AttnPatternModel protocol; the eval no longer imports any concrete target and the match-on-class dispatch is gone. Step factories drop the pattern_fn param. SimpleMLPDecomposedModel implements the method. - lab: HF_MODEL_FAMILIES registry (model_name -> arch_config/load/model_type) in experiments/lm/config.py — the ONLY place a model name selects a family; build_target/run_metadata consume it. - qwen tiny-target tests move to test_qwen3_8b.py (incl. per-layer QK-norm pattern test); parity suite re-pointed at the family loader. Full local run of affected suites green; make check clean. Crew-Address: task/ingest-qwen-3-8b Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: point README/CLAUDE.md target tables at the family split Crew-Address: task/ingest-qwen-3-8b Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * config: drop the qwen3 run yaml per the canonical-seat policy (PR #983) The launch config is a one-off run recipe, not a canonical seat — it rides the task thread (task:ingest-qwen-3-8b), and a launched run pins its own copy in the run dir. Crew-Address: task/ingest-qwen-3-8b Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(targets): frozen-weights dtype is the family's call, not a shared-module global Per review: drop `DT = jnp.bfloat16` from glu_transformer — HFWeights takes the dtype, load_decomposed_glu_from_hf threads it, and each family loader passes bf16 (matching its TargetConfig.supported_weights_dtypes). Crew-Address: task/ingest-qwen-3-8b Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review: registry owns model_class; drop dead target_aliases; family-neutral docstring - HFModelFamily gains model_class; _resolve_target derives the accepted set from the registry and cross-checks class <-> model_name (was a parallel hardcoded tuple — a second place enumerating families). - targets/target_aliases.py deleted: zero importers, and its docstring claimed a dispatch that no longer exists. - GLUDecomposedModel docstring stops enumerating concrete families. Crew-Address: task/ingest-qwen-3-8b Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: drop needless quoted annotations (quotes only for true self-references) Crew-Address: task/ingest-qwen-3-8b Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: PD User (shared) <pd-user@goodfire.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Per Oli on the bridge task (consumer-config-schema-drift, evolved to "generically sort out configs"): clean up the existing config sprawl and set the policy that stops it recurring.
What this does
1.
CONFIGS.md— the policy. The repo maintains a registry of <10 canonical LM config seats and commits nothing else. Sweep/profile/one-off yamls are never committed: a launched run's provenance already lives in the run-dir pin (launch_config.yaml), therefs/runs/snapshot/<id>ref, and wandb — the repo copy records nothing and only rots. Schema PRs must migrate every repo yaml in the same PR with an executed in-repo migration (the #966 pattern), never a comment-attached script (the #939 lesson: 97/104 stored runs became unopenable). Stored-run pins are never rewritten in place.2. CI parse gate (
param_decomp_lab/tests/test_repo_configs_parse.py): every LM yaml in the tree must schema-parse (LMExperimentConfig.model_validate) and passassert_canonical_algorithm_config. Schema drift in repo configs is now a red PR at the PR that causes it. The three broken archetype seats are quarantined in a shrink-onlyKNOWN_BROKENlist.3. Cleanup: 29 yamls deleted, 9 seats kept. Grounded in a parse sweep at tip (all 28 llama8b infra permutations parse fine; all 8 old lab archetype yamls are dead — we were maintaining the debris and letting the wanted ones rot):
configs/llama8b_l18_C49k_200k.yamlconfigs/llama8b_full32L_seq512_b128_dp128.yamlconfigs/pile_ppgd_bsc.yamlconfigs/llama8b_full32L_HSDP_b32_dp32_SAVESMOKE.yamlconfigs/llama8b_l18_b128_cmp32.yaml(loaded bytest_config.py,test_finetune_resume.py,test_llama_simple_mlp.py)configs/llama8b_l18-26_9layer_chunkwise.yaml(converted bytest_config.py)lm/— broken at tip, awaiting migration (KNOWN_BROKEN)Deleted: 10
*_PROFILE+ 1SAVESMOKE+ the seq512/HSDP scaling-matrix permutations + one-off l18 variants +pile_pgd1(24 fromparam_decomp/configs/), and the 5 dead non-seat lab yamls (gpt2-xl,gpt2_config,llama-3.1-8b,pile_llama_simple_mlp-12L,-4L_layerwise). All are in git history and in the run dirs that ran them.Curation calls to confirm (Oli)
l18_C49k_200k(the 200k-step recipe with launch provenance notes) overl18_b128_cmp32(5k steps).seq512_b128_dp128vs one of the HSDP variants.pile_pgd1deleted as superseded bypile_ppgd_bsc— veto if the non-persistent PGD config is still a live recipe.jose.yamlstays a seat (arch-research testbed, per Slack) orpile_ppgd_bsccovers the 4L-pile need and jose goes too.Sequencing with open PRs
configs/muon/— under this policy they wouldn't land; the sweeps already ran (findings in lore:2026-07-11--muon-cifn-matrix-pile4l,2026-07-13--smooth-l0-muon-matrix-100k) and the run dirs pin their configs. Suggest trimming to ≤1 canonical muon config if muon becomes the default recipe, else none.Verification: parse gate passes (5 passed, 3 skipped = the quarantined seats);
make formatclean;make typefails identically on clean tip in this env (pre-existing, missing optional deps).Follow-up (will file on the bridge board): migrate the three KNOWN_BROKEN archetype seats to the tip schema and empty the list.
Crew-Address: task/consumer-config-schema-drift
🤖 Generated with Claude Code
Edit history: first push accidentally omitted the deletions (a stash round-trip unstaged them) — the new parse gate itself flagged it. Second push deleted two yamls that are load-bearing test fixtures (
llama8b_l18_b128_cmp32,llama8b_l18-26_9layer_chunkwise) — restored as registry seats, and the policy now says a config a test loads is a seat by definition.