This repository was archived by the owner on Jul 17, 2026. It is now read-only.
refactor(logging): canonical wandb keys at the source, namespace-validated sink#985
Open
claude-spd1 wants to merge 1 commit into
Open
refactor(logging): canonical wandb keys at the source, namespace-validated sink#985claude-spd1 wants to merge 1 commit into
claude-spd1 wants to merge 1 commit into
Conversation
…dated sink The sink stacked three naming regimes: a short-key remap table (_METRIC_KEYS), startswith-based train/ prefixing, and verbatim pass-through — held together by a torch-overlay-parity constraint that retired with the torch trainer. Now every producer emits full canonical keys and MetricsSink.log asserts each key lives under the declared namespace tree (_WANDB_KEY_NAMESPACES), so a new metric joins the tree deliberately instead of drifting to the wandb top level. Emitted key strings are byte-identical to before — panels and overlays on existing runs are unaffected. (Only exception: a cfg.name override on the faith/imp terms now flows to the key like it always did for recon terms, instead of being ignored.) Also deletes dead lab code: infra/wandb.py's init_wandb (zero callers; still defined the retired 3-pool async slow_eval/step axis) and try_wandb (zero callers). Crew-Address: task/tidy-wandb-logging-structure Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Part 1 of the wandb-logging tidy (board task
tidy-wandb-logging-structure): the sink stops remapping keys and starts validating them.train/loss/<term.name>,train/schedules/*,train/grad_norms/*; the engine loop emitstrain/perf/*/train/mem/*directly. The_METRIC_KEYSremap table, thestartswith-basedtrain/prefixing, and the verbatim pass-through regime are all gone.MetricsSink.logasserts every key lives under the declared canonical namespace tree (_WANDB_KEY_NAMESPACES, documented inrun.py) — a new metric joins the tree deliberately instead of silently landing at the wandb top level.infra/wandb.py::init_wandb(zero callers; still defined the retired 3-pool asyncslow_eval/stepaxis) andtry_wandb(zero callers).Emitted key strings are byte-identical to before — panels, saved views, and overlays on existing runs are unaffected. One deliberate edge: a
name:override on the faith/imp loss configs now flows into thetrain/loss/<name>key (previously ignored, unlike recon terms — the override promises an instance identity and the key now keeps that promise).The rename set (unifying the slow tier under
eval/slow/*, readable grad-norm keys, threshold-free L0 keys, namespaced pretrain keys) is deliberately NOT in this PR — it breaks panel overlay against pre-tidy runs and is awaiting Oli's call on the board thread. The campaign-view CLI comes separately.Related Issue
Board task
tidy-wandb-logging-structure.Motivation and Context
Three naming regimes stacked in the sink, held together by a torch-overlay-parity constraint that retired with the torch trainer (tag
torch-oracle). A key matching none of the regimes landed at the wandb top level with no warning.How Has This Been Tested?
make checkclean (ruff + basedpyright over the workspace).pytest param_decomp/tests/test_recon_log_keys.py— rewritten to pin the canonical scheme: term-name → key derivation, plus a new sink test asserting off-tree keys fail fast.pytest param_decomp/tests/stacked_parity/(fixture-key → live-key mapping extended),test_generic_model_io.py,test_llama8b.py -m "not slow",test_llama_simple_mlp.py -m "not slow"— all green.pytest param_decomp_lab/experiments/{tms,resid_mlp}/— the toy end-to-end runs exercise the real engine loop + sink with the new validation: 42 passed.Does this PR introduce a breaking change?
No wandb-facing change (key strings identical). In-code:
run._METRIC_KEYSis gone and step-metrics dict keys are now the full wandb keys —test_stacked_parity.py's fixture mapping was updated accordingly;gen_stacked_fixtures.pyis untouched (it runs on the base branch by design).Crew-Address: task/tidy-wandb-logging-structure
🤖 Generated with Claude Code