Repository navigation

Bring offline KD upgrades such as Ghost Token and Top-P to Megatron K… - #2459

Merged

AAnoosheh merged 14 commits into

mainfrom

aanoosheh/topk-kd-topp-ghost

Oct 5, 2026

Commits on Sep 17, 2026
- Bring offline KD upgrades such as Ghost Token and Top-P to Megatron KD plugin
  AAnoosheh
  committed
  b4112bd
- Don't expose ghost token in MBridge script
  AAnoosheh
  committed
  95c9aa7
- Address review comments
  AAnoosheh
  committed
  3f751f3
Commits on Sep 22, 2026
- Make ghost_token not-togglable
  AAnoosheh
  committed
  df93c40
Commits on Sep 23, 2026
- Address more reviews
  AAnoosheh
  committed
  32be4ee
- Merge branch 'main' into aanoosheh/topk-kd-topp-ghost
  
  AAnoosheh
  and
  claude
  committed
  ef8233c
- Reduce student LM loss to a scalar in distillation GPU tests
  
  AAnoosheh
  and
  claude
  committed
  c1fb187
Commits on Sep 30, 2026
- Clarify that the ghost token is always appended in CHANGELOG
  
  AAnoosheh
  and
  claude
  committed
  51a8b60
- Merge branch 'main' into aanoosheh/topk-kd-topp-ghost
  
  AAnoosheh
  and
  claude
  committed
  57fc703
Commits on Oct 1, 2026
- Hard error on deprecated KDConfig args for simplicity
  AAnoosheh
  committed
  c1857f0
- Address review: free teacher fp32 copy early, input-only skip_lm_loss, dtype-tied ghost floor
  
  AAnoosheh
  and
  claude
  committed
  2ad0e70
Commits on Oct 5, 2026
- Rename TopKLogitsKLLoss to TopLogitsKLLoss and --logit_kl_topk to --logit_kl_top_k
  
  AAnoosheh
  and
  claude
  committed
  02151b3
- Merge branch 'main' into aanoosheh/topk-kd-topp-ghost
  
  AAnoosheh
  and
  claude
  committed
  7bb3c08
- Reject removed DistillationConfig fields in __new__; make skip_lm_loss a property
  
  AAnoosheh
  and
  claude
  committed
  83c1a1d

Navigation Menu

Repository navigation

Bring offline KD upgrades such as Ghost Token and Top-P to Megatron K… - #2459

Bring offline KD upgrades such as Ghost Token and Top-P to Megatron K…#2459
AAnoosheh merged 14 commits into
mainfrom
aanoosheh/topk-kd-topp-ghost

Commits on Sep 17, 2026

Bring offline KD upgrades such as Ghost Token and Top-P to Megatron KD plugin

Don't expose ghost token in MBridge script

Address review comments

Commits on Sep 22, 2026

Make ghost_token not-togglable

Commits on Sep 23, 2026

Address more reviews

Merge branch 'main' into aanoosheh/topk-kd-topp-ghost

Reduce student LM loss to a scalar in distillation GPU tests

Commits on Sep 30, 2026

Clarify that the ghost token is always appended in CHANGELOG

Merge branch 'main' into aanoosheh/topk-kd-topp-ghost

Commits on Oct 1, 2026

Hard error on deprecated KDConfig args for simplicity

Address review: free teacher fp32 copy early, input-only skip_lm_loss, dtype-tied ghost floor

Commits on Oct 5, 2026

Rename TopKLogitsKLLoss to TopLogitsKLLoss and --logit_kl_topk to --logit_kl_top_k

Merge branch 'main' into aanoosheh/topk-kd-topp-ghost

Reject removed DistillationConfig fields in new; make skip_lm_loss a property