Repository navigation

[1/2] Add a chat JSONL output mode to prepare_megatron_data_blend - #2791

Merged

yueshen2016 merged 6 commits into

mainfrom

yueshen/data-blend-chat-jsonl

Oct 15, 2026

Merged

[1/2] Add a chat JSONL output mode to prepare_megatron_data_blend#2791
yueshen2016 merged 6 commits into
mainfrom
yueshen/data-blend-chat-jsonl

Commits on Oct 14, 2026
- Add a chat JSONL output mode to prepare_megatron_data_blend
  
  yueshen2016
  and
  claude
  committed
  eef01dc
- Key cached chat parts by their selection and honor zero-row budgets
  
  yueshen2016
  and
  claude
  committed
  f4a96f6
- Harden chat JSONL blend inputs and match the tokenizing reader
  
  yueshen2016
  and
  claude
  committed
  4359544
- Tighten chat JSONL row checks and consolidate the tests
  
  yueshen2016
  and
  claude
  committed
  4ecf574
- Check source-row media, keep system prompts and require target_samples
  
  yueshen2016
  and
  claude
  committed
  157897c
- Drop text rows with leading text and floor the give-up threshold
  
  yueshen2016
  and
  claude
  committed
  312129b