Repository navigation
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
This was referenced Oct 22, 2026
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash-reexport
branch
from
October 22, 2026 17:59
3ac8822 to
9a9b3cc
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash2-examples
branch
2 times, most recently
from
October 22, 2026 22:09
2370f99 to
efc7bce
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash-reexport
branch
2 times, most recently
from
October 23, 2026 00:09
390e821 to
a2d6478
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash2-examples
branch
from
October 23, 2026 00:09
efc7bce to
82d5b56
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash-reexport
branch
from
October 23, 2026 04:51
a2d6478 to
dbddd39
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash2-examples
branch
from
October 23, 2026 04:51
82d5b56 to
8a2bd57
Compare
Zhehao-Hu
added a commit
that referenced
this pull request
Oct 23, 2026
…strict (#2839) > **[1/8] Day-0 speculative-decoding readiness flow.** Merge in order 1 → 8. Base: `main`. > Series: [1/8] #2839 (this) · [2/8] #2930 · [3/8] #2931 · [4/8] #2932 · [5/8] #2933 · [6/8] #2934 · [7/8] #2935 · [8/8] #2936 ### What does this PR do? Type of change: Bug fix [common/check_regression.py](https://github.com/NVIDIA/Model-Optimizer/blob/main/tools/launcher/common/check_regression.py) gates DFlash-family training runs on final loss and accuracy. Four things stopped that gate from catching a bad run: 1. **It read the wrong checkpoint on long runs.** `find_trainer_state()` sorted `checkpoint-*` as strings, so `checkpoint-8000` ranked after `checkpoint-12000` and an older checkpoint's metrics were checked. Any run whose saved steps differ in digit count was affected; the ~92K-step full DFlash2 recipe in `hf_online_dflash2.yaml` saves every 4000 steps. It now sorts by step number. 2. **An unreadable run passed.** With thresholds set, a missing `trainer_state.json`, or one with no training logs, only warned and exited 0. An empty threshold variable (`MAX_FINAL_LOSS=""`) crashed on `float('')`. 3. **A regression never failed the job.** `dflash_online_training.sh` runs the check as `check_regression.py … || true`, so Slurm reports `COMPLETED` even when the log says `REGRESSION`. The verdict also existed only as log text. 4. **A crashed run was still graded.** The script's `ERR` trap only records a failure, so after a failed `accelerate launch` it went on to export and run the check, and a checkpoint that survived the crash could pass. Changes: - `check_regression.py`: - Sorts checkpoints by step number. - Fails when thresholds are set but there are no metrics to check, including a `trainer_state.json` of the wrong shape. An empty threshold variable counts as unset. A threshold that isn't a finite number (`7,0`, `nan`) fails as `METRICS_MISSING`. - A NaN or infinite value for a metric that has a threshold fails as `TRAIN_REGRESSION`. - Takes `--json <path>` and writes `{pass, failure_class, trainer_state, metrics, thresholds, messages}` as strict JSON (non-finite metrics as `null`). `failure_class` is `null`, `TRAIN_REGRESSION` or `METRICS_MISSING`. The verdict is serialized before the file is opened, so a failure can't leave a truncated file. - Its existing log lines (`=== Regression Check (…) ===`, `PASS:` / `REGRESSION:`, `Regression check PASSED`) are unchanged, so existing log readers keep working. A failure now also prints `Regression check FAILED`. - `dflash_online_training.sh` writes the verdict to `${OUTPUT_DIR}/regression_check.json`, next to the exports. `REGRESSION_STRICT=1` drops the `|| true`, so a regression fails the task through the existing `ERR` trap and `exit_handler`. **The default (`0` or unset) is unchanged (warn only)**, so the other examples with possibly uncalibrated thresholds don't start failing. Any other value fails the task at startup, before installing or training. - `dflash_online_training.sh` also skips export and the check when training fails, still reporting the failure like any `ERR`-trap failure, and removes an old `regression_check.json` first thing, so no early exit can leave an earlier run's verdict behind. - `examples/Qwen/Qwen3-8B/hf_online_dflash2.yaml`: - Sets `REGRESSION_STRICT: "1"`. - Recalibrates `MAX_FINAL_LOSS` from 5.0 to 7.0. The 5.0 was inherited from `hf_online_dflash.yaml`, whose own reference curve only drops below 5.0 at step ~2500 of a 12.5K-step run. DFlash2 adds a selector loss term and this example stops at 2000 steps, so 5.0 fails every healthy run. `MIN_FINAL_ACC` stays at 0.15. - The `speculative-decoding` skill's `dflash.md` and `dflash2.md` describe the new gate: judge a run by `regression_check.json`, the two failure classes, strict mode, and DFlash2's 7.0 threshold. ### Usage ```bash MAX_FINAL_LOSS=7.0 MIN_FINAL_ACC=0.15 \ python3 common/check_regression.py /scratchspace/dflash2_bs16 --json /scratchspace/dflash2_bs16/regression_check.json echo $? # 0 pass, 1 regression or nothing to check ``` ```yaml environment: - MAX_FINAL_LOSS: "7.0" - REGRESSION_STRICT: "1" # fail the training task on a regression ``` ### Testing - `tools/launcher/tests/test_check_regression.py`: 4 new test functions (13 cases). Run against `main`'s `check_regression.py`, 17 of the file's 18 cases fail: - `test_latest_checkpoint_is_chosen_by_step_number` guards the string-sort bug. - `test_empty_threshold_var_is_unset` guards the `float('')` crash. - `test_cli_exit_code_and_json_verdict` is parametrized over a missing `trainer_state.json`, a truncated one, two of the wrong shape, a NaN loss, an unchecked infinite metric, a regression and a pass. Each case checks the exit code and parses the verdict as strict JSON, so it covers the missing-metrics hole and the `--json` contract. - `test_invalid_threshold_writes_failed_verdict` covers `7,0`, `nan` and `inf`. - `tools/launcher/tests/test_dflash_online_training.py` (5 cases) runs the real script in a sandbox: stubbed `pip`, `nvidia-smi`, `accelerate`, exporter and `torch`, a checkpoint, and an old passing verdict. Each case asserts the exact exit code. Against `main`'s script, all 5 fail: - A failing launch leaves no verdict and exits 1; a succeeding one grades the checkpoint and exits 0 with no `ERR`-trap error. - With a checkpoint over the threshold, `REGRESSION_STRICT=1` exits 1 and `0` exits 0, both writing `TRAIN_REGRESSION`. - `REGRESSION_STRICT=true` exits 1 without launching training and leaves no verdict. - Full launcher suite on this commit: `uv run --with pytest python3 -m pytest tests/`, 183 passed. - `dflash_online_training.sh`: `bash -n` passes. - **GPU (OCI-JHB, GB300, global batch 8):** five 2000-step runs of this example with this change passed the strict gate. Final loss was 6.37–6.71 and accuracy 0.167–0.177, against 7.0 / 0.15. The calibration run that motivated 7.0 (before this change) finished at 6.51 / 0.181. The seed-to-seed spread puts the loss margin at about 4%; see the variance note under Additional Information. - Not run: the strict-failure path and a real training crash on GPU (covered by the sandbox test above). ### Before your PR is "*Ready for review*" Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md) and your commits are signed (`git commit -s -S`). Make sure you read and follow the [Security Best Practices](https://github.com/NVIDIA/Model-Optimizer/blob/main/SECURITY.md#security-coding-practices-for-contributors) (e.g. avoiding hardcoded `trust_remote_code=True`, `torch.load(..., weights_only=False)`, `pickle`, etc.). - Is this change backward compatible?: ✅ (strict mode is opt-in; log output unchanged) - If you copied code from any other sources or added a new PIP dependency, did you follow guidance in `CONTRIBUTING.md`: N/A - Did you write any new necessary tests?: ✅ - Did you update [Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?: N/A - Did you get Claude approval on this PR?: ❌ - not yet run ### Additional Information - Part of the Day-0 speculative-decoding workflow: the `specdec-release` skill ([8/8]) gates training on `regression_check.json`, so it needs a verdict it can read and a job state that reflects it. - **Why not reuse `day0-release/scripts/gate_run.py`?** The two sit at different layers and check different things, so their schemas stay independent on purpose: - `gate_run.py` is an agent-side gate for an *evaluation* run. It reads a run summary the agent builds from NEL output, and checks per-task status, sample counts and judge errors. None of that exists for a training run. - `check_regression.py` runs *inside* the Slurm training task, on rank 0, and its verdict decides the task's exit status in strict mode. It reads `trainer_state.json` against thresholds from the launcher YAML, and has to work for launcher users who never load a skill. A launcher script importing a skill script would also invert the dependency: skills drive the launcher, not the other way round. - The skill-side counterpart of `gate_run.py` for training is `specdec-release/scripts/gate_train.py` ([6/8] #2934). It reads `regression_check.json` and returns the day-0 gate shape (`{pass, failure_class, detail, …}`), with `failure_class` from the shared triage vocabulary. That is the adapter layer, so `check_regression.py` doesn't need to match `gate_run.py`. - What the two share is the convention: a boolean `pass`, and a `failure_class` that is `null` on a pass. The launcher emits only `METRICS_MISSING` and `TRAIN_REGRESSION`, and `gate_train.py` treats any other value as `TRAIN_REGRESSION`, so the launcher can add detail fields without breaking the gate. - Other Qwen3-8B examples share the inherited `MAX_FINAL_LOSS: "5.0"` (`hf_online_domino.yaml`, `hf_online_lilicorr.yaml`). They are left unchanged and non-strict because there is no calibration run for them. - Variance: a seed-1 retrain landed at loss 6.705, 4.2% under the 7.0 floor. Accuracy has more headroom (11%). Recalibrating from more seeds is planned for the release gates, not this PR. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Training runs with an output directory now produce a regression-check verdict file reporting whether configured quality thresholds passed. * Regression checks account for missing or invalid metrics and select checkpoints by numeric step. * **Bug Fixes** * Failed training no longer proceeds to checkpoint export or regression checks. * Strict mode fails the task when regression checks fail; invalid strict-mode values stop the run before training starts. * **Documentation** * Updated DFlash2 guidance and example settings for the final-loss threshold and strict-mode behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Zhehao Hu <zhehaoh@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash-reexport
branch
from
October 23, 2026 21:24
dbddd39 to
6d46dab
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash2-examples
branch
from
October 23, 2026 21:24
8a2bd57 to
5b20c8d
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash-reexport
branch
from
October 23, 2026 21:35
6d46dab to
d217cd2
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash2-examples
branch
from
October 23, 2026 21:35
5b20c8d to
406058d
Compare
Zhehao-Hu
added a commit
that referenced
this pull request
Oct 23, 2026
…ction (#2930) > **[2/8] Day-0 speculative-decoding readiness flow.** Merge in order 1 → 8. Base: `main` ([1/8] has merged). > Series: [1/8] #2839 (merged) · [2/8] #2930 (this) · [3/8] #2931 · [4/8] #2932 · [5/8] #2933 · [6/8] #2934 · [7/8] #2935 · [8/8] #2936 ### What does this PR do? Type of change: Bug fix The vLLM smoke test (`common/specdec/vllm_smoke_test.sh`) is the serving check after draft training. It had six problems: 1. **It could serve the wrong checkpoint.** `ar_eval_mtbench.sh` and `vllm_smoke_test.sh` picked the newest checkpoint by sorting on a dash-separated field of the **whole path**, so a dash in a parent directory (e.g. `/data/my-drafts/...`) picked the wrong one. `quantize_drafter.sh` had a third, different sort. The smoke test also ignored `exported-checkpoint-final`. 2. **Every throughput figure printed `?`.** Timing used `bc`, which the vLLM image doesn't have. 3. **A drafter that never speculated passed.** With `MIN_ACCEPTANCE_LENGTH` set, a vLLM log with no acceptance length only warned. 4. **The acceptance length covered only the last few seconds.** It came from the log's "Mean acceptance length" line, which covers only vLLM's last logging interval. 5. **The verdict existed only as log text.** 6. **The server wait was fixed at 180 s.** Qwen3.8-27B with a DFlash2 drafter takes longer to start on GB300 (torch.compile, kernel JIT, CUDA graph capture, GDN warmups), so every smoke attempt stopped at `server_timeout`. The script's exit then tore the container down mid-warmup, and the engine's next exec failed with a ptxas ENOENT that read like the cause. Changes: - `service_utils.sh`: new `latest_checkpoint <dir> <prefix>`. It returns `<dir>/<prefix>final` if present, else the highest numeric step, keyed on the directory's own name. It replaces the three ad hoc sorts in `ar_eval_mtbench.sh`, `vllm_smoke_test.sh` and `quantize_drafter.sh`. - `vllm_smoke_test.sh`: - Prefers `exported-checkpoint-final`, as `quantize_drafter.sh` already did. - Times prompts with shell arithmetic and `python3` instead of `bc`. - Computes the mean acceptance length over the whole run from vLLM's cumulative `/metrics` counters: 1 + `spec_decode_num_accepted_tokens_total` / `spec_decode_num_drafts_total`. - With `MIN_ACCEPTANCE_LENGTH` set, fails when no speculative drafts were recorded. - Fails loudly if `service_utils.sh` is missing (it used to be optional, so the failure surfaced as a misleading `draft_missing`). Only a missing file is fatal: `source` returns the status of the file's last command, a rank-0 append that fails outside a launcher working directory. - `SERVER_TIMEOUT` sets the server wait in seconds (default 180) and appears in the timeout reason. - Checks `MIN_ACCEPTANCE_LENGTH` (a finite number), and `NUM_SPEC_TOKENS` and `SERVER_TIMEOUT` (positive integers without leading zeros, since `NUM_SPEC_TOKENS` goes into vLLM's JSON config), before any work, and fails with `stage: invalid_config` and a reason naming the value. Both checks run after the `EXIT` trap is set, so even these failures replace any earlier `SMOKE_JSON`. - Optional `SMOKE_JSON=<path>` writes the verdict on every exit: `{stage, pass, reason, method, target, draft, num_speculative_tokens, prompts passed/failed, tokens, average tok/s, mean_acceptance_length, min_acceptance_length}`. `stage` says how far it got (`invalid_config`, `starting`, `draft_missing`, `server_died`, `server_timeout`, `serving`, `completed`), so a downstream gate can tell "not servable" from "below the floor". The file's parent directory is created if needed, and it is strict JSON (a non-numeric value is written as `null`). The mean acceptance length is checked against the floor and written at full precision; only the log rounds it. ### Usage ```yaml task_2: script: common/specdec/vllm_smoke_test.sh environment: - HF_MODEL_CKPT: <<global_vars.hf_model>> - DRAFT_CKPT_DIR: <<global_vars.output_dir>> # newest export, final preferred - SPEC_METHOD: "dflash" - NUM_SPEC_TOKENS: "7" - MIN_ACCEPTANCE_LENGTH: "1.5" - SMOKE_JSON: <<global_vars.output_dir>>/smoke_check.json - SERVER_TIMEOUT: "900" # optional; default 180 s ``` ### Testing - `tools/launcher/tests/test_service_utils.py`: `latest_checkpoint` with a dash in a parent directory, `final` winning over steps, numeric rather than string order, and no checkpoint at all (3 cases). - `tools/launcher/tests/test_vllm_smoke_test.py` runs the script as rank 0 from a directory with no `modules/` (stubbed `pip` and `nvidia-smi`), up to the paths that stop before vLLM starts (6 cases). Against the script without these fixes, all 6 fail: - It reaches `draft_missing` rather than stopping at `source`. - `MIN_ACCEPTANCE_LENGTH` of `1,5` or `nan`, `NUM_SPEC_TOKENS` of `x` or `08` and `SERVER_TIMEOUT=15m` each fail as `invalid_config` with a strict-JSON `SMOKE_JSON`. Before, `NUM_SPEC_TOKENS=x` crashed in the `EXIT` trap and wrote no JSON. - `bash -n` on all four scripts. Launcher unit suite on this commit: 192 passed. - **GPU (OCI-JHB, GB300, vLLM v0.31.0)**, through `hf_online_dflash2.yaml`'s smoke task ([4/8]) and one smoke-only submission, six runs in all: - `latest_checkpoint` picked `exported-checkpoint-final` in every training run. - Per-prompt tok/s printed (e.g. 335 tok/s average), 8/8 prompts passed. - Mean acceptance length 1.68–1.79 against the 1.5 floor; `smoke_check.json` written each time. - **GPU, Qwen3.8-27B + DFlash2 (same cluster and image):** four smoke runs at the old fixed 180 s each wrote `smoke_check.json` with `stage: server_timeout`. A fifth with `SERVER_TIMEOUT=900` passed at AL 1.76. - Not run on GPU: - the `server_died` and no-drafts failure paths. They exit through the same `write_smoke_json` trap as the passing path; - creating `SMOKE_JSON`'s parent directory. A smoke-only run hit the missing directory before this was added; the rerun used an existing directory. ### Before your PR is "*Ready for review*" Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md) and your commits are signed (`git commit -s -S`). Make sure you read and follow the [Security Best Practices](https://github.com/NVIDIA/Model-Optimizer/blob/main/SECURITY.md#security-coding-practices-for-contributors) (e.g. avoiding hardcoded `trust_remote_code=True`, `torch.load(..., weights_only=False)`, `pickle`, etc.). - Is this change backward compatible?: ✅ (`SMOKE_JSON` is optional. With `MIN_ACCEPTANCE_LENGTH` set, a run with no speculative drafts now fails instead of passing; that was the bug.) - If you copied code from any other sources or added a new PIP dependency, did you follow guidance in `CONTRIBUTING.md`: N/A - Did you write any new necessary tests?: ✅ (`latest_checkpoint`; the smoke script itself is validated on GPU as above) - Did you update [Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?: N/A - Did you get Claude approval on this PR?: ❌ - not yet run ### Additional Information - The `specdec-release` skill's `gate_smoke.py` ([6/8]) reads `SMOKE_JSON`. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Improvements** * Checkpoint selection now prefers a `final` checkpoint, or otherwise chooses the highest-numbered checkpoint. * Smoke tests validate numeric settings, use a configurable server timeout, and report run results, failure reasons, and performance metrics in an optional JSON file. * Acceptance-length checks now use cumulative draft and accepted-token counts; prompt failures and unmet acceptance thresholds result in a failed test. * **Tests** * Added coverage for checkpoint selection and smoke-test configuration failures. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Zhehao Hu <zhehaoh@nvidia.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
hf_online_dflash2.yaml: - Add task_2: vLLM smoke test of the newest export (DRAFT_CKPT_DIR), gated on MIN_ACCEPTANCE_LENGTH 1.5 with 7 spec tokens, a floor that a healthy 2000-step run of this config clears. - Run task_0 and task_1 on the same TRT-LLM image (1.3.0rc20); task_0 was pinned to rc10 while task_1 took the factory default. Both examples: - Route every artifact path through global_vars.output_dir (training output, exports, SMOKE_JSON), so one override moves the producer and its consumer together, e.g. to rerun the smoke task against an earlier experiment. - Pin vllm/vllm-openai:v0.31.0 (multi-arch, contains vllm#52816 for DFlash2) instead of latest/nightly, and set TRITON_CACHE_DIR node-local. - hf_streaming_dflash2.yaml exports to <output_dir>/export via EXPORT_PATH. Signed-off-by: Zhehao Hu <zhehaoh@nvidia.com>
vLLM serves DFlash2 with a block of 1 + num_speculative_tokens (8 with the examples' NUM_SPEC_TOKENS 7) and ignores the checkpoint's block_size, while both examples trained on 16-token blocks. A dev-budget rerun of the online example with block 8 gave the same acceptance length as block 16 (1.75 at concurrency 1/8/32), so the served block costs nothing and removes a train/serve mismatch. It also matches the library default and the published DFlash2 drafters. The online example drops dflash_loss_decay_factor: the recipe's dpace objective ignores it, as the training log warns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Zhehao Hu <zhehaoh@nvidia.com>
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash-reexport
branch
from
October 28, 2026 00:21
d217cd2 to
84467d2
Compare
Zhehao-Hu
force-pushed
the
zhehaoh/launcher-dflash2-examples
branch
from
October 28, 2026 00:21
406058d to
c81e5eb
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Type of change: New example
Cleans up the two Qwen3-8B DFlash2 launcher examples so one submission trains, exports and serving-checks a drafter, and every artifact path moves with one override.
hf_online_dflash2.yaml:DRAFT_CKPT_DIR, final preferred), gated atMIN_ACCEPTANCE_LENGTH1.5 with 7 speculative tokens. A healthy 2000-step run of this config clears it (1.68–1.79 measured).1.3.0rc20). task_0 was pinned torc10while task_1 took the factory default.dflash_loss_decay_factor: the recipe'sdpaceobjective ignores it, as the training log warns.Both examples:
global_vars.output_dir(training output, exports,SMOKE_JSON). One override moves producer and consumer together, e.g. to rerun the smoke task against an earlier experiment.vllm/vllm-openai:v0.31.0(multi-arch; contains vllm#52816 for DFlash2) instead oflatest/nightly.TRITON_CACHE_DIRnode-local, so parallel jobs on different nodes never share a Lustre cache folder.hf_streaming_dflash2.yamlexports to<output_dir>/exportviaEXPORT_PATH.dflash_block_size=8(was 16, the recipe's value). vLLM serves DFlash2 with a block of 1 +NUM_SPEC_TOKENS= 8 and ignores the checkpoint'sblock_size, so the examples trained on a block they never serve. Block 8 also matches the library default and the published DFlash2 drafters.Usage
Testing
uv run launch.py --yaml … --to-yamlresolves both examples. Launcher unit suite on this commit: 191 passed.gpus_per_node=4andtraining.gradient_accumulation_steps=2to keep the global batch at 8. Each took about 40 minutes end to end, plus queue time. Results: loss 6.37–6.71, accuracy 0.167–0.177, smoke AL 1.68–1.79.hf_streaming_dflash2.yamlon GPU (path changes only, checked with--to-yaml).Before your PR is "Ready for review"
Make sure you read and follow Contributor guidelines and your commits are signed (
git commit -s -S).Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded
trust_remote_code=True,torch.load(..., weights_only=False),pickle, etc.)./scratchspace/exportto/scratchspace/dflash2/export.)CONTRIBUTING.md: N/AAdditional Information
speculative-decodingskill's configure stage ([8/8]) covers fittinggpus_per_nodeand the accumulation steps.