[Zhehao-Hu]

[8/8] Day-0 speculative-decoding readiness flow. Merge in order 1 → 8. Base: [7/8] #2935. Uses everything before it.
Series: [1/8] #2839 · [2/8] #2930 · [3/8] #2931 · [4/8] #2932 · [5/8] #2933 · [6/8] #2934 · [7/8] #2935 · [8/8] #2936 (this)

What does this PR do?

Type of change: New feature

Completes the specdec-release agent skill. Asked "train a DFlash2 drafter for Qwen3-8B and tell me whether it's ready", the agent goes from no drafter to a readiness verdict in a fixed sequence, with a gate after every stage. It is a conductor: it submits the existing launcher pipelines and enforces the gates from [6/8] and [7/8]. It doesn't re-implement training, serving or benchmarking.

Scope: DFlash and DFlash2. Only dflash_online_training.sh writes the regression_check.json that gate_train reads, so EAGLE3 and DSpark (train_eagle*.sh) are refused before anything is submitted, as are Domino (no vLLM path) and LiLiCorr (SGLang only). Extending to EAGLE3 means making its training and serving scripts write the same verdict files.

setup ─▶ train pipeline (data → train+export → smoke) ─▶ freeze ─▶ A|B bench ─▶ closeout
         gate_train + gate_smoke                         freeze_drafter  gate_bench + gate_correctness (report)

Verdicts:

• DRAFTER_READY: every gate passed. Ready for release review; it is not a publish decision, and the gates set no per-model performance bar.
• INFEASIBLE: training, export or serving failed, and triage found no fix within the run.
• ANOMALOUS: the measurement is invalid after the allowed retries, or freeze failed.

SYSTEMIC (cluster or credentials down) aborts the run, as in day0-release; it is not a verdict.

plugins/modelopt/skills/specdec-release/:

• SKILL.md: inputs, the stage sequence, how to run each gate, state and resume, the triage table, and the report.
  • Inputs include existing_drafter, which skips training but not its gates, and approve_submissions: each | none. each is the default: the agent shows every cluster job (launcher, sbatch, weight staging, including a sub-agent's) and waits for an OK. One OK covers one exact job; a retry or a changed partition, image or command needs a new one.
  • Staging: Step 1 stages a target the cluster lacks: reuse a readable hf_local copy, else one download job into hf_local at a pinned revision, then check the shards. A 401/403 on a gated model is USER_CONFIG_ERROR. It lives here until it moves to the common skill for ptq and day0-release.
  • Do not advance past a failed gate, and never read a verdict from a log or a job state.
  • Watching: the agent starts watching in the same turn as every submission, and tracks result files rather than job states.
  • State: specdec_state.json records each stage's status, experiment and job ids, overrides and gate verdicts. On resume the agent re-runs nothing that passed, re-watches instead of resubmitting, and re-verifies the frozen drafter before benchmarking.
  • Log: every step goes to work_log.md as ACTION or HUMAN_INTERVENTION, so interventions can be counted.
  • Report: SPECDEC_REPORT.md, with the setup, gate verdicts, the frozen drafter's checksums and the A|B table.
  • Triage of CHECKPOINT_NOT_SERVABLE acts on the smoke JSON's stage, not the log's last traceback: server_timeout resubmits with a larger SERVER_TIMEOUT ([2/8]).
• scripts/freeze_drafter.py: copies the export to /cicd/drafts/<target>/<algo>/<run_id> through a .partial sibling, re-hashes before the rename, writes SHA256SUMS, and makes the files read-only. It returns a manifest_sha256, and --verify --manifest-sha256 re-checks it on resume. The bench serves the frozen copy, which freeze proves byte-identical to the export at freeze time.
• scripts/gate_train.py --export-only: checks an existing drafter's export without a training verdict.
• tests/: freeze_drafter and --export-only cases, plus evals.json with 6 trigger and non-trigger prompts.

plugins/modelopt/skills/speculative-decoding/:

• SKILL.md routes the end-to-end flow to specdec-release.
• references/stages/configure.md gains "fit every task to the cluster":
  • set gpus_per_node to the node size on whole-node clusters;
  • when a training task's GPU count changes, scale gradient_accumulation_steps to keep the global batch.

Usage

Use specdec-release to train a DFlash2 drafter for Qwen/Qwen3-8B on OCI-JHB with the dev budget, and tell me whether the drafter is ready for release review.

Resume the spec-dec run in this directory.

Testing

• Skills suite on this commit: 163 passed, 1 skipped. Launcher suite: 192 passed.

• End-to-end on OCI-JHB (GB300) with Claude Code, unattended apart from submission approvals:

  Run	Path	Result
  Variance pass	existing_drafter (M0 drafter) and full train (seed 1)	13 gate and freeze verdicts, all passed; both drafters DRAFTER_READY
  Rehearsal 1 (fresh session, only the plugin and a launcher env)	full train	DRAFTER_READY, 0 interventions. Exposed a fresh-venv launcher bug, fixed in Pin "leptonai<0.47" to fix tools/launcher CI job #2892 and [launcher] Add build-system so uv run installs modelopt_launcher #2921
  Rehearsal 2	full train, then killed and resumed in a new session	DRAFTER_READY; the resume re-verified the freeze and re-ran nothing. 3 unlogged nudges, fixed here (watch after every submission, load the cluster in every shell, run dir)
  Rehearsal 3	full train, one session	DRAFTER_READY, 0 interventions and no nudges. Watched and re-armed its own watchers, ran every remote command through remote_run, and caught two SSH failures at submit by checking squeue (nothing queued), retrying with approval

• Results across the variance pass and rehearsals:

  • training loss 6.37–6.71, accuracy 0.167–0.175;
  • smoke AL 1.68–1.75;
  • bench AL 1.71–1.79, speedup 1.36–1.42 at C=1 and 1.23–1.33 at C=32;
  • greedy exact match 0.956–0.963.

• In the rehearsals, the bench and smoke jobs exited 7–17 s after writing their results, with no hangs.

• First new target, Qwen3.8-27B + DFlash2 (no prior example): DRAFTER_READY (loss 2.93, smoke AL 1.76, speedup 1.40/1.29/1.06 at C=1/8/32), with 6 interventions beyond approvals. They led to the staging step, the approval scope and stage-first triage above, and [2/8]'s SERVER_TIMEOUT.

• Not run: the INFEASIBLE and ANOMALOUS triage paths on GPU (their gates' failure classes are unit-tested), local Docker job hosts, and DFlash end to end (in scope, but only DFlash2 was run).

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

• Is this change backward compatible?: ✅ (new skill; one routing line and one configure note in speculative-decoding)
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A (stdlib only)
• Did you write any new necessary tests?: ✅
• Did you update Changelog?: ✅ (Misc entry)
• Did you get Claude approval on this PR?: ❌ - not yet run

Additional Information

• Consistent with plugins/modelopt/skills/day0-release: the same gate contract, verdict style and use of the common skill's remote_exec.sh.
• About 590 lines of source, over the ~500 guideline. Roughly 300 of them are SKILL.md, which has to describe the whole flow in one place for the agent.
• Next: release criteria (SPEED-Bench on GB200, thresholds with a repeat policy) and approve_submissions: none once the flow has a clean track record.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}