feat(kimi-k3): add DSpark Q8 artifact and shared speculative runtime

[davide221]

Summary

• add a validated, config-driven DSpark/DFlash Safetensors-to-GGUF converter
• publish the Kimi-K3 DSpark drafter as a 2.39 GB Q8_0 GGUF
• enable Kimi-K3 through the existing shared DFlash/DSpark runtime and normal --draft / --draft-device server configuration
• add bounded batched Kimi verification over the existing model-neutral NVMe MoE route-batch interface
• add ReplaySSM: speculative KDA state remains pristine and only accepted recurrent transitions are committed; MLA rollback remains position based
• keep DSpark Markov correction, confidence scheduling, acceptance, feature-ring management, and decode policy in common/
• add target cost hints so external-memory targets can avoid over-verification and prefer cheap captured-state rollback over a full SSD replay
• fix the shared host-capture path for F16/BF16/Q8 feature-ring storage

Kimi-specific code is limited to the architecture forward/state adapter. There is no second DSpark implementation or Kimi-specific speculative loop.

Artifact

Converted from RadixArk/Kimi-K3-DSpark revision 56ce616ad7486f0e96cbb51ef23ed5a1bce1d92d:

• 2,249,289,601 parameters; all 62 source tensors mapped exactly
• 4,498,585,858-byte BF16 source -> 2,390,153,888-byte GGUF (53.13%)
• 38 Q8_0 tensors and 24 F32 tensors
• sampled relative RMSE RMS: 0.00544127; maximum tensor: 0.00557332
• output SHA256: 848d12be5283a4717c08b1b1263420980dfc61b5a59067e343296fd7f4998435

Artifact: https://huggingface.co/Lucebox/Kimi-K3-DSpark-Q8_0-GGUF

Runtime validation

Validated on lucebox4 with the 14-shard Kimi-K3-UD-IQ1_S target (~594 GB), ROCm/gfx1151, SSD-routed experts, and an 8 GiB device expert cache:

• target load: 57.93 GiB resident + 495.26 GiB file-backed routed experts
• shared runtime loads the DSpark Markov/confidence heads and the F16 feature mirror
• autoregressive and DSpark paths produced the exact same IDs for the smoke sequence: 11 291 11 275 (a, b, c)
• ReplaySSM diagnostic: 3 fast commits, 0 full replays, 0 failed fallbacks
• adaptive verification reduced the first implementation from 73.4 GiB to 47.4 GiB of SSD payload and from 0.18 to 0.28 tok/s
• final short-run DSpark acceptance: 4/5 verified rows (80%)

The honest baseline on this single-Strix, IQ1_S target is still faster: 0.44 tok/s AR versus 0.28 tok/s DSpark. This PR establishes the correct reusable runtime and exposes batched work for the dual-owner R9700 + Strix path; it does not claim a speedup from a drafter trained against BF16 when the target is aggressively IQ1_S-quantized. Acceptance calibration and dual-device throughput tuning remain follow-up benchmark work.

The drafter device is explicit, so the Q8 model can remain on the R9700 while the existing heterogeneous expert placement owns target work:

--draft /models/Kimi-K3-DSpark-Q8_0.gguf --draft-device hip:<r9700>

The same path was also exercised through the OpenAI-compatible server API.

Tests

ROCm full build: passed
ctest: 346/346 passed
real Kimi-K3 AR/DSpark output-equivalence smoke: passed
real Kimi-K3 ReplaySSM no-replay diagnostic: passed
OpenAI-compatible server request: passed
converter ruff: passed
converter unittest: 3/3 passed
git diff --check: passed

Stacked on the NVMe streaming branch; retarget to main after the base PR lands.

[davide221]

Kimi-K3 DSpark workload probe on lucebox4

Tested the final runtime on one canonical-style prompt per workload with 24 greedy output tokens, the 14-shard Kimi-K3-UD-IQ1_S target, the published Q8 DSpark drafter, F16 feature mirror, SSD-routed experts, and an 8 GiB device expert cache.

Workload	Prompt tokens	Decode	Verify rows	Draft steps	Avg commit/step	SSD payload (whole run)
HumanEval truncate_number (repository canonical prompt)	81	0.13 tok/s	66	16	1.50	1103.1 GiB
GSM8K canonical arithmetic prompt	48	0.16 tok/s	48	14	1.71	698.6 GiB

ReplaySSM behaved correctly in both runs: 30 fast commits total, zero full replays, and zero failed fallbacks. Generated continuations were coherent and the GSM8K answer was correct (6).

Interpretation: GSM8K improves acceptance relative to HumanEval, but the accepted length remains far below the ~3-5 range normally needed for DSpark speedups. The drafter was trained against the BF16 target, while this target is aggressively IQ1_S-quantized; additionally, SSD expert traffic scales with every verified row. This is currently an acceptance/calibration + external-memory cost problem, not a missing Kimi runtime path.

[davide221]

DSpark acceptance-contract correction

Auditing against the published RadixArk checkpoint and current SGLang DSpark runtime found three concrete mismatches:

• block_size=7 means 7 draft proposals, while target verification is [current token] + 7 proposals (width 8). The shared Luce path previously discarded draft hidden row 0 and produced only 6 shifted proposals.
• the draft GGUF loader ignored the published YaRN-16 RoPE metadata.
• the graph hardcoded RMS epsilon instead of using the checkpoint value (1e-5).

Commit 098dcb42 fixes those contracts without changing legacy DS4/Laguna semantics, removes the fixed-size Markov readback, and adds a row-sensitive regression test proving every DSpark hidden row is consumed.

lucebox4 validation

ROCm/gfx1151, Kimi-K3-UD-IQ1_S target, Q8_0 DSpark draft, SSD-routed experts:

• corrected Kimi XTML GSM8K sample: 6.00 average committed tokens/target pass, 24 tokens in 4 target passes, histogram 4:2, 8:2, 75% accepted verification rows
• previous raw/unformatted prompt after the runtime correction: 2.40 average committed tokens/pass, illustrating that the native Kimi chat template materially affects acceptance
• canonical raw HumanEval truncate_number sample: correct completion, 3.43 average committed tokens/pass
• C++ regression suite: passed
• HIP test_deepseek4_unit and smoke_kimi_k3_forward build: passed

The GSM sample is now consistent with the model-card aggregate (5.4176 over 1,319 questions), but this is not a claim that one sample reproduces the full benchmark. The published numbers use the original evaluation stack/target; this deployment uses a Q8 drafter against an aggressively quantized IQ1_S target. A full dataset run is still required for an exact aggregate comparison.

[davide221]

Rebased onto the current NVMe/Kimi base (24045990), so this stack now inherits the allocation-safe automatic caches, lazy quant-padding initialization, and cache-first SSD overlap. The only rebase conflict was adjacent shared-runtime includes; both the dynamic-backend memory query and shared speculative runtime are retained.

Combined-tree validation on Lucebox4:

• full HIP + dynamically loaded CUDA build: passed
• mixed HIP/CUDA interop and native-memory reporting: passed
• streamed-expert numerical suite on HIP: passed
• streamed-expert numerical suite on CUDA: passed
• DSpark loader/proposal/verification contract tests: passed
• feature/architecture gate: 175 assertions, 0 failures
• git range-diff: the three DSpark commits are unchanged apart from the required include integration

The standalone Python converter unittest was not rerun on Lucebox4 because that host does not currently have NumPy installed; the previously validated converter commit is unchanged by the rebase.