DeepSeek V4 Flash on Strix Halo (gfx1151): 4 defects in the monolithic DS4 path, 2 validated patches attached

[GRAVESIS]

Reported against Luce-Org/lucebox at commit c3b71e4b2ed2d08db96d55e7985f20035aa08d6f, using the ROCMFP2 Strix quant from Hugging Face Lucebox/DeepSeek-V4-Flash-ROCMFPX.

Found while qualifying local models on a two-node Strix Halo cluster. Everything below comes from the single-node monolithic DS4 configuration on one of those boxes. Four defects, two of them with patches that are validated on this hardware and inlined at the end, one with a working sidecar card, and one diagnosed but not fixed. Happy to raise the patched items as PRs instead if that is easier to review.

Summary

#	Defect	Severity	Status
1	Non-fused DSpark verify reads past the logits tensor and aborts	Fatal, deterministic	Patch below, validated
2	Prefix cache never stores or restores for DeepSeek models	Severe performance	Patch below, validated
3	No deepseek4 model-card family, so reasoning silently disables	Silent quality loss	Root cause named, working card below, code fix suggested
4	DSpark output is not greedy-identical to target-only decoding	Correctness question	Diagnosed, not fixed

Environment

• AMD Ryzen AI Max+ 395 (Strix Halo), gfx1151, wave size 32, 128 GB unified memory, 122880 MiB reported to the HIP device
• ROCm 7.2.4, HIP backend, dflash_server built as build-hip-strix from the commit above
• Target: DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf, 102,320,631,200 bytes, sha256 8fa6c30d9badd8e72f83c62952fe24b2e8cd647e103aff086bece328cb9c9208
• Drafter: DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf, 11,304,737,056 bytes, sha256 48883d35b8a67ecfd2858a90e12a47d04cb5ac581acef868ca0f58544816f746
• Common serving flags: --target-device hip:0 --ds4-fused-decode --ds4-expert-top-k 6 --cache-type-k f16 --cache-type-v f16 --max-ctx 16384 --chunk 2048
• Backend banner: [deepseek4] monolithic execution requested, fused_decode=on, 43 layers, 256 experts (6 routed), 97161.2 MB GPU buffer

Defect 1: non-fused DSpark verify aborts on any multi-token batch

Symptom

With speculative decoding on and DFLASH_DS4_FUSED_VERIFY unset, the server aborts on the first speculative step:

[ds4-spec] hidden nnan=0/20480 rms=0.1703 ctx_len=0
[ds4-spec] dbg ds_ok=1 q=2 lt=35 draft=[35 223 -1 -1]
/src/server/deps/llama.cpp/ggml/src/ggml-backend.cpp:384: GGML_ASSERT(offset + size <= ggml_nbytes(tensor) && "tensor read out of bounds") failed
...
libggml-base.so.0(ggml_abort+0x11f)
libggml-base.so.0(ggml_backend_tensor_get+0x1d1)
dflash_server() [0x4c3fa8]
dflash_server() [0x542747]
dflash_server() [0x547953]

Reproduced twice with identical stacks.

Root cause

Resolved by disassembling the symbolled binary. The failing call is server/src/deepseek4/deepseek4_graph.cpp:6684, reached through run_deepseek4_dspark_spec_decode to DeepSeek4DFlashTarget::verify_batch to deepseek4_dspark_verify_forward to deepseek4_step_layer_range.

In the dynamic output block:

6643  const bool last_only = n_tokens > 1;
6644  const int output_tokens = last_only ? 1 : n_tokens;
6646  inp = ggml_new_tensor_2d(ctx, GGML_TYPE_F32, n_embd, output_tokens);
...
6682  if (verify_hooks && verify_hooks->all_logits_out) {
6683      verify_hooks->all_logits_out->resize((size_t) w.n_vocab * n_tokens);
6684      ggml_backend_tensor_get(logits, verify_hooks->all_logits_out->data(), 0,
6685                              sizeof(float) * (size_t) w.n_vocab * n_tokens);

For a verify batch of n_tokens the logits tensor holds one row, while the hook reads n_tokens rows. deepseek4_dspark_verify_forward always sets hooks.all_logits_out (deepseek4_dspark_spec.cpp:379-386), so every non-fused multi-token verify overruns by exactly n_tokens - 1 rows. The failure cannot depend on data. Single-token sequential verify is safe.

The fused verify graph handles widths 2 to 4 and is gated on the environment flag at deepseek4_graph.cpp:6019-6021, which is why the published recipe never hits this path.

Reproduction

DFLASH_DS4_SPEC=1 \
DFLASH_DS4_DRAFT=/models/DeepSeek-V4-Flash-DSpark-draft-Q4RMFP4-denseF16.gguf \
DFLASH_DS4_SPEC_Q=4 \
LUCE_MMVQ_MAX_NCOLS=4 \
DFLASH_DS4_DSPARK_DEBUG=1 \
./server/build-hip-strix/dflash_server /models/DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf \
  --target-device hip:0 --ds4-fused-decode --ds4-prefill exact \
  --ds4-expert-top-k 6 --cache-type-k f16 --cache-type-v f16 \
  --max-ctx 16384 --chunk 2048

DFLASH_DS4_FUSED_VERIFY deliberately unset. Send any chat completion. The server aborts as soon as the drafter proposes more than one token.

Fix

-            const bool last_only = n_tokens > 1;
+            const bool need_all_logits = verify_hooks && verify_hooks->all_logits_out;
+            const bool last_only = n_tokens > 1 && !need_all_logits;
             const int output_tokens = last_only ? 1 : n_tokens;

Clamping the read instead would be unsound. The single physical row holds the prediction after the last input token, so a clamped copy to offset zero mislabels it as the first position and the remaining rows stay zero-filled. deepseek4_dspark_verify_forward takes a per-row argmax to decide acceptance, so acceptance decisions would be silently wrong rather than noisy.

Validation

Patched binary sha256 e630f1b0b78cf8648855dc90f268336a52422643781d9559f18da187b23e5a57. All four probes (warmup, sustained 2048-token decode, maths, code) complete with no abort. Two independent source reviews agreed the wider projection is the sound minimal repair and confirmed the allocator supports it: the dynamic output graph has invariant topology, and ggml_gallocr re-reserves when a later graph needs larger tensors (ggml-alloc.c:996-1058).

One reviewer suggested deriving output_tokens directly and adding a shape assertion before the read, which looks worth doing to stop a later edit reintroducing the mismatch.

Note on exit status

The process exits 139 rather than the 134 expected from abort(). Tracing shows GGML_ASSERT reaching plain abort() with no registered callback, so the segfault appears to come from teardown or a second thread. It does not affect the diagnosis, but it may indicate a separate shutdown fault.

Defect 2: the prefix cache never stores or restores for DeepSeek models

Symptom

With --prefix-cache-slots 32 (startup confirms prefix_cache = 32 slots), a three-turn conversation whose prompts strictly extend one another (7,016 then 7,051 then 7,067 tokens) logs on every request:

[pc] enabled: cap=32 family=laguna
[server] chat CACHE chatcmpl_0000000000000000 restore=false slot=-1 prefix_len=0 effective_prompt=7016 ... snap_slot=-1 snap_pos=0 full_snap_slot=-1 full_snap_pos=0
[server] chat CACHE chatcmpl_0000000000000001 restore=false slot=-1 prefix_len=0 effective_prompt=7051 ... snap_slot=-1 snap_pos=0 full_snap_slot=-1 full_snap_pos=0
[server] chat CACHE chatcmpl_0000000000000002 restore=false slot=-1 prefix_len=0 effective_prompt=7067 ... snap_slot=-1 snap_pos=0 full_snap_slot=-1 full_snap_pos=0

Every turn re-prefills the whole history, about 365 seconds each at roughly 19 tokens per second. That makes multi-turn agent use impractical.

Root cause, two independent defects

Chat markers mis-resolve. resolve_chat_markers (server/src/server/prefix_cache.cpp:15) tries qwen, then gemma, then falls through to a laguna branch (line 40) that only requires tok.encode("<system>") and friends to be non-empty, which holds for any BPE tokenizer. The cache then searches DeepSeek prompts for <system> sequences that never occur. find_all_boundaries returns empty and prepare_inline_snap returns {-1, 0} (prefix_cache.cpp:273-284), so no snapshot target is ever reserved.

The monolithic backend cannot snapshot. DeepSeek4Backend::snapshot_save (server/src/deepseek4/deepseek4_backend.cpp:1072) is a stub:

bool DeepSeek4Backend::snapshot_save(int slot) {
    if (slot < 0 || slot >= PREFIX_SLOTS) return false;
    // TODO: Implement snapshot save (copy KV cache + HC state to CPU)
    return false;
}

restore_and_generate_impl (line 1093) is likewise a TODO that falls through to a full prefill, and generate_impl never reads GenerateRequest::snap_slot or snap_pos. Since the HTTP layer only commits a cache entry when backend_.snapshot_used(...) returns true (http_server.cpp:2983,3002), nothing is ever cached. Either --ds4-fused-decode or a non-exact --ds4-prefill forces this backend through requires_monolithic_model() (deepseek4_backend.cpp:349).

No configuration works around it. --prefill-cache-slots N enables only an exact whole-prompt hash, useless for extending turns, and it still depends on backend snapshot support. The layer-split adapter does implement DS4 snapshots (deepseek4_layer_split_adapter.cpp:626,685) but requires multi-device placement and forbids dense prefill and fused decode.

Reproduction

Start the server with the common flags above plus --prefix-cache-slots 32 --prefill-cache-slots 0 --disk-prefix-cache off, then send three chat completions where each prompt is the previous conversation plus one short turn. Every request logs restore=false slot=-1 prefix_len=0 and pays a full prefill.

Fix

The state capture already exists and is exercised by the spec-decode rollback path. deepseek4_snapshot_save / _restore (deepseek4_graph.cpp:6862) captures raw and compressed KV, indexer compressed KV, attention and indexer compressor states, the HC residual and cur_pos. The patch therefore:

1. Adds a deepseek marker family before the laguna fallback, using the template's own tokens (<|begin_of_sentence|>, <|end_of_sentence|>, <|User|>, <|Assistant|> in their full-width DeepSeek forms), each required to encode to a single token so other vocabularies are unaffected.
2. Implements snapshot_save through the existing helper.
3. Extends do_prefill with snap_slot and snap_pos, splitting the chunk walk at the snapshot position and saving there, because the compressor and HC state are sequential and cannot be truncated after the fact.
4. Implements restore_and_generate_impl as restore plus suffix-only prefill at base position cur_pos, with a fresh-prefill fallback for an invalid slot, a failed restore, or an empty suffix.

About 110 lines, tagged cache-fix-20260729 in the patch below. The marker hunk is the part most worth a maintainer's eye:

+    // cache-fix-20260729: DeepSeek family. The renderer emits
+    // <|begin_of_sentence|>{system}<|User|>...<|Assistant|>...<|end_of_sentence|>
+    // (see chat_template.cpp DEEPSEEK4). Every marker must encode to a single
+    // special token; otherwise fall through so behaviour for other
+    // vocabularies is unchanged.
+    auto ds_bos  = tok.encode("<|begin_of_sentence|>");
+    auto ds_eos  = tok.encode("<|end_of_sentence|>");
+    auto ds_user = tok.encode("<|User|>");
+    auto ds_asst = tok.encode("<|Assistant|>");
+    if (ds_bos.size() == 1 && ds_eos.size() == 1 &&
+        ds_user.size() == 1 && ds_asst.size() == 1) {
+        out.family = "deepseek";
+        out.sys_role_prefix = {ds_bos[0]};
+        out.end_msg_seqs = {{ds_eos[0]}};
+        out.next_role_starts = {{ds_user[0]}, {ds_asst[0]}};
+        return true;
+    }

(The literals in the patch file are the full-width DeepSeek marker characters, not the ASCII forms shown here.)

Validation

Same three-turn probe on the patched binary (36c87b30eec95be5da79bb75fc4993f7d6d78c2a09e9eadbdab26a5eb84a26e7):

[pc] enabled: cap=32 family=deepseek
[snap] inline slot=0 cur_pos=7023
[pc] inline snapshot requested=7023 saved=7023 slot=0
[pc] inline-snap committed slot=0 prefix_len=7023
[pc] lookup hit slot=0 prefix_len=7023 (of 7067 total)
[deepseek4] restored slot=0 cur_pos=7023, prefilling 44 suffix tokens

Turn	Prompt tokens	Prefill before	Prefill after	Answer
0	7,016	377.0 s	378.2 s	identical
1	7,051	365.0 s	365.8 s	identical
2	7,067	365.1 s	2.5 s	identical

All three answers are byte-identical to the unpatched baseline under greedy decoding, so the restored state is the same state and not merely a faster one. The one-turn lag is inherent: a boundary can only be snapshotted on the turn after its content is generated. Snapshotting at the end of generation as well would remove the turn-one cost.

Defect 3: no deepseek4 model-card family, so reasoning silently disables

Symptom and root cause

resolve_model_card normalises general.name to a filename stem, so DeepSeek V4 Flash Src becomes deepseek-v4-flash-src. No such card ships in share/model_cards/, and family_fallback (server/src/server/model_card.cpp:257-289) has branches for qwen35, qwen36, qwen3, gemma4 and laguna, but none for deepseek4, even though deepseek4 is a first-class architecture elsewhere in the tree. Every DeepSeek V4 model without a sidecar therefore drops to the hard fallback (model_card.cpp:344) with max_tokens=16000 and hard_limit_reply_budget=4096.

The consequence is quiet and severe. With --default-max-tokens 2048, the derived thinking budget becomes max(0, 2048 - 4096) = 0, so the server answers with reasoning entirely disabled while reporting success. Our first quality run was invalid for this reason and we did not know until we read the resolution log. A single stderr warning when the hard fallback engages would have saved the run.

Two further notes for anyone reproducing this. The card only resolves when the server's working directory or binary location puts share/model_cards on the search path, and --model-name does not redirect card lookup: it only sets the /v1/models label. Separately, this server engages reasoning only when the request carries reasoning_effort. The chat_template_kwargs route does not.

Reproduction

Start the server on any DeepSeek V4 Flash GGUF with no matching sidecar and --default-max-tokens 2048, then send a request with "reasoning_effort": "high". The reply arrives with no thinking phase, and startup logs the hard fallback rather than a card path.

Suggested fix

Add a deepseek4 branch to family_fallback with a smaller reply budget matching the terse DeepSeek style, warn on stderr when the hard fallback engages, and tighten the laguna branch in resolve_chat_markers so it stops claiming unknown tokenizers (same root cause as defect 2).

A working sidecar we used for the runs in this report, sha256 dc4624a72fac9d87f022f16704cbce97157ce98295cfe9dc2901d35ffba64497, validates against share/model_cards/_schema.json. With it, startup resolves the card and clamps the high tier to max_ctx - hard_limit_reply_budget as expected:

share/model_cards/deepseek-v4-flash-src.json

{
  "name": "DeepSeek V4 Flash Src",
  "source": "https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash",
  "verified_at": "2026-07-29",
  "max_tokens": 32768,
  "complex_problem_max_tokens": 32768,
  "hard_limit_reply_budget": 1024,
  "sampling": {
    "temperature": 1.0,
    "top_p": 1.0,
    "top_k": 0,
    "min_p": 0.0,
    "presence_penalty": 0.0,
    "repetition_penalty": 1.0
  },
  "reasoning_effort_tiers": {
    "low": 4096,
    "medium": 8192,
    "high": 16384,
    "x-high": 24576,
    "max": 32256
  },
  "notes": "Authored locally on 2026-07-29 for the Lucebox ROCMFP2 quality rerun; not vendor-published. Sampling follows DeepSeek's official local recommendation (temperature 1.0, top_p 1.0). hard_limit_reply_budget 1024 reflects the terse DeepSeek V4 Flash reply style noted in the Lucebox source comments (originally 512 in ds4_eval). DeepSeek V4 Flash defaults to thinking mode; the max tier is nominal because true Think Max needs at least 384K context. Effort tiers are locally derived, not vendor figures."
}

Defect 4: DSpark output is not greedy-identical to target-only decoding

Speculative decoding with correct rejection sampling should preserve the target distribution exactly. On this hardware it does not.

Four probes were run at temperature 0 in three configurations: target-only, DSpark through the published fused-verify path, and DSpark through the patched non-fused path. All eight identity comparisons against target-only fail, with divergence appearing early in the output rather than at the tail.

Probe	Target sha256 (prefix)	Fused verify	Patched non-fused
warmup	3d6b876b	differs	differs
speed (2048 tokens)	19240055	differs	differs
maths	f3a54684	differs	differs
code	684f2671	differs	differs

The most plausible explanation, suggested during review, is that batched verification changes floating-point reduction order, so a per-row argmax flips whenever two candidates are close. That would make the divergence benign in distribution while still breaking any claim of identical output. It cannot be confirmed from source alone.

The practical implication is that DSpark should not be presented as a quality-free speedup until an acceptance test with an explicit numerical tolerance exists. We suggest comparing each verify row against a trusted single-token forward pass for widths 1 to 4 and requiring identical argmax wherever the winning margin exceeds the tolerance.

We should also report that DSpark was slower than target-only decoding in every configuration we measured:

Probe	Target-only	Fused verify	Patched non-fused
warmup	23.9 t/s	12.3	11.2
speed (2048 tokens)	23.0 t/s	14.6	10.7
maths	23.0 t/s	23.1	15.3
code	22.8 t/s	20.5	13.1

The observation below probably explains part of that.

Related observation: the drafter can run with an empty feature window

The debug log shows ctx_len=0 entering the drafter after a completed prefill. --ds4-prefill exact forces single-token prefill steps, and with --ds4-fused-decode each step takes the fused fast path (deepseek4_graph.cpp:6042), whose signature has no hooks parameter, so the per-layer feature capture at lines 6577-6588 never runs and spec_feat_window_ stays empty. A comment in verify_batch (deepseek4_dspark_spec.cpp:104-105) shows the author knew reused graphs skip hooks and forced single-token verifies onto the dynamic path for that reason, but prefill did not get the same treatment.

The drafter tolerates the empty window and still proposes tokens, so the target stays correct and only acceptance suffers. We have not patched this. Two routes look reasonable: pass allow_decode_graph_reuse=false from prefill when capture hooks are set, or have the fused decode step decline when hooks are present.

This may matter for the published configuration. The layer-major multi-token prefill path also sits above the dynamic hook handling and has no hooks parameter, so sparse prefill may starve the drafter in the same way. If so, published acceptance rates and the 32 tokens per second headline may have headroom.

Quality numbers for this quant

We could not find a published broad quality evaluation for the ROCMFP2 quant, so these may be useful. EvalPlus HumanEval+ (164 tasks) run through the repository's own server/scripts/quality_humaneval_plus.py for canonical prompts, completion extraction and sandboxed grading, driving an already-running server rather than spawning one. Temperature pinned to 0.

Configuration	pass@1
Non-thinking, greedy	87.2% (143/164)
Thinking, reasoning_effort high, 20,480 token budget	95.1% (156/164)

Method for the thinking figure: non-thinking passes plus a thinking rerun of only the non-thinking failures.

Three notes for anyone repeating this:

• A run at a 4,096 token budget truncated seven tasks, because the whole generation shares the budget with the roughly 16,000 token high-effort thinking tier. Budget at least 20,480 for high effort.
• Four non-thinking failures were smart-quote characters breaking Python syntax. They vanish under thinking, so they look like a sampling artefact rather than a decode-level fault.
• Sustained evaluation load at high GPU clocks tripped a 95 degree abort about 18 minutes in. At auto clocks the same load peaks near 88 degrees and holds.

For context, this 2.88 bit-per-parameter quant reaching 95.1% with reasoning matches the 153 GiB Q4 two-node configuration we measured on the same fixture, which also reached 95.1%.

Other measurements

Non-thinking, greedy, needle retrieval at depth.

Occupied context	Prefill	Decode
8K	about 5 minutes	24.7 t/s
16K	about 13 minutes	19.1 t/s

Prefill runs at roughly 18 to 22 tokens per second in both exact and dense modes. dense chunking works, but attention compute dominates, and with --ds4-prefill dense the chunk size resolves to 1 anyway (deepseek4_backend.cpp:767-770), so both modes take single-token steps. A 28K request returned HTTP 400. Separately, exact-literal recall degrades with occupied context: a planted KESTREL-77 came back as KESTREL-7-7 on all four attempts, and we have not established whether that is specific to this quant.

Patch

Both patched defects in one diff against c3b71e4b2ed2d08db96d55e7985f20035aa08d6f. Defect 1 is the deepseek4_graph.cpp hunk; defect 2 is the rest.

cache-fix-20260729.patch

diff --git a/server/src/deepseek4/deepseek4_backend.cpp b/server/src/deepseek4/deepseek4_backend.cpp
index 90e8284..f195bd1 100644
--- a/server/src/deepseek4/deepseek4_backend.cpp
+++ b/server/src/deepseek4/deepseek4_backend.cpp
@@ -750,7 +750,7 @@ bool DeepSeek4Backend::unpark(ParkTarget target) {
 
 int DeepSeek4Backend::do_prefill(const std::vector<int32_t> & tokens,
                                   const DaemonIO & io,
-                                  int kv_offset) {
+                                  int kv_offset, int snap_slot, int snap_pos) {
     // The all-hot layer-range path supports causal chunked prefill. The
     // optimized graph snapshots the previous raw SWA window, attends over
     // that snapshot plus the current ubatch, and commits only the final SWA
@@ -808,10 +808,17 @@ int DeepSeek4Backend::do_prefill(const std::vector<int32_t> & tokens,
     DeepSeek4StepTelemetry tel_acc;
     int steps = 0;
 
-    for (int i = 0; i < n_total; i += chunk) {
+    for (int i = 0; i < n_total; ) {
         if (io.cancelled) return pos;
 
-        const int n_tok = std::min(chunk, n_total - i);
+        int n_tok = std::min(chunk, n_total - i);
+        // cache-fix-20260729: split the chunk at the requested snapshot
+        // position so the cache is consistent exactly there. Any chunk
+        // boundary is a safe split: deepseek4_step_layer_range already
+        // splits internally at learned-compressor boundaries.
+        if (snap_slot >= 0 && snap_pos > pos && snap_pos < pos + n_tok) {
+            n_tok = snap_pos - pos;
+        }
 
         // Embed tokens
         std::vector<float> embed(w_.n_embd * n_tok);
@@ -863,6 +870,16 @@ int DeepSeek4Backend::do_prefill(const std::vector<int32_t> & tokens,
         }
         last_logits_ = std::move(logits);
         pos += n_tok;
+        i += n_tok;
+        // cache-fix-20260729: inline snapshot at the requested boundary.
+        // cache_.cur_pos == pos here (the step just committed to pos).
+        if (snap_slot >= 0 && pos == snap_pos) {
+            if (snapshot_save(snap_slot)) {
+                std::printf("[snap] inline slot=%d cur_pos=%d\n",
+                            snap_slot, snap_pos);
+                std::fflush(stdout);
+            }
+        }
     }
     if (timing) {
         log_step_tel("prefill", n_total, steps, elapsed_s(phase_t0), tel_acc);
@@ -969,6 +986,16 @@ bool DeepSeek4Backend::do_decode(int committed, int n_gen,
 
 GenerateResult DeepSeek4Backend::generate_impl(const GenerateRequest & req,
                                                 const DaemonIO & io) {
+    return generate_from_state(req, io, /*kv_offset=*/0);
+}
+
+// cache-fix-20260729: shared prefill+decode tail behind generate_impl and
+// restore_and_generate_impl. kv_offset > 0 resumes on a just-restored cache:
+// only prompt[kv_offset..] is prefilled, at that base position. Decode and
+// the sampler's penalty history still see the full prompt, and `committed`
+// stays an absolute position either way.
+GenerateResult DeepSeek4Backend::generate_from_state(
+        const GenerateRequest & req, const DaemonIO & io, int kv_offset) {
     GenerateResult result;
     DaemonIO out_io = io.with_token_callback(req.on_token);
     auto t0 = Clock::now();
@@ -978,7 +1005,16 @@ GenerateResult DeepSeek4Backend::generate_impl(const GenerateRequest & req,
     }
 
     // Prefill
-    int committed = do_prefill(req.prompt, out_io);
+    int committed;
+    if (kv_offset > 0) {
+        const std::vector<int32_t> delta(req.prompt.begin() + kv_offset,
+                                         req.prompt.end());
+        committed = do_prefill(delta, out_io, kv_offset,
+                               req.snap_slot, req.snap_pos);
+    } else {
+        committed = do_prefill(req.prompt, out_io, /*kv_offset=*/0,
+                               req.snap_slot, req.snap_pos);
+    }
     if (committed < 0) {
         result.fail(GenerateErrorCode::PrefillFailed);
         return result;
@@ -1071,8 +1107,12 @@ GenerateResult DeepSeek4Backend::generate_impl(const GenerateRequest & req,
 
 bool DeepSeek4Backend::snapshot_save(int slot) {
     if (slot < 0 || slot >= PREFIX_SLOTS) return false;
-    // TODO: Implement snapshot save (copy KV cache + HC state to CPU)
-    return false;
+    // cache-fix-20260729: CPU-resident copy of the full recurrent state —
+    // raw + compressed KV, attention/indexer compressor states, HC residual
+    // and cur_pos. deepseek4_snapshot_save frees any previous snapshot in
+    // the slot before writing.
+    if (!snap_backend_) return false;
+    return deepseek4_snapshot_save(cache_, snap_backend_, snapshots_[slot]);
 }
 
 void DeepSeek4Backend::snapshot_free(int slot) {
@@ -1092,8 +1132,37 @@ int DeepSeek4Backend::snapshot_cur_pos(int slot) const {
 
 GenerateResult DeepSeek4Backend::restore_and_generate_impl(
         int slot, const GenerateRequest & req, const DaemonIO & io) {
-    // TODO: Implement snapshot restore + generate
-    (void)slot;
+    // cache-fix-20260729: restore the CPU snapshot into the live cache and
+    // prefill only the prompt suffix past the snapshot position. Anything
+    // that cannot be restored exactly falls back to a fresh full prefill,
+    // which matches the pre-fix behaviour.
+    if (slot >= 0 && slot < PREFIX_SLOTS && snapshots_[slot].ctx) {
+        const int snap_pos = snapshots_[slot].cur_pos;
+        // An empty suffix cannot re-seed decode logits: the snapshot stores
+        // no logits, and re-stepping the last token would advance the
+        // compressor state a second time. Exact-length hits therefore also
+        // take the fresh-prefill path.
+        if (snap_pos > 0 && snap_pos < (int)req.prompt.size()) {
+            if (deepseek4_snapshot_restore(snapshots_[slot], cache_)) {
+                // The host feature window belongs to the previous request,
+                // not to the restored state.
+                spec_feat_window_.clear();
+                std::fprintf(stderr,
+                    "[deepseek4] restored slot=%d cur_pos=%d, prefilling "
+                    "%zu suffix tokens\n",
+                    slot, snap_pos, req.prompt.size() - (size_t)snap_pos);
+                return generate_from_state(req, io, snap_pos);
+            }
+            std::fprintf(stderr,
+                "[deepseek4] snapshot restore failed slot=%d — full prefill\n",
+                slot);
+        } else {
+            std::fprintf(stderr,
+                "[deepseek4] snapshot slot=%d pos=%d unusable for prompt=%zu "
+                "— full prefill\n",
+                slot, snap_pos, req.prompt.size());
+        }
+    }
     return generate_impl(req, io);
 }
 
diff --git a/server/src/deepseek4/deepseek4_backend.h b/server/src/deepseek4/deepseek4_backend.h
index 1cb3b53..90f247c 100644
--- a/server/src/deepseek4/deepseek4_backend.h
+++ b/server/src/deepseek4/deepseek4_backend.h
@@ -88,8 +88,17 @@ private:
     void release_spec_drafter(bool mark_parked);
 
     // Prefill prompt tokens in chunks, return absolute committed position.
+    // cache-fix-20260729: a non-negative snap_slot requests an inline
+    // snapshot: the chunk walk splits at absolute position snap_pos and
+    // saves the cache state there.
     int do_prefill(const std::vector<int32_t> & tokens, const DaemonIO & io,
-                   int kv_offset = 0);
+                   int kv_offset = 0, int snap_slot = -1, int snap_pos = -1);
+
+    // cache-fix-20260729: shared prefill+decode tail. kv_offset > 0 resumes
+    // on a just-restored cache: only prompt[kv_offset..] is prefilled, at
+    // that base position.
+    GenerateResult generate_from_state(const GenerateRequest & req,
+                                       const DaemonIO & io, int kv_offset);
 
     // Autoregressive decode loop.
     bool do_decode(int committed, int n_gen,
diff --git a/server/src/deepseek4/deepseek4_graph.cpp b/server/src/deepseek4/deepseek4_graph.cpp
index b14af44..cd9d7df 100644
--- a/server/src/deepseek4/deepseek4_graph.cpp
+++ b/server/src/deepseek4/deepseek4_graph.cpp
@@ -6640,7 +6640,12 @@ bool deepseek4_step_layer_range(
             ggml_context * ctx = ggml_init(params);
             if (!ctx) return false;
 
-            const bool last_only = n_tokens > 1;
+            // dspark-fix-20260729: the non-fused verify path must project
+            // logits for every batch row when the verify hook requests them;
+            // a single-row projection under-allocates the tensor the hook
+            // reads n_tokens rows from (out-of-bounds read, exit on assert).
+            const bool need_all_logits = verify_hooks && verify_hooks->all_logits_out;
+            const bool last_only = n_tokens > 1 && !need_all_logits;
             const int output_tokens = last_only ? 1 : n_tokens;
             ggml_tensor * inp = ggml_new_tensor_2d(
                 ctx, GGML_TYPE_F32, n_embd, output_tokens);
diff --git a/server/src/server/prefix_cache.cpp b/server/src/server/prefix_cache.cpp
index 4ab20c5..28dbeb2 100644
--- a/server/src/server/prefix_cache.cpp
+++ b/server/src/server/prefix_cache.cpp
@@ -37,6 +37,24 @@ bool resolve_chat_markers(const Tokenizer & tok, ChatMarkers & out) {
         return true;
     }
 
+    // cache-fix-20260729: DeepSeek family. The renderer emits
+    // <｜begin▁of▁sentence｜>{system}<｜User｜>...<｜Assistant｜>...
+    // <｜end▁of▁sentence｜> (see chat_template.cpp DEEPSEEK4). Every marker
+    // must encode to a single special token; otherwise fall through so
+    // behaviour for other vocabularies is unchanged.
+    auto ds_bos  = tok.encode("<｜begin▁of▁sentence｜>");
+    auto ds_eos  = tok.encode("<｜end▁of▁sentence｜>");
+    auto ds_user = tok.encode("<｜User｜>");
+    auto ds_asst = tok.encode("<｜Assistant｜>");
+    if (ds_bos.size() == 1 && ds_eos.size() == 1 &&
+        ds_user.size() == 1 && ds_asst.size() == 1) {
+        out.family = "deepseek";
+        out.sys_role_prefix = {ds_bos[0]};
+        out.end_msg_seqs = {{ds_eos[0]}};
+        out.next_role_starts = {{ds_user[0]}, {ds_asst[0]}};
+        return true;
+    }
+
     // Try Laguna family: XML-style markers.
     auto start_sys = tok.encode("<system>");
     auto end_sys   = tok.encode("</system>");

[davide221]

@GRAVESIS thanks for reporting them, feel free to open a PR with the patches, we are working on some of them