fix(pflash): reuse Qwen3 drafter K/V buffers

[cheese-cakee]

Summary

• reuse the current layer's full-sequence RoPE K and V buffers on the default NoPE scoring path
• preserve per-layer RoPE K and Q-tail state when DFLASH_FP_NOPE_TAIL=0
• guard empty-layer models and add a pure allocation-policy regression test

Root cause

The Qwen3 drafter allocated full-sequence K and V tensors for every layer even though the default NoPE path consumes the RoPE K/V pair before advancing to the next layer. At the contexts reported in #147, one BF16 K or V tensor is approximately 350 to 390 MiB, so repeated per-layer reservations exhaust a 24 GB device and force compression to fall back to extraction.

The default path now retains one reusable RoPE K buffer and one reusable V buffer. The legacy post-RoPE scoring path still retains the per-layer K and Q-tail tensors that it reads after the forward loop; V remains reusable in both modes.

Fixes #147.

Scope

This PR contains only the Qwen3 buffer-lifetime fix and its allocation-policy test. It does not include the separately preserved FlashPrefill phase-one kernel hardening or any hunks overlapping draft PR #542.

Verification

• isolated change built and ran on RTX 3090, CUDA 12.0, with BSA enabled
• isolated 180K and 200K NIAH compression runs completed without allocation failure, empty output, or excerpt fallback
• both long-context runs retained the key and answer and produced the exact expected answer 0438574
• repeated 180K and 200K compressed outputs were byte-identical to the fully qualified candidate
• default and DFLASH_FP_NOPE_TAIL=0 short-context outputs matched the current-main baseline byte-for-byte
• the allocation-policy unit test covers default reuse, legacy per-layer state, empty models, and the reported allocation sizes
• strict standalone compilation passed with -Wall -Wextra -Werror
• git diff --check passed
• fresh independent review found no actionable code issue

Limitations

• timings collected during qualification are diagnostic only and support no performance claim
• other quadratic FlashPrefill scratch allocations are unchanged and remain possible follow-up work
• final repository CI and maintainer hardware validation remain authoritative

[cubic-dev-ai]

1 issue found across 4 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/qwen3/qwen3_graph.cpp">

<violation number="1" location="server/src/qwen3/qwen3_graph.cpp:421">
P3: The `nope_tail ? 0u : (size_t)il` buffer-index idiom is now computed three separate times in the same function — as `li` in the allocation loop (line ~326), `layer_cache_idx` in the forward loop (~421), and `layer_cache_idx` again in the scoring loop (~786). Because the indexing rules (index 0 when reusing in NoPE mode, per-layer index in legacy mode) live in three copies, a future change to the buffer plan or to the reuse policy must be applied consistently in all three or the code silently falls out of sync with `Qwen3DrafterBufferPlan`. Consider consolidating the index derivation into a small local helper (e.g. a lambda or a method on the plan struct) so the reuse policy is defined and read in one place.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

No issues found across 4 files

_{Re-trigger cubic}