Support standalone Qwen3.5 MTP companion GGUFs (-md ... --spec-type mtp) - #43
Merged
Thireus merged 1 commit intoAug 16, 2026
Conversation
A predictor-only MTP companion for qwen35/qwen35moe declares the full
block count (n_main + nextn_predict_layers) but ships only the NextN
block. Loading one with -md aborted at
check_tensor_dims: tensor 'blk.0.attn_norm.weight' not found
because create_qwen35{,moe}_tensors() creates every main block as
hard-required. Step-3.5 and DeepSeek-V4 already have an "mtp_only"
path for this; Qwen3.5 did not, and the surrounding MTP plumbing did
not recognise a Qwen3.5 companion either.
- llama-load-tensors: detect the companion (nextn > 0 and blk.0
attn_norm absent) and mark the absent main blocks
TENSOR_SKIP|TENSOR_NOT_REQUIRED, mirroring create_step35_tensors().
Guard the "9B shares q_proj" fixup, which has no trunk to borrow
from, and skip empty blocks in the split-mode-graph loop, where
split_recurrent_tensors() would assert on the missing ssm_in/wqkv.
- llama-model: add llama_model_is_qwen35_family() and let Qwen3.5 use
the common MTP package contract, so a companion classifies as
COMPANION and a plain target as TARGET_ONLY (which is what makes the
target export the hidden states the companion consumes). Also accept
enorm/hnorm as a predictor tail, since 9B/4B NextN blocks can be
built without eh_proj.
- llama-hparams: map qwen35moe n_layer 49 and 61 to 122B-A10B and
397B-A17B, so a companion of those sizes no longer prints "?B".
- speculative: recognise the Qwen3.5 companion, validate its arch and
head count, keep the embedded-tail + ordinary-draft combination
working, and force pooling_type = NONE on the companion context.
- common: reject a companion passed as -m with an actionable message
instead of aborting inside the graph builder.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A predictor-only MTP companion for qwen35/qwen35moe declares the full block count (n_main + nextn_predict_layers) but ships only the NextN block. Loading one with -md aborted at
check_tensor_dims: tensor 'blk.0.attn_norm.weight' not found
because create_qwen35{,moe}_tensors() creates every main block as hard-required. Step-3.5 and DeepSeek-V4 already have an "mtp_only" path for this; Qwen3.5 did not, and the surrounding MTP plumbing did not recognise a Qwen3.5 companion either.
llama-load-tensors: detect the companion (nextn > 0 and blk.0 attn_norm absent) and mark the absent main blocks TENSOR_SKIP|TENSOR_NOT_REQUIRED, mirroring create_step35_tensors(). Guard the "9B shares q_proj" fixup, which has no trunk to borrow from, and skip empty blocks in the split-mode-graph loop, where split_recurrent_tensors() would assert on the missing ssm_in/wqkv.
llama-model: add llama_model_is_qwen35_family() and let Qwen3.5 use the common MTP package contract, so a companion classifies as COMPANION and a plain target as TARGET_ONLY (which is what makes the target export the hidden states the companion consumes). Also accept enorm/hnorm as a predictor tail, since 9B/4B NextN blocks can be built without eh_proj.
llama-hparams: map qwen35moe n_layer 49 and 61 to 122B-A10B and 397B-A17B, so a companion of those sizes no longer prints "?B".
speculative: recognise the Qwen3.5 companion, validate its arch and head count, keep the embedded-tail + ordinary-draft combination working, and force pooling_type = NONE on the companion context.
common: reject a companion passed as -m with an actionable message instead of aborting inside the graph builder.
I have read the contributing guidelines
Self-reported review complexity: