Skip to content

MTP: NextN layer가 있는 qwen35moe 파일에 draft가 없음 (Role::Unused) #30

Open

Description

@midagedev

NextN(MTP) layer를 가진 qwen35moe GGUF 파일(예: Qwen3.8-35B-A3B distill)은 그 layer를 읽기만 하고 쓰지 않습니다(Role::Unused). MTP draft가 없어서 decode가 MTP로 얻을 수 있는 속도를 그대로 놓칩니다.

  • 근거: crates/model/src/arch/qwen35moe/roles.rs:173-175(trunk 뒤 layer는 Role::Unused), crates/model/src/arch/qwen35moe/mtp.rs:70, 285(draft와 target 모두 qwen4exp만 받음), spec.rs:180(mtp: Vec::new()). MTP body는 Body38에만 있습니다(crates/app/src/arch/qwen3moe/mtp.rs:16).
  • 할 일: qwen35moe target과 draft를 받도록 mtp.rs를 넓히고, Body35에 MtpBody를 붙이고, qwen3 seat에 draft를 연결합니다. 공통 owner는 이미 있으니 모델 쪽 부분만 붙이면 됩니다.

메인테이너가 진행합니다. 관심 있으시면 댓글로 알려 주세요.

Checklist

Common first, then the model: the draft's mode and feed have one owner before Body35 attaches.

  • The model side reads the NextN layer (in_file, model: a qwen35moe file's next-token layer is described as an in-file draft (#30 M0+M1) #165).
  • Lift the draft's walk mode and feed into one owner in crates/gpu. Today each draft layer carries its own copy
    of the same three modes (MtpMode for Qwen3.8, NextnMode for GLM: Eager, Graph, Store) and its own feed type,
    and each MtpBody glue maps the shared WalkMode/Feed onto them one to one (crates/app/src/arch/qwen3moe/mtp.rs
    mtp_feed/mtp_mode, crates/app/src/arch/glm5next/mtp.rs nextn_feed/nextn_mode). After the lift, Mtp38, the
    GLM NextN layer and mtp35.rs take the shared types directly, and both glue files lose their mappings.
  • Body35's MtpBody glue on top of that: only what the architecture differs in (the head, the hidden width, the
    prompt calls), wired into the qwen3 seat. mtp35.rs (the draft layer on the card) is already on main.
  • M2b-1, mainline's MTP step taps as a reference set: the work is on branch q35m2b (closed PR refset, tools: mainline's Qwen3.6 MTP step taps as a reference set #242, head
    a4e71119). Port it onto main in the same round, and fix its open review item first: an existing destination with
    a trailing slash makes the dump installer delete both the old set and its replacement.
  • M2b-2: the card's draft step held to the M2b-1 set tap by tap in gate_qwen35moe_mtp.
  • First measurement: the acceptance rate on this model, then decode with the draft on against off.

Activity

  1. midagedev commented on Oct 12, 2026

    @midagedev
    OwnerAuthor

    다음 릴리스 decode 계획에 넣으면서 이득을 유도했습니다 [derived].

    • 3090에 통째로 올린 경우: 2행 창에서 +19…+46 %, 4행 창에서 +35…+110 %. 바운드는 카드 launch 고정비입니다.
    • 12 GB 하이브리드: 0.3.2가 이 파일의 로드를 고친 뒤에 해당하고, 바운드는 호스트입니다.
    • 이 모델의 수락률은 아직 잰 적이 없습니다. 첫 측정은 수락률입니다.
      MTP: attach MiMo-V2.6-Flash's sidecar MTP to the common head owner #74(MiMo의 sidecar MTP)와 같은 공통 head owner로 붙입니다.
  2. midagedev commented on Oct 20, 2026

    @midagedev
    OwnerAuthor

    Status (2026-10-20): the model side reads the NextN layer (in_file, PR #165). Left: an MtpBody for Body35, so qwen35moe files get a draft.

  3. midagedev commented on Oct 22, 2026

    @midagedev
    OwnerAuthor

    상태 (2026-10-22): 체크리스트 2번(draft의 walk mode와 feed를 한 owner로)은 #584, #625로 main에 들어갔습니다(crates/gpu/src/draft.rs). 남은 것은 Body35용 MtpBody glue와 qwen3 seat 연결입니다.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions