MTP vs Non‑MTP Performance on Qwen3.6 27B and Gemma 4 31B
Run your own benchmarks to decide whether to use MTP for your workloads; consider disabling MTP if quality suffers.
Run your own benchmarks to decide whether to use MTP for your workloads; consider disabling MTP if quality suffers.
Summary
A developer benchmarks the Qwen 3.6 27B model with and without MTP on a 4× RTX 5070 Ti setup, noting that non‑MTP produces higher quality code‑review outputs and fewer tokens. The non‑MTP version achieves roughly 2000 pp/s and 50–60 tg/s, while the MTP version reaches about 1300 pp/s and 100–120 tg/s, indicating a real‑world speed advantage of only ~20 %. MTP consumes more VRAM and context, leading to a higher token count (~60 k vs ~40 k) and a slight degradation in output quality for agent tasks. The same pattern is observed with the Gemma 4 31B model, where MTP also shows reduced quality and higher context usage.
The author details the llama.cpp command‑line flags used for both models, including spec-type draft-mtp and spec-draft-n-max 2 for MTP. They also note that the MTP draft head is provided by the Unsloth team and that disabling thinking can be done via chat-template kwargs. The discussion concludes that while MTP offers theoretical speedups, the practical benefits may be limited for certain workloads.
Key changes
- Non‑MTP Qwen 3.6 27B yields higher quality code‑review outputs
- Non‑MTP throughput ~2000 pp/s and 50–60 tg/s; MTP ~1300 pp/s and 100–120 tg/s
- MTP consumes more VRAM and context, resulting in ~60 k tokens vs ~40 k for non‑MTP
- MTP degrades output quality for agent tasks
- Similar trend observed with Gemma 4 31B model
- llama.cpp flags for MTP: --spec-type draft-mtp and --spec-draft-n-max 2
- MTP draft head provided by Unsloth team
- Disabling thinking via chat-template kwargs reduces MTP overhead