Qwen3.6‑35B‑A3B‑MTP Model Released with Modest Speed Gains
Download the Qwen3.6‑35B‑A3B‑MTP‑GGUF model from HuggingFace, run benchmarks on your GPU, and compare the modest 6 % (Q4) and 2.5 % (Q8) speed gains to the 27B variant before deciding on deployment.
Download the Qwen3.6‑35B‑A3B‑MTP‑GGUF model from HuggingFace, benchmark it on your GPU, and compare the modest speed gains to the 27B variant before deciding on deployment.
Summary
A new Qwen3.6‑35B‑A3B‑MTP‑GGUF model has been uploaded to HuggingFace, featuring isolated MTP layers and a convert.py script. Benchmarks show only a 6 % speed increase for Q4 quant and a 2.5 % increase for Q8 quant compared to the dense 35B model, a stark contrast to the 2‑2.5× gains seen on the 27B variant. Performance varies by hardware: on two 5070 Ti + 3090 GPUs, Q8 throughput rose from 110 t/s to 165 t/s; on a 5090 FE, Q4 throughput increased from 215 t/s to 228.83 t/s, while Q8 went from 148.20 t/s to 152.02 t/s. Draft acceptance rates were 0.61434 for Q4 and 0.54754 for Q8. The results are preliminary and may change as the MTP implementation matures or as the model architecture is refined.
The article includes detailed prompt‑based timing data, showing prompt evaluation times, token‑level throughput, and total times for both Q4 and Q8 on the 5090 FE + 3090 setup. It also notes that the MTP implementation may be limited by the Qwen3.6‑35B architecture or by the current llama.cpp MTP support.
Overall, the 35B variant offers only marginal speed improvements, suggesting that deployment decisions should weigh the modest gains against the additional complexity of MTP.
Key changes
- New Qwen3.6‑35B‑A3B‑MTP‑GGUF model available on HuggingFace
- MTP layers isolated with convert.py included
- Q4 speed increase only 6 %, Q8 2.5 % versus 2‑2.5× on 27B
- Hardware‑specific gains: 5070 Ti + 3090 Q8 from 110 t/s to 165 t/s; 5090 FE Q4 from 215 t/s to 228.83 t/s; Q8 from 148.20 t/s to 152.02 t/s
- Draft acceptance rates: 0.61434 (Q4) and 0.54754 (Q8)
- Results preliminary; may change with future MTP or model updates
- MTP implementation may be limited by Qwen3.6‑35B architecture or llama.cpp support
- Benchmark data shows prompt eval times, token throughput, and total times for both quant levels