MTP Support Added to llama.cpp via PR #22673

An update to llama.cpp introduces Multi‑Threaded Pipeline (MTP) support via PR #22673, allowing users to enable MTP with the flag `--spec-type mtp --spec-draft-n-max 3`. The author rebuilt the radv container from the kyuz0/amd-strix-halo-toolboxes repository to run the new code on an AI Max 395 GPU with 128 GB DDR5‑8000 memory. Running the Qwen3.6‑35B3A3B‑MTP‑GGUF model from Hugging Face, token throughput rose from roughly 40 tokens/s without MTP to between 60 and 80 tokens/s, depending on the prompt. The pipeline throughput (PP) remained unchanged, and the two GGUF files used were about 36 GB each.

Performance gains were most noticeable on simple math prompts, while more complex subjects saw smaller improvements. The author noted that the GGUF size and PP stability suggest the change is primarily a scheduling optimization. Future tests on the larger Qwen 3.5 122B model are planned, with potential launch‑parameter tweaks. Overall, the MTP feature demonstrates a tangible speed boost for large‑model inference on AMD GPUs.