Gemma‑4 MTP Support on MLX – Not Yet Available

by /u/purealgo · Llama

Use a framework that supports Gemma‑4 MTP, such as llama.cpp or vLLM, instead of MLX.

What to do now

Use a framework that supports Gemma‑4 MTP, such as llama.cpp or vLLM, instead of MLX.

Summary

Google recently released Multi‑Token Prediction (MTP) drafters for Gemma‑4, pairing the main model with a lightweight drafter to predict several tokens ahead and verify them in parallel, achieving 2‑3× faster inference. A user attempted to run Gemma‑4 MTP on the MLX framework but was unsuccessful, indicating that MLX currently does not support Gemma‑4 MTP. The article is a question rather than a release, and no workaround or alternative is provided. The discussion highlights that while Gemma‑4 MTP is available in other frameworks, MLX lacks support at this time.

Key changes

Google released MTP drafters for Gemma‑4, pairing the main model with a lightweight drafter.
MTP predicts several tokens ahead and verifies them in parallel, speeding inference 2‑3×.
A user attempted to run Gemma‑4 MTP on MLX but failed.
MLX currently does not support Gemma‑4 MTP.
No workaround or alternative is provided in the article.
The discussion highlights that Gemma‑4 MTP is available in other frameworks.
The article is a question rather than a release.
The article indicates that MLX lacks support for Gemma‑4 MTP at this time.

Affects

internal

Story evolution

Customer impact

Analyzing matches…

Ask about this story

Impact on an agency? Which customers? Compare historically Risks of waiting