Gemma 4 QAT Balanced Releases with MTP – 20M Downloads
Patch your inference pipelines to use the new Gemma 4 Balanced QAT models with MTP for 35‑53% speedup, ensuring you use Q4_K_M quantization and mmproj for vision.
Migrate your inference scripts to the new Gemma 4 Balanced QAT models, enable MTP, and adjust sampling parameters as recommended.
Summary
On the heels of a milestone, the author reports almost 20 million downloads on Hugging Face and nearly 5 000 Discord members for the new Gemma 4 QAT releases. Two balanced variants, Gemma 4 26B‑A4B and Gemma 4 31B, are now available in QAT form with MTP draft head for speculative decoding. The balanced models include a light reasoning preamble, no personality changes, and are uncensored, making them ideal for creative writing, role‑play, and emotional intelligence tasks. Quantization uses the Q4_K_M format, yielding 16.8 GB for the 26B model and 18.7 GB for the 31B, while the mmproj vision module is bundled in both.
MTP delivers a 35 % speedup on the 26B and 53 % on the 31B with identical output quality, and sampling defaults are set to temp = 0.6, top_k = 64, top_p = 0.9, min_p = 0.05, repeat_penalty = 1.1. The author also notes that the models show no looping, stable sampling, and long‑context coherence across re‑runs. For users, the recommended approach is to use the Q4_K_M quantization, enable MTP, and apply the provided sampling parameters to achieve the best trade‑off between speed and quality.
Key changes
- Gemma 4 26B‑A4B and 31B QAT releases now available
- Balanced variants include light reasoning preamble and no personality changes
- MTP draft head added, providing 35 % speedup on 26B and 53 % on 31B
- Quantization uses Q4_K_M, 16.8 GB (26B) and 18.7 GB (31B)
- mmproj vision module bundled in both models
- Sampling defaults: temp = 0.6, top_k = 64, top_p = 0.9, min_p = 0.05, repeat_penalty = 1.1
- Models exhibit no looping, stable sampling, and long‑context coherence