RTX 3090 Achieves 100 k Tokens/s with MTP‑Enabled Qwen 3.6 27B on llama.cpp
Run llama‑server with the provided command line to benchmark a 3090 at 50 t/s using 100 k context, MTP, and flash‑attn.
Execute the provided llama‑server command with the specified flags to benchmark your RTX 3090 at 50 t/s using 100 k context, MTP, and flash‑attn.
Summary
An enthusiast demonstrated that a single RTX 3090 can process 100 k tokens per second using llama.cpp with the MTP‑enabled Qwen 3.6 27B GGUF model. The benchmark was achieved by running llama‑server with a 100 k context size, 2 k batch size, 1 k unbatch size, and flash‑attn enabled, while limiting the speculative draft to 2 tokens. The command also uses 4‑bit KV cache (--cache-type‑k/v q4_0) and 99 GPU layers (--ngl 99). The author notes that a draft of 3 tokens was too demanding for the 3090 at high context, causing performance degradation. This setup shows that large‑context inference is feasible on consumer GPUs when combined with MTP and flash‑attn. The shared Hugging Face model is available at RDson/Qwen3.6‑27B‑MTP‑Q4_K_M‑GGUF. Users can replicate the results by following the provided command line and adjusting batch sizes for their hardware.
Key changes
- 3090 achieves 100 k tokens per second with MTP‑enabled Qwen 3.6 27B GGUF
- Command uses 100 k context, 2 k batch size, 1 k unbatch size, and flash‑attn
- Speculative draft limited to 2 tokens; draft 3 too heavy for 3090
- Uses 4‑bit KV cache (--cache-type‑k/v q4_0) and 99 GPU layers (--ngl 99)
- Hugging Face model at RDson/Qwen3.6‑27B‑MTP‑Q4_K_M‑GGUF
- Demonstrates large‑context inference feasible on consumer GPUs with MTP and flash‑attn