Briefing

Benchmarking the new b9200 update optimizing Qwen

ai-dev
by /u/swizzcheezegoudaSWFA · Llama

Test the new b9200 build with optimized flags to improve token generation speed.

What to do now

Patch your llama-server to the b9200 build and apply the optimized launch command with single parallel slot and reduced lookahead to boost agent performance.

Summary

The author benchmarks the new b9200 release of llama‑cpp with the Qwen 3.6 27B‑IQ4_NL.gguf model on an RTX 3090 and Ryzen 7 5700G. The b9200 build fixes the memory‑traffic overhead that previously copied full logits for every token during prompt processing. Default MTP flags (--spec‑draft‑n‑max 6, --spec‑draft‑p‑min 0.75) caused poor performance for agentic loops, with prompt speeds around 560 t/s and token generation 17 t/s. By limiting to a single parallel slot, dropping the lookahead to 3, and removing the p‑min threshold, the optimized launch command boosts prompt processing to ~611 t/s and peak token generation to 27.44 t/s. Draft acceptance rates rise from 22‑26 % to ~70‑77 %. The new launch flags include --spec‑type draft‑mtp, --spec‑draft‑n‑max 3, --ctx‑size 65536, --parallel 1, --flash‑attn on, --cache‑type‑k q8_0, and --cache‑type‑v q8_0. The author notes that an undervolted RTX 3090 can still reach nearly 30 t/s on a dense 27B model when the VRAM bus is unclogged. The update is especially beneficial for strict, multi‑turn agentic workflows.

Key changes

  • b9200 build fixes mtp memory traffic overhead, removing full logits copy per token
  • Default MTP flags (--spec-draft-n-max 6, --spec-draft-p-min 0.75) cause poor performance for agentic loops
  • Optimized launch command limits to single parallel slot, reduces lookahead to 3, removes p-min threshold
  • Prompt processing increased from ~560 t/s to ~611 t/s
  • Token generation peak increased from 17.06 t/s to 27.44 t/s
  • Draft acceptance rate improved from 22‑26 % to ~70‑77 %
  • Using --parallel 1 unclogs VRAM bus, enabling ~30 t/s on 27B model
  • Launch command includes --spec-type draft-mtp, --spec-draft-n-max 3, --ctx-size 65536, --flash-attn on, --cache-type-k q8_0, --cache-type-v q8_0

Affects

internal

Source angles · 2 perspectives

Reddit r/LocalLLaMA
Independent angle

Benchmarking the new b9200 update: Optimizing Qwen 3.6 27B mtp for Hermes Agent on a single RTX 3090

Open
Reddit r/LocalLLaMA
Independent angle

Qwen 3.6 27B on 24GB VRAM setup: backend comparisons, quant choice and settings (llama.cpp, ik_llama.cpp, BeeLlama, vllm)

Open

Customer impact

Analyzing matches…

Ask about this story

Impact on an agency? Which customers? Compare historically Risks of waiting