DeepSeek V3.2 Support in llama.cpp Deepseek‑dsa Branch
Clone the deepseek‑dsa branch of llama.cpp and test the provided GGUFs for OOM issues.
Clone the deepseek‑dsa branch of llama.cpp and test the provided GGUFs for OOM issues.
Summary
The deepseek‑dsa branch of llama.cpp supports DeepSeek V3.2 GGUFs, including Q4_K_M (~404GB) and Q8_0 (~714GB) variants. The repository hosts three light GGUFs: DeepSeek‑V3.2‑light‑GGUF, DeepSeek‑V3.2‑Speciale‑light‑GGUF, and DeepSeek‑V3.2‑Exp‑light‑GGUF. The chat template to use is models/templates/deepseek‑ai‑DeepSeek‑V3.2.jinja. Users may encounter OOM errors in CUDA's ggml_top_k() and can mitigate them by lowering the ubatch size or increasing the -fitt value. The pull request link is https://github.com/ggml-org/llama.cpp/pull/21149. The article provides clone instructions and notes on memory management. No new features beyond the DeepSeek model support are announced. The post invites users to test the branch and report issues.
Key changes
- DeepSeek V3.2 GGUFs available in Q4_K_M (~404GB) and Q8_0 (~714GB).
- Branch deepseek‑dsa in llama.cpp.
- Chat template path models/templates/deepseek‑ai‑DeepSeek‑V3.2.jinja.
- OOM mitigation: lower ubatch size or increase -fitt.
- GGUF sizes: Q4_K_M ~404GB, Q8_0 ~714GB.
- Repo link to pull/21149.
- Supported GGUFs: DeepSeek‑V3.2‑light‑GGUF, DeepSeek‑V3.2‑Speciale‑light‑GGUF, DeepSeek‑V3.2‑Exp‑light‑GGUF.
- Provide clone instructions and memory management notes.