Recent llama.cpp updates for SYCL/Intel
Apply the SYCL PRs to your llama.cpp build to improve Intel GPU inference speed and kernel compatibility.
Apply the SYCL PRs to your llama.cpp build to improve Intel GPU inference speed and kernel compatibility.
Summary
The latest llama.cpp updates for SYCL/Intel bring several performance and stability improvements. Flash Attention now uses the XMX engine via oneDNN graph API (SDPA) on KV f16 for Xe2, boosting Qwen3.6‑27b‑Q8_0 prefill speed by 1.21× at p=512 and 4.26× at p=80k. The PRs also increase the minimum buffer size for USM system allocations, add support for the OP XIELU operator, and enable conv2d_dw kernels in fp16. Additionally, a bug fix addresses a large error in get_rows for Q2_K, Q4_K, and Q5_K. These changes improve inference speed on Intel GPUs and expand kernel compatibility. The updates are available on GitHub and can be integrated by rebuilding llama.cpp with SYCL support. The community is encouraged to test the new kernels on their hardware.
Key changes
- Flash Attention uses XMX engine via oneDNN graph API (SDPA) on KV f16 for Xe2
- Qwen3.6‑27b‑Q8_0 prefill speed up 1.21× at p=512 and 4.26× at p=80k
- Minimum buffer size for USM system allocations increased
- Support for OP XIELU operator added
- conv2d_dw kernel type fp16 supported
- Bug fix for get_rows error in Q2_K, Q4_K, Q5_K
- Improves inference speed on Intel GPUs
- Available on GitHub for SYCL builds