vibevoice.cpp: Pure-C++ ggml Port of Microsoft VibeVoice for TTS and ASR
Patch: integrate vibevoice.cpp into LocalAI, test TTS/ASR pipelines on target hardware, and benchmark CUDA vs CPU performance.
Patch: integrate vibevoice.cpp into LocalAI, test TTS/ASR pipelines on target hardware, and benchmark CUDA vs CPU performance.
Summary
vibevoice.cpp is a pure‑C++ ggml port of Microsoft VibeVoice that ships a 0.5B realtime TTS model and a 7B‑parameter ASR model with speaker diarization. The library supports CPU, CUDA, Metal, Vulkan and hipBLAS backends and can be used as a single binary or a libvibevoice.so with a flat C ABI. Inference times are 28 s on CUDA Q4_K, 150 s on CPU Q4_K and 1929 s on CPU Q8_0, with peak RAM usage of 6 GB, 8 GB and 26 GB respectively for a 68 s sample. It ships pre‑converted GGUFs on HuggingFace, removes the need for Python, vLLM or torch at inference, and integrates directly into the LocalAI backend. Limitations include a 17‑minute audio peak that still consumes 26 GB on CPU, no streaming output, and an encoder activation pool that dominates memory. Future optimisations target graph reuse in the diffusion head and streaming support. The release demonstrates that a lightweight C++ implementation can match upstream performance while simplifying deployment.
Key changes
- Pure‑C++ ggml port of Microsoft VibeVoice
- 0.5B realtime TTS model with pre‑converted GGUFs on HuggingFace
- 7B‑parameter ASR model with speaker diarization and JSON segments
- Supports CPU, CUDA, Metal, Vulkan, hipBLAS backends
- Single binary or libvibevoice.so with flat C ABI
- Inference RTF: 28 s CUDA Q4_K, 150 s CPU Q4_K, 1929 s CPU Q8_0
- No Python, vLLM or torch required at inference
- Integration ready for LocalAI backend