ds4.c: Native Inference Engine for DeepSeek V4 Flash
Build and run ds4.c to host DeepSeek V4 Flash locally on Mac with Metal for high‑performance inference.
Compile ds4.c on a Mac with 128GB RAM and run the server to test local inference performance.
Summary
ds4.c is a small native inference engine specifically built for DeepSeek V4 Flash, using a Metal graph executor and DS4‑specific loading, prompt rendering, KV state, and server API glue. It supports a 1 million‑token context window and can run with 2‑bit quantization, enabling high‑performance inference on Macs with 128 GB of RAM. The engine exposes an HTTP API compatible with OpenAI and Anthropic, providing endpoints such as /v1/chat/completions, /v1/completions, and /v1/messages. Tool schemas are rendered into DeepSeek’s DSML format and mapped back to OpenAI tool calls, allowing seamless integration with coding agents. The server supports speculative decoding via the MTP path, offering a slight speedup for greedy decoding. ds4.c is built on llama.cpp and GGML, with a focus on local inference and disk‑resident KV cache. The project is released under an MIT license and includes scripts to download the required GGUF models. Users can compile and run ds4.c on a Mac to host DeepSeek V4 Flash locally for development and testing.
Key changes
- ds4.c is a native inference engine for DeepSeek V4 Flash using Metal
- Supports a 1 million‑token context window and 2‑bit quantization
- Provides an HTTP API compatible with OpenAI and Anthropic endpoints
- Tool schemas are mapped to DSML and back to OpenAI tool calls
- Supports speculative decoding via the MTP path
- Built on llama.cpp and GGML with disk‑resident KV cache
- Released under MIT license with scripts to download GGUF models
- Can be compiled and run on a Mac to host DeepSeek V4 Flash locally