ggufy: Zig‑Based CLI and GUI for Model Quantization
Download ggufy to quickly quantize models with low RAM usage and cross‑platform support.
Install ggufy from GitHub and test quantization on your models.
Summary
ggufy is a new tool written in Zig that provides a single‑file CLI and GUI for quantizing and converting models between gguf and safetensors formats. It is efficient in RAM usage, quantizing ZiT in about 1.5 minutes on a typical machine. The tool supports a wide range of datatypes, from q3_k to q8_0, f32, bf16, f16, f8_e4m3, f8_e5m2, scaled fp8, mxfp8, and nvfp4.
It is cross‑platform, building for Linux, Windows, and macOS (both ARM64 and x86). GitHub Actions provide pre‑built binaries for all platforms. While it does not yet support SDNQ, the author plans to add it if time permits. The project is open source on GitHub.
ggufy offers a lightweight alternative for model quantization, especially useful for users with limited GPU memory.
Key changes
- Single‑file CLI and GUI executables written in Zig.
- Efficient RAM usage; quantizes ZiT in ~1.5 min on typical machine.
- Supports conversion to/from gguf and safetensors.
- Handles datatypes q3_k to q8_0, f32, bf16, f16, f8_e4m3, f8_e5m2, scaled fp8, mxfp8, nvfp4.
- Cross‑platform: Linux, Windows, macOS (ARM64 & x86).
- GitHub Actions provide pre‑built binaries for all platforms.
- Does not yet support SDNQ; potential future addition.
- Open source on GitHub: https://github.com/qskousen/ggufy.