Benchmarks for Local Qwen as a Codex Validator and Co‑Agent

The author describes a workflow where Codex performs the main repository work while a local Qwen model acts as a validator, challenger, and quality checker. The evaluation suite, built around this use case, tests Qwen3.6 27B GGUF profiles via llama.cpp, including Bartowski and Unsloth variants, with 128k and 65k context sizes and q8/f16 KV cache options. Results show that the best 128k profiles—bartowski‑128k‑f16, bartowski‑128k‑q8, and unsloth‑128k‑q8—tied for top performance, and q8 KV cache did not degrade accuracy in this suite. Context size proved more critical than f16 versus q8 KV precision, as 65k profiles failed when the suite required more than 65k context. The unsloth‑128k‑f16 variant loaded but hit local memory and throughput limits on long‑context cases, stressing the 5090 GPU. The author notes that Qwen excels at UI design, often taking the lead when UI is involved, while Codex handles implementation, and that the combined workflow prevents silent bypasses and overbuilding.

The project page and GitHub repo provide the reproducible evaluation suite, inviting feedback from developers using local models as coding companions or sidecar agents. The author specifically seeks real‑world test cases that highlight failures such as missed directives, overbuilding, UI judgment, and long‑context misses.

Overall, the study demonstrates that a local Qwen model can effectively complement Codex in a coding workflow, improving quality checks and UI design oversight.