New Benchmark Shows Language Models Struggle to Build Full Software Projects

ProgramBench is a newly released benchmark that tests whether large language models (LLMs) can design and implement complete software projects from only a description and accompanying documentation. The benchmark contains 200 tasks that range from small command‑line utilities to large, widely used systems such as FFmpeg, SQLite, and the PHP interpreter. It employs agent‑driven fuzzing to generate end‑to‑end behavioural tests, allowing the evaluation of a model’s output without prescribing a particular implementation structure.

Nine state‑of‑the‑art LLMs were evaluated on ProgramBench. None of the models fully solved any task, and the best performer passed 95 % of the tests on only 3 % of the tasks. The study found that the models tend to produce monolithic, single‑file implementations that diverge sharply from human‑written code, underscoring the current limitations of LLMs in making high‑level architectural decisions. The authors released a public dataset and evaluation framework, providing a valuable resource for future research into holistic program synthesis.

The benchmark’s focus on holistic code generation, rather than isolated bug fixes or feature additions, represents a significant shift in how we assess language models for software engineering. By requiring models to architect and implement entire codebases, ProgramBench exposes gaps in current LMs that can guide the next generation of development tools and research. The results suggest that while LLMs can generate correct code for small tasks, they struggle with architectural design and maintainability, highlighting the need for improved training data and evaluation methods that capture these higher‑level aspects of software creation.

The release of ProgramBench and its accompanying dataset is expected to spur further investigations into how LLMs can be trained or augmented to handle complex software design tasks. Researchers and developers can now benchmark progress, identify weaknesses, and develop new techniques to bridge the gap between current language models and the demands of real‑world software engineering.