Open Agent Leaderboard Launches Benchmark for Full Agent Systems

IBM Research has released the Open Agent Leaderboard, an open evaluation framework that benchmarks complete agent systems rather than just the underlying models. The leaderboard pairs with the Exgentic framework and reports both quality and cost across six diverse benchmarks—SWE‑Bench, BrowseComp+, AppWorld, tau2‑Bench Airline & Retail, tau2‑Bench Telecom, and a research‑style task—providing a spectrum of real‑world scenarios. Each row in the leaderboard shows a specific agent paired with a specific model, along with average success rate, cost per task, and per‑benchmark breakdowns, revealing how agent architecture and tool use drive performance. The top configurations demonstrate that the same model can yield vastly different scores and costs depending on the surrounding agent system. General‑purpose agents already match or outperform specialized systems on many tasks, and failure behavior can increase cost by 20–54 %. Exgentic enables reproducible evaluations and is open‑source, encouraging community contributions of new agents, benchmarks, and models. The leaderboard now includes open‑weight models such as DeepSeek V3.2 and Kimi K2.5, which trail closed‑source models by 18–29 % on average. By exposing both quality and cost, the leaderboard helps teams decide which agent design is worth deploying in production.