AI Twitter Recap: Harnesses, Agents, and Benchmark Shifts
Run gpt‑5.2‑codex on Terminal‑Bench 2.0 to confirm the 66.5 % score and benchmark TSP on 1024 MI300X GPUs for 173 M tok/sec.
Run gpt‑5.2‑codex on Terminal‑Bench 2.0 to confirm the 66.5 % score and benchmark TSP on 1024 MI300X GPUs for 173 M tok/sec.
Summary
The recap highlights that the harness is becoming the product boundary, with context pipeline lock‑in outweighing model weights, as shown by Anthony Maio’s claim that the context pipeline drives competitive advantage. Mason Drxy reported that tuning prompts and middleware in the harness boosted gpt‑5.2‑codex from 52.8 % to 66.5 % on Terminal‑Bench 2.0 and raised gpt‑5.3‑codex by 20 % on tau2‑bench. Open harnesses are maturing, with Hermes Agent Kanban for visual multi‑agent coordination, a Spanish‑language war‑room UI, and LangChain’s deepagents/LangGraph adding model‑specific harness profiles, schema migrations, node‑level error handlers, timeouts, and new streaming primitives. PyFlue extended the agent harness concept into Python, positioning harnesses as the missing layer between raw model calls and durable agents.
Model‑agnostic orchestration is a design goal, with deepagents‑cli supporting Kimi, Qwen, GLM, hosted Ollama, OpenRouter, LiteLLM, Baseten, and LangChain Fleet adding multi‑model sub‑agent routing, enabling teams to cut agent costs by >20 ×. Coding‑agent UX is shifting developer behavior, with dbreunig’s commandments—implement to learn, rebuild often, E2E tests, document intent, maintain spec—and pricing instability, as a single Copilot message can consume 60 M+ tokens costing hundreds of dollars against a $40 subscription. Agents are spreading beyond coding, exemplified by Codex Security’s five AppSec workflows, Google Slides generation, a fully local llama.cpp assistant, and Noustiny’s Hermes‑based video‑generation pipeline. Benchmark design is under revision, with HiL‑Bench testing spec completeness, MathArena as a continuously maintained platform, and Meta FAIR’s Autodata showing a 34‑point gap between weak and strong solvers on a CS QA task.
Systems updates include folded Tensor and Sequence Parallelism (TSP) on 1024 MI300X GPUs, 128K context, 8 GPUs per model copy, achieving 173 M tok/sec versus 86 M for TP+SP, and an extension to MoE MLPs for larger training runs.
Key changes
- Harness context pipeline lock‑in drives competitive advantage over model weights
- gpt‑5.2‑codex accuracy increased from 52.8 % to 66.5 % on Terminal‑Bench 2.0; gpt‑5.3‑codex improved by 20 % on tau2‑bench
- Hermes Agent Kanban, Spanish war‑room UI, and LangChain deepagents/LangGraph added model‑specific harness profiles, schema migrations, node‑level error handlers, timeouts, and new streaming primitives
- PyFlue introduced a Python agent harness layer between raw model calls and durable agents
- deepagents‑cli now supports Kimi, Qwen, GLM, hosted Ollama, OpenRouter, LiteLLM, Baseten; LangChain Fleet added multi‑model sub‑agent routing
- Copilot message can consume 60 M+ tokens, costing hundreds of dollars against a $40 subscription; 15 messages ~221 tokens
- Codex Security plugin offers five AppSec workflows; Google Slides generation via Codex; fully local llama.cpp assistant; Noustiny Hermes‑based video‑generation pipeline
- Folded Tensor and Sequence Parallelism (TSP) on 1024 MI300X GPUs, 128K context, 8 GPUs per model copy, achieves 173 M tok/sec versus 86 M for TP+SP; extended to MoE MLPs