Deep Agents Open Source Agent Architecture Introducing Rubrics: Build Agents that Evaluate and Correct Their Work
Add RubricMiddleware to your DeepAgent to enforce rubric compliance.
Add RubricMiddleware to your DeepAgent to enforce rubric compliance.
Summary
RubricMiddleware, added to Deep Agents, introduces a grader sub‑agent that evaluates an agent’s output against a rubric and iterates until all criteria are met or a maximum number of iterations is reached. The middleware is configured with a smaller LLM (e.g., Claude‑haiku‑4‑5), a system prompt defining grading rules, optional tools for evidence (such as run_test_suite), and a max_iterations limit. When an agent finishes, the grader reviews the run; if any rubric item fails, the feedback is injected back into the conversation and the agent runs again. The loop terminates on satisfied, max_iterations_reached, failed, or grader_error. The middleware is useful for tasks with clear success criteria, such as code generation, test passing, or naming conventions. The API is currently in beta and may change.
The example demonstrates a Python function that initially fails a test, receives targeted feedback from the grader, and then passes all tests on the second iteration. RubricMiddleware shifts the burden of catching variance from developers to the system, enabling more reliable agents on correctness‑critical tasks.
Key changes
- Adds a grader loop on top of base agent
- Grader uses a smaller LLM (e.g., Claude‑haiku‑4‑5)
- System prompt defines grading criteria
- Optional tools (e.g., run_test_suite) for evidence
- max_iterations limits fix‑regrade loops
- Middleware injects per‑criterion feedback
- Supports code generation, test passing, naming conventions
- Beta API may change