DigitalOcean Launches Inference Router to Optimize LLM Routing

DigitalOcean has released its Inference Router, a new AI-native proxy that automatically selects the most suitable large language model for each request based on task intent, cost, and latency. The router is powered by a 30‑billion‑parameter Mixture‑of‑Experts model fine‑tuned for intent detection, outperforming GPT‑5.1 and Claude Sonnet 4.5 on multi‑turn conversations. It ships with preset routers for Software Engineering, General, Writing, and Knowledge Base workflows, each offering Optimal, Cost‑Efficiency, and Speed‑Optimization selection policies. Developers can also create custom routers by defining tasks, model pools, and fallback models, and test routing behavior in the DigitalOcean Model Studio Playground. The underlying Plano engine classifies intent with a small language model and ranks candidate models using live cost and latency data from DigitalOcean’s pricing API and Prometheus. The router exposes a simple API endpoint where the model field is prefixed with “router:”, and the response includes the chosen model and matched task.

The Inference Router also provides an Evals feature for dataset‑based quality assessment and a dashboard that tracks total requests, token usage, task match rate, and fallback rate. By integrating the router, teams can reduce overpayment for frontier models, lower inference costs, and avoid brittle routing logic in application code.