Reimagining ML Operations with Agent Skills: a new maturity model for on-call

Anyscale Agent Skills, released last month, provide token‑efficient, pre‑built skills that let Ray pipelines be built, deployed, and operated with minimal human intervention. The authors introduce a new on‑call default that splits ML Ops into three phases—day 0 (build golden‑path templates, measure time to first PR), day 1 (deploy platform interfaces, track failure rate), and day 2 (operate, measure MTTR and business impact). Each phase now has a core success metric and the skills are organized into three families: workload skills that generate Ray code for training, serving, and data pipelines; platform skills that run, inspect, and fix code on Anyscale; and infrastructure skills that deploy Anyscale on Kubernetes or VMs. By automating these activities, the on‑call tax drops from 20‑30 % of a team’s sprint to a few hours, while still allowing engineers to focus on higher‑value work. The maturity model proposes stages—from the current “coffee‑break” state to an open‑loop first‑responder model—guiding teams toward increasingly autonomous operations. Anyscale’s unified Ray runtime gives agents a single Python‑native control plane, reducing token burn and translation overhead across system boundaries. The article emphasizes that the new skills can be enabled today with minimal investment and deliver day‑one benefits in build, deploy, and incident response. Finally, the authors invite platform leaders to evaluate this paradigm to accelerate velocity for researchers and platform engineers alike.