OpenAI Releases Multipath Reliable Connection (MRC) Protocol for AI Supercomputers
Learn that OpenAI released MRC, a new protocol that spreads transfers across hundreds of paths, uses SRv6 static source routing, and eliminates BGP, enabling microsecond failure recovery on 800 Gb/s interfaces.
Review the MRC specification and assess whether it can replace BGP in your AI training clusters.
Summary
OpenAI has released the Multipath Reliable Connection (MRC) protocol, a new networking standard built into 800 Gb/s interfaces that can spread a single transfer across hundreds of paths. The protocol, co‑developed with AMD, Broadcom, Intel, Microsoft and NVIDIA, is published as an Open Compute Project contribution so the wider industry can adopt it. MRC extends RoCE and adds SRv6‑based static source routing, allowing packets to be sprayed across multiple 100 Gb/s planes, trimmed on congestion, and rerouted around failures in microseconds. It has already been deployed on OpenAI’s NVIDIA GB200 supercomputers, the Oracle Cloud Infrastructure site in Abilene, Texas, and Microsoft’s Fairwater cluster, training multiple frontier models for the 900 million weekly ChatGPT users.
MRC’s design eliminates the need for dynamic routing protocols like BGP, simplifying the control plane and reducing failure surface. By embedding a sequence of switch identifiers in the IPv6 address, each switch performs a static lookup and forwards the packet, while the sender can immediately drop a congested or failed path and probe for recovery. The protocol’s packet‑spraying and trimming mechanisms reduce false positives and keep synchronous training jobs running with minimal idle GPU time. OpenAI’s paper, “Resilient AI Supercomputer Networking using MRC and SRv6,” details the performance gains and reliability improvements achieved with this new approach.
Key changes
- MRC protocol spreads a single transfer across hundreds of paths on 800 Gb/s interfaces, enabling multi‑plane 100 Gb/s networks
- Uses SRv6‑based static source routing, embedding switch identifiers in IPv6 addresses to eliminate dynamic routing (BGP)
- Implements packet spraying, trimming, and load‑balancing to reduce congestion and false‑positive failure detection
- Provides microsecond‑scale failure detection and rerouting, minimizing idle GPU time in synchronous training jobs
- Published as an Open Compute Project contribution, allowing industry adoption
- Deployed on NVIDIA GB200 supercomputers, Oracle Cloud Infrastructure in Abilene, and Microsoft Fairwater clusters