PCIe Bandwidth Not a Bottleneck for Multi‑GPU LLM Inference on RTX 5060 TI
Test your PCIe bandwidth; add a third RTX 5060 via an NVMe‑PCIe5 x4 adapter if you need more compute.
Test your PCIe bandwidth; add a third GPU via an NVMe‑PCIe5 x4 adapter if you need more compute.
Summary
The author runs two RTX 5060 TI 16GB GPUs and plans a third, noting the PCIe configuration: GPU0 on a full x16 Gen5 slot running at 8x, GPU1 on a PCI‑E 4.0 x4 via chipset.
A benchmark script using vLLM with TP=2 and 32k context shows peak bandwidth of 3–4 GB/s during prefill, only 40–50 % of the weak 4.0 x4 link, indicating that the GPUs are VRAM‑ or compute‑limited rather than PCIe‑limited.
The author concludes that adding a third 5060 through an NVMe‑to‑PCIe 5.0 x4 adapter using a CPU‑connected M2 slot should not hit a PCIe bottleneck, while a fourth GPU would likely be constrained by chipset lanes on the motherboard.
Key changes
- 2x RTX 5060 TI 16GB GPUs used
- GPU0 on x16 Gen5 slot running at 8x
- GPU1 on PCIe 4.0 x4 via chipset
- Peak bandwidth 3–4 GB/s (~40‑50 % of 4.0 x4 link)
- Adding third 5060 via NVMe‑PCIe5 x4 adapter likely safe
- Adding fourth GPU likely bottlenecked by chipset lanes