Raptor: UDP + RaptorQ + WireGuard for Fast, Low‑CPU S3 Uploads

The author revisits a long‑standing challenge of moving millions of small JSON files to S3, noting that bandwidth is not the bottleneck but round‑trip latency and TLS negotiation dominate. To overcome this, the author prototyped a UDP‑based transfer that splits files into packets, uses RaptorQ for error‑correcting coding, and encrypts traffic with WireGuard, all orchestrated by a lightweight CLI called raptor. The raptor binary, built with .NET 10 AOT, weighs only 3.5 MB on both Linux and Windows and supports configurable bandwidth (--rate-mbps), confirmation levels (--confirm), and repair packet overhead (--overhead). Benchmarks show raptor outperforms aws s3 sync by a small margin and aws s3 cp by 25× for 1 KB–1 MB files, while consuming up to 10× less CPU than TLS‑based uploads. The tool’s design leverages UDP’s low overhead, RaptorQ’s forward error correction to tolerate packet loss, and WireGuard’s lightweight ChaCha20 encryption to secure data in flight without the heavy TLS handshake. By avoiding per‑request TLS negotiation, raptor reduces CPU usage dramatically, making it attractive for greenfield projects that need high‑throughput, low‑latency data ingestion. The author invites contributors to clone the repo, experiment with different backends, and tweak the CLI options to further improve performance. The article emphasizes that while raptor loses advantage for large files due to fixed bandwidth, its benefits shine for small, frequent uploads typical in static API scenarios.