Moving Media Storage Out of Postgres: A Reserve/Finalize Workflow
Move media uploads to DO Spaces with presigned URLs, keep encryption keys client‑side, and use a reserve/finalize workflow so the server never sees plaintext.
Implement the reserve/finalize workflow and presigned S3 uploads to eliminate server‑side plaintext storage.
Summary
The author explains why storing large media blobs in Postgres BYTEA caused WAL explosions, slow backups, memory pressure, and connection hold times. To solve this, he moved to a reserve/finalize workflow that stores encrypted media in DO Spaces via presigned URLs while the server only records size and hash. The client encrypts the file with AES‑GCM, streams it to S3, and the server never sees the plaintext or the encryption key.
The new architecture uses POST /media/reserve to obtain an upload URL and media_id, then PUT to the presigned URL, followed by POST /media/finalize to mark the media as ready. The media key is wrapped for each recipient and sent in the message, allowing the recipient to download and decrypt locally. Client timeout settings were increased to 10 minutes and retries disabled to avoid orphan blobs. Streaming encryption was implemented to keep memory usage low on mobile devices, and TOAST limits were avoided by external storage.
Key changes
- POST /media/reserve returns upload URL, media_id, and AES‑GCM key.
- Client encrypts file, streams to DO Spaces via presigned URL.
- POST /media/finalize records size and SHA‑256 hash only.
- Server never sees plaintext or encryption key.
- Media key wrapped per recipient and sent in message.
- Client timeout set to 10 minutes, retries disabled.
- Streaming encryption implemented to avoid memory pressure.
- TOAST limitations avoided by external storage.