Streaming pg_dump straight to S3/R2 (no local disk)

Pipe pg_dump through encryption into aws s3 cp - for zero-disk backups: commands, multipart sizing, and failure handling.

The pipeline

pg_dump "$URL" --format=custom --compress=6 \
  | age -r age1yourkey... \
  | aws s3 cp - "s3://backups/db/$(date +%F).dump.age" \
      --expected-size 5000000000

No temp files, bounded memory, encrypted before it leaves the box. --expected-size matters above ~50 GB so the CLI picks multipart part sizes that don't hit the 10,000-part cap.

Failure handling people forget

If pg_dump dies mid-stream, the pipe closes and aws s3 cp fails — good — but with plain cp - you can leave incomplete multipart uploads costing storage forever. Add a lifecycle rule: AbortIncompleteMultipartUpload after 1 day. Check pipefail is set in bash or the pipeline 'succeeds' when pg_dump fails:

set -euo pipefail

R2 notes

Cloudflare R2 speaks the same S3 API (--endpoint-url https://<account>.r2.cloudflarestorage.com) with zero egress fees — which matters enormously if you restore-test backups, because verification downloads every byte back out. That's why Firedrill stores on R2.