Companies discover corrupt backups during actual outages more often than hardware failure statistics suggest. You need defined RPO/RTO, a 3-2-1 strategy, and scheduled disaster simulation.

The two numbers that matter

RPO defines how much data you accept losing: 15 minutes? 24 hours? RTO defines how long recovery may take. Every technical decision derives from these two numbers, and they belong to the business. Engineering teams that invent a 5-minute RPO when the product would tolerate 4 hours pay dearly for the confusion.

The 3-2-1 rule and a fourth item

Three copies, two media types, one offsite. The fourth item is immutability. AWS documentation recommends S3 Object Lock to protect backups against ransomware. A backup reachable by the same credential as the primary database protects against disk failure and nothing else.

Recovery layers

  • Continuous replication: PITR with WAL archiving in Postgres covers minutes of loss
  • Nightly full backups: cover history and older point-in-time recovery
  • Cross-region copies: survive losing an entire region

Each layer serves one incident class. One layer never suffices.

Automated restore testing

Monthly automated restore into a scratch environment, with checksum verification and query sampling. Restore time comes out of the test measured, never estimated from a spreadsheet. The measured number becomes your real RTO and feeds conversations with enterprise customers.

A runbook for the operator under pressure

Exact commands, access prerequisites, a decision tree separating corruption, accidental deletion, and regional loss. Write for someone who just woke up at 3 AM. A runbook that assumes a calm operator with full context fails when you need it most.

Quarterly game days

Delete a non-production database and restore it against the clock, every quarter. Findings feed the next iteration of the plan. Teams that never practiced restore discover during the exercise that the bucket credentials are missing, or that the 400GB dump does not fit available disk.

Recovery as a commercial requirement

LGPD breach notification windows and enterprise SLAs turn documented recovery capability into a commercial requirement. The disaster recovery plan stopped being an internal ops topic.