Disaster Recovery Strategies
Concept
Disaster recovery is a spectrum, not a single decision — every strategy trades standby infrastructure cost against how fast and how completely you can recover after a Region (or major dependency) goes down. The four common tiers, in increasing cost and decreasing RTO/RPO:
- Backup and Restore — No standing infrastructure in the recovery Region. Data is backed up (snapshots, exports) and everything is rebuilt from scratch during a disaster.
- Pilot Light — Only the most critical core (typically the database, replicating continuously) runs in the recovery Region at all times. Application/compute tiers exist as configuration but are switched off, scaled up only during failover.
- Warm Standby — A scaled-down but fully functional copy of the entire production stack runs continuously in the recovery Region (smaller instance sizes/counts). Scale it up to full capacity on failover.
- Multi-Site Active/Active — Full production-scale infrastructure runs simultaneously in two or more Regions, actively serving live traffic in normal operation, not just standing by.
Tradeoffs
| Strategy | RTO / RPO | Standing cost | Failover complexity |
|---|---|---|---|
| Backup & Restore | Hours | Lowest (backup storage only) | High — rebuild from scratch under pressure |
| Pilot Light | Tens of minutes | Low (DB replication + idle config) | Medium — scale up the app tier |
| Warm Standby | Minutes | Medium (a smaller live copy, always running) | Low — scale the existing copy up |
| Multi-Site Active/Active | Seconds to near-zero | Highest (full duplicate capacity) | Lowest — traffic is already flowing there |
The underlying tension is the same one that runs through every resilience decision: money spent on idle standby capacity buys you recovery speed you hope to never need. There is no strategy that is "best" independent of the actual RTO/RPO the business requires and what it's willing to pay to guarantee it.
When to use / when not to
- Backup & Restore fits workloads where a multi-hour outage is genuinely tolerable and minimizing standing cost matters more than recovery speed — internal tools, non-critical batch systems.
- Pilot Light fits systems with a real but not extreme urgency, and where the data tier is the expensive/slow part to rebuild (so keeping it warm while compute stays cold is the efficient split).
- Warm Standby is the fit whenever a question (or a real requirement) specifies an RTO in the tens-of-minutes range, rules out paying for a full duplicate, and doesn't want the secondary Region serving live traffic unless it's actually needed.
- Multi-Site Active/Active only earns its cost when near-zero downtime is a hard business requirement (payments, trading systems) — anything less critical and it's paying full active-capacity cost for a recovery speed nobody asked for.
Common pitfall
Picking a DR tier based on what "sounds safe" rather than the actual stated RTO/RPO and budget constraint. Over-provisioning (Multi-Site Active/Active for a system that could tolerate Warm Standby) burns budget that could fund resilience work elsewhere; under-provisioning (Backup & Restore for a system that actually needs a 30-minute RTO) fails silently until the day it's tested for real.
Principal Engineer Lens
This is a pure cost-vs-speed negotiation, and the Principal-level answer is never "pick the safest option" — it's translating a business RTO/RPO requirement into the cheapest tier that actually satisfies it, and being able to justify that translation explicitly in an architecture review. Naming the failure mode of the tier you didn't pick (why Backup & Restore's rebuild time would blow the RTO, or why Active/Active's cost isn't justified here) is what signals real ownership of the tradeoff, not just familiarity with the four names.