Design Highly Available and/or Fault-Tolerant Architectures

Three Concepts, Precisely Distinguished

• High Availability (HA): system stays up "as often as possible"; when a component fails, it's replaced/fixed as fast as possible to minimize downtime. HA does not mean zero failures or zero downtime — brief downtime (e.g., users having to re-log-in on failover) is acceptable.
• Fault Tolerance (FT): the system keeps operating through a component failure — no downtime at all, because redundant active components absorb the failure instantly. Costs more than HA because you're running full duplicate capacity live, not standby.
• Disaster Recovery (DR): pre-planned process for what to do when (not if) a disaster occurs. DR planning must happen before the disaster — "the worst time to figure out recovery is during the disaster." Backups must live offsite/off-building/in-the-cloud, never co-located with primary systems. Practice DR drills so the real event goes smoothly.

The Worked Example (single server → HA → FT)

• Single server, no redundancy = outage on failure.
• HA fix: spin up a replacement fast, or run active/standby with failover (some downtime, e.g. re-login, acceptable).
• FT fix: run two active servers simultaneously — if one fails, the other is already serving, zero downtime.

Design Inputs Beyond "add more servers"

Redundancy design must also account for: data replication, traffic management, failure detection — not just duplicate compute/DB counts. AWS global infrastructure (Regions/AZs) is the substrate that makes both HA and FT achievable.

Finding Single Points of Failure (SPOF)

• Technique: work backwards from failure — for each component, ask "what happens if this dies?" (web tier, app tier, DB tier, etc.), then design mitigation + DR strategy for each.

RTO / RPO — the DR Sizing Inputs

• RPO (Recovery Point Objective): max acceptable time since the last recoverable data point — drives backup frequency.
• RTO (Recovery Time Objective): max acceptable delay between outage and restored service — drives how "warm" your DR environment must be.

DR Strategy Ladder

• Active/passive: Backup & Restore → Pilot Light → Warm Standby (increasing cost, decreasing RTO/RPO).
• Active/active: Multi-site active-active (near-zero RTO/RPO, highest cost/complexity).
• These aren't mutually exclusive — mix strategies per-component based on criticality.

Storage & Database Backup Options to Know

• S3: low-cost, extremely durable object storage — natural DR target.
• EFS / FSx: file storage DR options — know availability, encryption, and limitations of each.
• RDS Multi-AZ: automatic failover if primary DB fails — know the mechanics (standby promotion) and roughly how long access is interrupted during failover.
• Aurora Global Database: cross-Region failover — know the typical failover time it's designed for.
• DynamoDB Global Tables: multi-Region, multi-active replication.
• AWS Elastic Disaster Recovery: DR for legacy / non-cloud-native apps, both on-prem and cloud-based.
• AMIs + EC2 Image Builder: DR strategy building block for EC2-based recovery (rebuild known-good images fast).

Networking Building Blocks for Resilience

Route tables, VPC peering, Transit Gateway, Site-to-Site VPN, Direct Connect (+ Direct Connect Gateway), Route 53 Resolver — all foundational to designing HA/FT across VPCs and hybrid environments.

Deployment & Verification Tooling

• Automate deployments: Elastic Beanstalk, CloudFormation, OpsWorks (and ECS/EKS for containers).
• Check resource/code integrity: Amazon Inspector, Amazon CodeGuru.

The "Self-Healing" Pattern (single-instance / single-AZ failure)

• Elastic Load Balancer spanning multiple AZs + EC2 Auto Scaling Group sized to a min/max/desired configuration = the standard answer for surviving an instance failure or an entire AZ outage.
• Global failover: Route 53 routing policies (including failover routing) + AWS Global Accelerator for availability/performance across Regions.
• RDS Proxy (again, cross-referenced here): reduces Aurora/RDS failover time by up to 66%, and offloads credential management to Secrets Manager + IAM — critical for serverless apps with high connection churn.

Observability Feeds Resilience

• CloudWatch (metrics/alarms) + AWS X-Ray (distributed tracing) for continuous improvement per the Well-Architected Framework.
• EventBridge for near-real-time automated response to environment changes.

Unusual but Recurring Exam Callout: Amazon Polly / Comprehend

• Scenario: large volume of IT service requests, manual ticket categorization causes delays and SLA breaches at scale, especially during outages.
• Amazon Comprehend: auto-classifies requests (ML/NLP) to route them to the right team faster.
• Amazon Polly: adds spoken/self-service output (e.g., in an Amazon Connect contact center) — flagged as reappearing on recent exams, so don't skip it as a curiosity.

Exam Angle

Expect explicit HA-vs-FT-vs-DR definition traps, RTO/RPO-to-strategy matching, "how do I survive one AZ loss" (ALB + ASG) vs. "how do I survive Region loss" (Route 53/Global Accelerator + cross-Region DB strategy) framing, and at least one Comprehend/Polly-flavored curveball.

Practical Examples

HA vs. FT, same app, two designs: An internal HR portal used during business hours can tolerate a 60-second blip during failover — deploy it as HA: one active EC2 instance behind an ALB with an ASG (min=1, max=2) that replaces a failed instance automatically; users see a brief error and retry. A payment-authorization service that cannot drop a single transaction — deploy it as FT: two (or more) instances active simultaneously behind the ALB at all times, so a failure of one is invisible to callers, zero downtime, at roughly double the steady-state compute cost.

RTO/RPO driving the DR tier: An internal wiki — RPO of 24 hours (daily backup is fine), RTO of 8 hours (IT can restore from backup tomorrow) → Backup and Restore. A core banking ledger — RPO of seconds, RTO of minutes → Warm Standby or Multi-Site Active-Active, because losing even a minute of transaction data is unacceptable.

Self-healing pattern, concretely: An ASG spans 3 AZs in us-east-1 with min=3, max=9, desired=3. One AZ (us-east-1a) has an outage. The ALB health checks mark that AZ's instances unhealthy, stops routing to them, and the ASG launches replacement instances in the two healthy AZs to maintain desired capacity — the app keeps serving traffic through the whole event with no human intervention.

RDS Multi-AZ failover, what actually happens: Primary RDS instance in us-east-1a crashes. AWS automatically promotes the synchronously-replicated standby in us-east-1b and flips the DB's CNAME to point at it — typically a 60–120 second interruption, during which the app sees connection errors and should retry with backoff. This is why RDS Proxy pairs well here: it can mask much of that failover window from the application.

Aurora Global Database for cross-Region DR: A SaaS company's primary Aurora cluster is in us-east-1; they add an Aurora Global Database secondary cluster in eu-west-1 with sub-second replication lag. If us-east-1 has a Region-wide event, they manually (or via automation) promote the eu-west-1 secondary to a standalone writable cluster — typically achieving an RTO measured in under a minute for the promotion itself, though DNS cutover adds more.

DynamoDB Global Tables: A gaming leaderboard needs to be writable from both US and EU Regions with low latency for players in both, and survive a full Region outage without manual failover. DynamoDB Global Tables replicates writes bidirectionally between Regions automatically — either Region can serve reads/writes at any time (multi-active), unlike Aurora Global Database's single-writer model.

AWS Elastic Disaster Recovery for legacy apps: A company has a 15-year-old on-prem Windows app that can't be re-architected before a compliance deadline requiring cloud DR. AWS Elastic Disaster Recovery (DRS) continuously replicates the on-prem server's block storage to a low-cost staging area in AWS; on a real disaster, it launches full EC2 replicas within minutes — no app rewrite required.

Comprehend + Polly, concretely: A telecom's IT helpdesk gets 5,000 tickets/day, manually triaged, causing SLA misses during outages. Amazon Comprehend auto-classifies incoming tickets (network vs. billing vs. hardware) so they route to the right team instantly; Amazon Polly reads back a "your ticket has been received and routed to Network Ops" confirmation over the phone via Amazon Connect, with zero human agent time spent on that step.