SAA-C03 Study Guide: Domain 2 — Designing Resilient Architectures

1. Domain Overview and Scaling Fundamentals

Domain 2, Designing Resilient Architectures, accounts for 26% of the SAA-C03 exam. The absolute core of this domain is the systematic elimination of Single Points of Failure (SPOFs). In the "AWS Way," resiliency is not just about keeping a server running; it is about building a system that survives the failure of any single component—be it an instance, a disk, or an entire Availability Zone (AZ).

Vertical Scaling vs. Horizontal Scaling

Feature	Vertical Scaling (Scale Up/Down)	Horizontal Scaling (Scale Out/In)
Mechanic	Increasing the size (CPU, RAM) of an existing resource.	Adding more resources of the same size to a fleet.
Trade-offs	Involves downtime; reaches a Hard Hardware Limit (the largest instance available).	Supports High Availability; zero downtime; Theoretically Infinite scale.
Elasticity	Low; difficult to automate.	High; managed by the Auto Scaling Group (ASG).
AWS Example	Changing a t3.medium to a t3.2xlarge.	Adding EC2 instances to an ASG across multiple AZs.

Elasticity is the automated application of horizontal scaling. By using an ASG, we match capacity to demand in real-time, which addresses three Well-Architected pillars: Performance, Cost, and Reliability.

2. High Availability (HA) vs. Fault Tolerance (FT)

Architects must distinguish between three levels of resilience based on the business's budget and uptime requirements:

High Availability (HA): Focused on minimizing downtime through automated recovery. The system is up "as often as possible."
Fault Tolerance (FT): The system operates through a failure with zero service degradation. This is significantly more expensive as it requires "full duplicate capacity" running live at all times.
Disaster Recovery (DR): The process for recovering systems after a major regional or infrastructure disaster.

Resilience Progression: A Worked Example

Consider an application's evolution from a single server to a highly resilient state:

Single Server: No redundancy. If the server fails, you are offline until manual intervention.
Highly Available (HA) Design: ALB + ASG with instances in two AZs.

User Experience: If an instance fails, there may be a brief blip or a required re-login while the ALB health checks fail and traffic is routed to the remaining healthy instance.
Fault Tolerant (FT) Design: Multiple active instances across AZs with enough redundant capacity to absorb a failure instantly.

User Experience: If an instance fails, there is zero downtime and no impact on the user because the redundant capacity is already active and serving traffic.

3. Decoupling and Loose Coupling with Messaging Services

Resilience is achieved by ensuring components are autonomous. If Service A must be online for Service B to work, you have a "distributed monolith."

Synchronous Integration: Components are coupled in availability (e.g., a direct HTTP call).
Asynchronous Integration: Uses durable stores (SQS, DynamoDB) to buffer requests.

Amazon SQS Deep Dive

Feature	Standard Queues	FIFO Queues
Ordering	Best-effort; messages can be out of order.	Guaranteed exact order (First-In-First-Out).
Delivery	At-least-once (duplicates possible).	Exactly-once processing (no duplicates).
Throughput	Nearly unlimited.	300 msg/sec (up to 3,000 with batching).

Visibility Timeout (0s–12h, default 30s): When a worker picks up a message, it becomes invisible to others. If the worker crashes, the timeout expires and the message reappears. The Fix: If a message is processed twice by two workers, increase the visibility timeout.
Message Retention Period (60s–14 days, default 4 days): How long an unconsumed message stays in the queue before SQS deletes it automatically. Not a concurrency control.
DelaySeconds (0s–900s, default 0): Postpones delivery of new messages so they aren't visible to consumers immediately after being sent. A queue configured with this set is called a "Delay Queue" — it staggers delivery timing but has no bearing on message ordering.
WaitTimeSeconds (0s–20s): Enables long polling — a ReceiveMessage call waits up to this long for a message to arrive instead of returning empty immediately. Reduces empty responses and cost, not related to duplicate processing.
Dead-Letter Queue (DLQ): A separate queue for messages that fail processing N times, managed by a redrive policy.

Amazon SNS & The Fan-Out Pattern

SNS is a push-based pub/sub service. The "Fan-Out" architecture allows one event to trigger multiple processes simultaneously:

An event is published to an SNS Topic.
The topic pushes to multiple SQS Queues (for parallel processing), Lambda (for logic), or S3 (via Firehose for archival).

Why SNS + SQS (not direct HTTP/S subscriptions): A raw HTTP(S) endpoint subscription is push-only and undurable — if the downstream service is down when SNS pushes, the message can be lost. Fronting each subscriber with its own SQS queue makes the fan-out durable: the message sits in the queue until that consumer is back up and polls for it. This is also why "SNS → multiple SQS queues" beats "one SQS queue → multiple Lambda triggers" for fan-out: a standard SQS message is deleted once any single consumer processes it, so one queue cannot deliver the same message to several independent services — each downstream service needs its own subscription and its own queue.

Amazon EventBridge

A serverless event bus that uses Rules to route events from AWS, custom apps, or SaaS (Salesforce) to targets. It is the modern standard for complex event-driven orchestration.

4. Disaster Recovery (DR) Strategies and Metrics

DR is governed by two metrics:

RTO (Recovery Time Objective): Maximum tolerable downtime.
RPO (Recovery Point Objective): Maximum tolerable data loss.

The Tell: "Back online in 1 hour" = RTO. "No more than 5 minutes of data loss" = RPO. Synthesis: Tight RPO requires Continuous Replication; Tight RTO requires Pre-provisioned Compute.

The DR Strategy Ladder

Strategy	Cost	RPO	RTO	AWS Implementation
Backup & Restore	Lowest	Hours	24 hours or less	S3 Snapshots; restored post-event.
Pilot Light	Low	Minutes	Hours	RDS Read Replica alive; Compute at zero.
Warm Standby	Medium	Seconds	Minutes	Scaled-down version of stack in DR Region.
Multi-Site	Highest	Near-Zero	Near-Zero	Full duplication; Active-Active (Route 53).

Figures per AWS's own Disaster Recovery of Workloads on AWS whitepaper. Note Pilot Light's RTO sits in the hours range (compute must scale up from zero) — this is why a 30-minute RTO requirement rules out Pilot Light in favor of Warm Standby, as in the worked example below.

Worked Example: Requirement is "RTO of 30 mins and RPO of 5 mins" at the lowest cost. The Answer: Warm Standby. Pilot Light's RTO is too high (hours to provision compute), while Multi-Site is too expensive.

5. Resilient Compute and Networking Patterns

The Self-Healing Pattern

ALB + ASG across multiple AZs. The ALB performs health checks; if an instance fails, the ASG launches a replacement in a healthy AZ to maintain Desired Capacity.

ASG Default Termination Policy (Scale-In)

When an ASG scales in, AWS evaluates criteria in this order to pick which instance to terminate:

AZ balance first: Terminate from whichever AZ currently has the most instances, to keep the fleet balanced across AZs.
Oldest launch configuration/template within that AZ: Among the candidates in the over-represented AZ, prefer the instance running the oldest launch configuration or launch template version.
Closest to the next billing hour: Used only as a tiebreaker after the above two criteria are exhausted.

Common Trap: Billing-hour proximity is not the primary rule — it only breaks ties after AZ balance and configuration age have been considered.

Cross-Region Resilience

Route 53 Failover Routing: Active-Passive DNS-based failover.
AWS Global Accelerator: Uses Anycast IPs and the AWS global backbone. Best for TCP/UDP (non-HTTP) protocols and faster failover than DNS.

VPC Connectivity & Common Traps

Gateway Endpoints: For S3 and DynamoDB. They are FREE and use route table targets. Use these to save costs compared to NAT Gateways.
Interface Endpoints (PrivateLink): For most other services. Uses an ENI and incurs costs.
Common Trap (Lambda): A Lambda function in a VPC loses internet access unless a NAT Gateway is present in the public subnet.

Lambda Concurrency Controls & Cold Starts

Provisioned Concurrency: Pre-initializes a specified number of execution environments so they are already warm, ready to respond with no init latency. This is the direct fix for cold starts, and it can be scaled on a schedule (e.g., ramped up before a known peak-traffic window).
Reserved Concurrency: Caps (and guarantees) the maximum number of concurrent execution environments for a function. It controls scaling ceilings and protects downstream resources (like a database) from being overwhelmed — it does not pre-warm anything, so it has no effect on cold starts.
Lambda@Edge: Runs Lambda code at CloudFront edge locations to reduce network latency to the caller. It addresses network distance, not the function's own initialization (cold-start) time.
Timeout: Controls how long a single invocation is allowed to run before being terminated. Unrelated to cold-start latency, which occurs before the handler code even starts running.

6. Resilient Database Architectures

RDS Multi-AZ vs. Read Replicas

Multi-AZ (High Availability): Synchronous replication to a standby. Failover-only.
Read Replicas (Performance): Asynchronous replication to readable copies.

Common Trap: Multi-AZ provides zero performance benefit. It is for durability and failover.

Architect's Note: Replicas vs. Caching

A Read Replica is not a cache. A replica still involves database overhead (connections, SQL parsing, auth). Amazon ElastiCache (Redis/Memcached) provides sub-millisecond, in-memory performance, offloading the database entirely for frequent hits.

Global Resilience & Serverless Integration

Aurora Global Database: Primary Region + 5 Secondaries. RPO < 1s, RTO < 1min.
DynamoDB Global Tables: Multi-active, bidirectional replication.
RDS Proxy: Sits between the application and the database, pooling and sharing a small set of established DB connections across many callers. This is the fix for Lambda's "too many connections" errors — each concurrent Lambda invocation would otherwise open its own DB connection, quickly exhausting the database's connection limit; RDS Proxy multiplexes those onto a managed pool instead. It also reduces failover times by up to 66% and offloads credential management to AWS Secrets Manager + IAM, critical for serverless (Lambda) resilience where connection churn is high.

7. Exam-Critical Decision Rules (The "Tells")

Scenario Requirement	The Correct "Tell"
"Order must be preserved / No duplicates"	SQS FIFO
"Minimize cold starts / Consistent low latency"	Lambda Provisioned Concurrency
"Block specific malicious IP across a subnet"	Network ACL (NACL) — Security Groups cannot Deny.
"Maintain stateful sessions on a single target"	ALB Sticky Sessions
"SFTP to S3 without managing servers"	AWS Transfer Family
"Most cost-effective millisecond access to rarely used archive"	S3 Glacier Instant Retrieval
"Shared storage for thousands of EC2 instances"	Amazon EFS
"Centralized firewall traffic inspection at scale"	Gateway Load Balancer (GLB)
"Auto-classify helpdesk tickets using ML"	Amazon Comprehend
"Least operational overhead for containers"	AWS Fargate
"Low-cost, encrypted config (no rotation)"	SSM Parameter Store
"Auto-rotate DB passwords every 30 days"	AWS Secrets Manager