Hermes Wiki

Domain 3 Wrap-Up — Review Checklist

Purpose

Zoom-out review across all five Domain 3 task statements — storage, compute, database, network, data ingestion/transformation — with an explicit reminder that scalability cuts across all five, not just the storage/compute lessons.

Storage Recall

  • Object vs. block vs. file storage distinction, mapped to S3, EFS, FSx, EBS.

Compute Recall

  • Scaling options across EC2, Batch, EMR, Fargate, Lambda — know how each scales, not just that it exists.
  • SQS and SNS capabilities — specifically called out via a scenario: SQS queue processed by Lambda — how does error handling behave at scale (retries, DLQs), and what hard/soft quotas constrain the Lambda function (concurrency limits, payload size, etc.)?

Serverless Beyond Lambda

  • API Gateway: throttling and quotas configuration to protect APIs from being overwhelmed — a direct performance lever, not just an access-control feature.
  • Step Functions, EventBridge — orchestration/event-routing tools to know at a functional level.
  • AWS AppSync: managed, scalable GraphQL interface that combines data from multiple sources (DynamoDB, Lambda, HTTP APIs) into one API. Know what GraphQL is as a fundamental — this is called out explicitly as something to not skip.

Database Recall Checklist

  • Capabilities/limits of RDS, Aurora, DynamoDB, ElastiCache, DAX, Redshift.
  • Which services support Read Replicas, and how Read Replica mechanics differ per service.
  • Which services offer cross-Region support, and how global databases are implemented (Aurora Global Database, DynamoDB Global Tables).
  • How auto-scaling is implemented per database service (differs meaningfully between RDS storage auto-scaling and Aurora Serverless ACU scaling).
  • Caching strategies: lazy loading vs. write-through — know the trade-offs of each for a caching layer.
  • Capacity planning per solution: e.g., new DynamoDB app with unclear access patternson-demand capacity is the safer initial choice over provisioned (avoids under/over-provisioning guesswork).
  • Migrated legacy app on RDS MySQL, mostly read-heavy traffic → choose between Read Replicas and ElastiCache based on the actual bottleneck (read scaling vs. repeated-query caching).

Network Recall Checklist

  • CloudFront, Global Accelerator, VPC endpoints for optimized network traffic.
  • Build a VPC from scratch: subnets, routing, internet gateways, peering, Transit Gateway.
  • Hybrid configuration: Site-to-Site VPN and Direct Connect.

Data Ingestion/Transformation Recall Checklist

  • Ingestion patterns, DataSync, Storage Gateway, AWS Transfer Family basics.
  • Snow Family for massive data transfers — know the use cases it's designed for.
  • Kinesis streaming options for data visualization pipelines.
  • Athena, Lake Formation, QuickSight basics.
  • Glue for transformation; EMR for large-scale distributed processing.

Practical Examples

SQS + Lambda error handling at scale: An order-processing Lambda triggered by SQS fails intermittently due to a downstream API timeout. By default, SQS retries the message per the queue's maxReceiveCount (visibility timeout expiring re-delivers it), and after that count is exceeded, the message moves to a configured Dead-Letter Queue (DLQ) where it can be inspected and reprocessed manually — without a DLQ, failed messages either loop forever or silently vanish.

Lambda quotas biting a real workload: A team fans out image processing across Lambda and starts seeing TooManyRequestsException. That's the concurrency limit (default 1,000 concurrent executions per account per Region) being hit — fixed by requesting a limit increase or by using reserved/provisioned concurrency to guarantee capacity for this function specifically without starving others.

AppSync tying data sources together: A mobile app's "product detail" screen needs data from DynamoDB (product info), a Lambda function (real-time inventory check), and a third-party REST API (reviews) — fetching from three endpoints separately is slow and chatty on mobile networks. AppSync exposes one GraphQL endpoint where the client requests exactly the fields it needs in a single round trip, and AppSync resolves each field from its respective backend.

On-demand vs. provisioned DynamoDB capacity: A brand-new startup launching an MVP has no traffic history and can't predict read/write volume — on-demand capacity avoids both under-provisioning (throttling) and over-provisioning (wasted spend) risk during that uncertain period. Once traffic stabilizes into a predictable daily pattern six months later, switching to provisioned capacity with auto-scaling becomes cheaper for the now-known baseline load.

Read Replica vs. ElastiCache, the actual decision: A migrated legacy RDS MySQL app is read-heavy, but the reads are highly repetitive (the same "get user profile" query fired constantly for the same users) — ElastiCache wins here because it can absorb nearly all of that repeat traffic from memory. If instead the reads were broad and varied (many different, mostly-unique reporting queries), a Read Replica would be the better fit since caching wouldn't get many cache hits anyway.

API Gateway throttling protecting the backend: A public API sees a traffic spike from one misbehaving client hammering it. Usage plans + API keys with per-key throttling limits (e.g., 10 requests/sec, burst 20) stop that one client from degrading service for everyone else, without needing to touch the backend Lambda/EC2 capacity at all.

  • D3_HighPerformingAndScalableStorageSolution_Insights (unresolved)
  • D3_HighPerformingAndElasticComputeSolution_Insights (unresolved)
  • D3_HighPerformingDatabaseSolution_Insights (unresolved)
  • D3_HighPerformingAndScalableNetworkArchitecture_Insights (unresolved)
  • D3_HighPerformingDataIngestionAndTransformationSolution_Insights (unresolved)
Hermes Wiki