Compute Pricing Models
Concept
Cloud compute is sold under fundamentally different pricing contracts, and choosing among them is an architecture decision, not a procurement afterthought — the pricing model has to match the workload's actual availability and predictability profile. AWS's EC2 offers this as a clean illustration (the same shape exists on every major cloud):
- On-Demand — pay per second/minute with no commitment, no upfront cost. Highest per-hour price, in exchange for total flexibility to start/stop anytime.
- Reserved Instances (or Savings Plans) — commit to a given instance type/family in a region for a 1- or 3-year term, in exchange for up to ~72% off On-Demand pricing. Standard RIs are locked to a specific configuration; Convertible RIs trade some discount depth for the ability to swap instance family/size mid-term.
- Spot Instances — bid for the provider's spare, unused capacity at up to ~90% off On-Demand, with the tradeoff that the provider can reclaim that capacity with as little as a two-minute warning when On-Demand/Reserved demand rises.
Tradeoffs
| Model | Discount vs. On-Demand | Commitment | Interruption risk | Best fit |
|---|---|---|---|---|
| On-Demand | None (baseline) | None | None | Unpredictable, spiky, or short-lived workloads |
| Reserved / Savings Plans | Up to ~72% | 1–3 years | None (you keep the capacity) | Steady-state, predictable baseline load |
| Spot | Up to ~90% | None (but revocable) | High — reclaimed with ~2 min notice | Fault-tolerant, restartable, stateless batch/async work |
The underlying tension is the same shape as every cost pillar decision: you're trading a commitment (in money, in flexibility, or in fault-tolerance engineering effort) for a discount. The discount isn't free — Reserved trades flexibility, Spot trades availability guarantees, and both require the workload to actually fit the constraint or the "savings" become an outage.
When to use / when not to
- On-Demand fits variable or unpredictable traffic where the cost of guessing wrong on a commitment (overcommitting to Reserved capacity you don't use) outweighs the discount.
- Reserved/Savings Plans earn their cost only against a workload with a genuinely predictable steady-state floor — sizing the commitment to peak rather than floor traffic turns the "discount" into wasted spend on idle reserved capacity.
- Spot fits workloads that can tolerate being killed mid-execution and restarted elsewhere: batch processing, CI/CD runners, stateless horizontally-scaled workers, machine learning training jobs with checkpointing. It is a poor fit for anything stateful without its own replication, or anything on a latency-sensitive user-facing hot path where an unscheduled 2-minute-notice termination is user-visible.
- Most mature production environments blend all three: Reserved/Savings Plans for the steady-state floor, On-Demand for the variable portion above that floor, and Spot for anything fault-tolerant and batch-shaped — a pattern that typically nets 35–50% blended savings without overcommitting to a fixed-capacity bet.
Common pitfall
Reserving capacity sized to a workload's peak, not its floor — teams often reserve enough to cover the busiest expected day, which leaves expensive committed capacity sitting idle the rest of the time and erases most of the intended savings. The correct sizing question is "what's the minimum load this system runs at 24/7, 365 days a year" — everything above that floor belongs on On-Demand or Spot, not Reserved.
Principal Engineer Lens
This is fundamentally a cost-pillar exercise in matching commitment shape to demand shape — the same reasoning a capacity planner applies to physical infrastructure, just compressed into a purchasing decision made in a console. The Principal-level tell in an architecture review is naming why a given workload sits in a specific pricing tier (fault-tolerance profile, predictability, statefulness) rather than defaulting everything to On-Demand out of inertia or over-committing to Reserved out of a vague "let's save money" directive without checking the workload actually has a stable floor.
Reel Script
Setup: Ask: if your team's AWS bill is dominated by On-Demand pricing on servers that have been running nonstop for two years, is that actually flexibility you're paying for, or just money left on the table?
Concept walkthrough: Walk through the three models — On-Demand (pay-as-you-go, full flexibility, most expensive), Reserved (commit for 1-3 years, up to ~72% off, but locked in), Spot (bid for spare capacity, up to ~90% off, but can be reclaimed with 2 minutes' notice). Frame each as trading something specific — money, flexibility, or availability guarantee — for a discount.
Real example tie-in: Walk through a real mixed fleet: the baseline web servers that never scale below a known floor go Reserved; the autoscaling layer that handles traffic spikes stays On-Demand; the nightly batch/ETL jobs that can checkpoint and resume go on Spot, since a 2-minute-notice termination costs nothing but a restart.
Tradeoffs & alternatives: Name the classic mistake explicitly — reserving to peak instead of floor, which leaves expensive idle capacity sitting around. Contrast with sizing to floor and using On-Demand/Spot to cover everything above it.
Principal Engineer takeaway: The strong answer in a review isn't "we use Reserved Instances to save money" — it's being able to point at a specific workload's statefulness and predictability profile and explain why that specific pricing tier is the right fit, not a default.
Related
Sources: