Developer/Compute/Serverless/Fundamentals/serverless-cold-starts-and-mitigation

Serverless Cold Starts and Mitigation

Concept

A FaaS platform (AWS Lambda, and equivalents on other clouds) doesn't keep a function's execution environment running between invocations by default — it's the whole premise of "no idle cost." When a request arrives and no warm environment is available to handle it, the platform has to provision one from scratch: pull the deployment package, start the runtime (JVM, Node, Python interpreter), run the function's static initialization code (imports, client construction, connection setup), and only then execute the actual handler. That provisioning work is the cold start, and it's added latency the caller pays on top of the function's real work — anywhere from under 200ms for a lightweight Node/Python function with minimal dependencies, to multiple seconds for a JVM-based function with a heavy dependency graph (Spring-style frameworks are the canonical worst case). A warm start — a request landing on an environment the platform kept alive from a recent previous invocation — skips all of that and pays only the handler's actual execution time.

What actually drives cold-start duration is mostly under the function author's control: runtime choice (interpreted languages like Node/Python cold-start faster than a JVM doing full class loading and JIT warmup), deployment package size (a smaller package means less to pull and unpack), how much work happens in top-level initialization code versus inside the handler, and whether the function needs a VPC attachment (historically added meaningful cold-start latency for ENI provisioning, though this has substantially improved on modern Lambda). Memory allocation also matters indirectly — Lambda allocates CPU proportionally to configured memory, so a higher memory setting can shrink both cold-start and steady-state execution time even when the function doesn't need the extra RAM itself.

Two purpose-built mitigations exist for cases where the above tuning still isn't enough:

Provisioned concurrency keeps a specified number of execution environments permanently initialized and idle, ready to serve a request with double-digit-millisecond latency — at the cost of paying for that idle capacity continuously (roughly $0.015/GB-hour), regardless of whether it's actually handling traffic.
SnapStart (available for Java, and now Python/.NET runtimes on modern Lambda) takes a memory-and-disk snapshot of a fully-initialized execution environment after first init, and restores new environments from that snapshot instead of re-running initialization from scratch — reducing P99 cold starts dramatically for the runtimes it supports, at no extra charge, but landing at sub-second (not double-digit-millisecond) latency, and the two features are mutually exclusive on the same function version.

Tradeoffs

Mitigation	Latency	Cost	Constraints
Do nothing (accept cold starts)	100ms-3s+ on cold invocations, near-zero on warm	No extra cost	Fine for latency-tolerant or infrequent workloads
Runtime/package tuning (lighter runtime, smaller package, less top-level init work)	Meaningfully reduces cold-start duration, doesn't eliminate it	No extra runtime cost, some engineering effort	Doesn't fully close the gap for large/complex functions
Provisioned concurrency	Double-digit-millisecond, consistent	Pays for idle capacity 24/7 whether traffic arrives or not	Works with every runtime; easy to over-provision and overspend
SnapStart	Sub-second, better than unmitigated cold start but not as fast as provisioned concurrency	Free	Limited runtime support (Java 11/17/21, Python 3.12+, .NET 8); mutually exclusive with provisioned concurrency on the same version

The organizational failure mode worth naming explicitly: teams reach for provisioned concurrency because it gives the best latency number, then leave it running at a size sized for peak traffic long after the reason for sizing it that way has passed — turning what should be a bounded, deliberate cost into a standing bill nobody revisits. The rule of thumb from practitioners is blunt: it is not uncommon to see organizations spend hundreds of dollars a month on provisioned concurrency solving what is, measured honestly, a much smaller latency problem.

When to use / when not to

Accept unmitigated cold starts by default for anything latency-tolerant or infrequently invoked (background jobs, internal tooling, low-traffic endpoints) — the added engineering and cost of mitigation isn't worth it when nothing downstream is sensitive to an occasional multi-hundred-millisecond spike.
Reach for SnapStart first when the runtime supports it and the workload is cost-sensitive with sporadic traffic — it's free and closes most of the gap.
Reach for provisioned concurrency specifically when a synchronous, user-facing path has a hard latency SLA that sub-second SnapStart-level performance still doesn't meet, and traffic is predictable enough to size it without gross over-provisioning.
Don't skip the cheap wins (trim the deployment package, move client construction and imports that don't depend on request data to module-level so they run once per environment rather than being needlessly re-verified, drop unused dependencies) before reaching for either paid mitigation — they cost nothing but engineering time and often close a meaningful fraction of the gap on their own.

Common pitfall

Sizing provisioned concurrency once, at launch, for expected peak traffic, and never revisiting it as real traffic patterns diverge from that estimate — either under-provisioned (so peak traffic still hits unmitigated cold starts because the estimate was too low) or, far more commonly, over-provisioned (so a system that's been redesigned, deprecated, or simply never grew into the traffic level it was sized for keeps paying full idle-capacity cost indefinitely). The fix is treating provisioned concurrency the same way you'd treat any other capacity-planning number — reviewed against actual traffic on a schedule, with autoscaling for provisioned concurrency itself (Lambda supports this) rather than a fixed number set once and forgotten.

Engineering Lens

Cold starts are a direct, visible tax on the core serverless promise — "pay only for what you use, nothing sits idle" — because avoiding the tax means paying for idle capacity again, just under a different name (provisioned concurrency). The judgment call that actually matters in a design review isn't "should we use provisioned concurrency" in the abstract, it's naming the specific latency SLA that a synchronous user-facing path actually has, and showing the mitigation choice (none, SnapStart, provisioned concurrency, or a non-serverless execution model entirely) was sized to that number rather than chosen by default or by whichever option looked best in a benchmark. The severity framing matters too: cold starts have genuinely moved, for most workloads, from "architecture-breaking" a few years ago to "rounding error" today — the interesting engineering decision is recognizing which workloads are the exception to that, not treating every Lambda function as if it needs mitigation by default.