Developer/RateLimiting/LeakyBucket/Fundamentals/leaky-bucket-algorithm

Leaky Bucket: Traffic Shaping vs. Rate Limiting

Concept

Leaky bucket is the inverse framing of token bucket, and the difference isn't cosmetic — it changes what the algorithm is actually good for. Requests (or packets, in its original networking context) arrive at whatever rate the client sends them and queue into a bucket of fixed capacity; the bucket drains — "leaks" — at a constant, fixed rate regardless of how bursty the input was. If the bucket is full when a new request arrives, that request is dropped (or, in a rate-limiter framing, rejected). See Rate Limiting Algorithms for where it sits relative to the other three algorithms; this note goes deeper into the two implementations that get conflated under the same name, and why leaky bucket is the wrong default for API rate limiting even though it's a textbook rate-limiting algorithm.

Two implementations, same name, different behavior:

As a queue (the original networking framing) — requests are buffered in an actual FIFO queue and processed/forwarded at the fixed leak rate. This shapes traffic: the output rate to whatever's downstream is perfectly smooth no matter how bursty the input was, at the cost of added latency for anything queued behind an already-full bucket.
As a meter (the common API rate-limiter framing, sometimes called "leaky bucket as a meter") — no actual queue exists; the bucket is just a counter that increments per request and decrements at the fixed leak rate, and a request is rejected outright if the counter is already at capacity rather than being queued and delayed. This drops the traffic-shaping property (nothing is smoothed, since rejected requests just fail immediately) but keeps the "no bursts get through" enforcement — functionally, this version behaves like a token bucket with the refill and consumption logic mirrored, and the two are often implemented with near-identical code.

The traffic-shaping (queue) version is what leaky bucket is actually for: protecting a fixed-capacity downstream resource — a legacy system, a hardware device, a queue consumer with a hard throughput ceiling — from a bursty upstream producer, by converting bursty arrivals into a perfectly steady departure rate. That's a fundamentally different goal from "let this client through at a fair rate," which is the problem token bucket and the window-based algorithms are solving.

Tradeoffs

Approach	Burst handling	Latency impact	Best fit
Leaky bucket (queue/shaping)	Smooths bursts into a constant output rate	Adds queuing delay to bursty traffic	Protecting a fixed-capacity downstream from a bursty producer
Leaky bucket (meter)	Rejects bursts outright, no smoothing	None (fail-fast, same as any counter-based limiter)	Rare — usually token bucket is a better fit for the same behavior with clearer semantics
Token bucket	Allows bursts up to bucket size, then throttles to average	None (accept or reject at request time)	Client-facing API fairness — rewards idle-then-burst clients instead of punishing them

The real tradeoff is about what happens to a burst, not just whether it's allowed: leaky bucket (queue form) delays it, leaky bucket (meter form) and every window-counter variant reject it, and token bucket is the only one of the four that lets it through un-delayed. Which of those three outcomes is correct depends entirely on whether the caller can tolerate added latency (queue), needs a hard boundary (meter/window), or should be rewarded for having been idle (token bucket).

When to use / when not to

Use the queue form specifically when the goal is protecting a fixed-throughput downstream consumer — smoothing a bursty producer (e.g., a webhook fan-in, an IoT device fleet reporting on no fixed schedule) before it hits a queue consumer, database write path, or legacy system with a hard per-second ceiling it cannot exceed without degrading.
Don't reach for leaky bucket as a public API rate limiter's primary algorithm — it can't express "allow occasional bursts from well-behaved idle clients," which is exactly the behavior most API consumers expect and which token bucket handles natively. Using leaky bucket here trades away burst tolerance for no real benefit over a window-based counter.
Use it in its original context — network/packet-level traffic shaping (QoS policies, egress bandwidth shaping) — where smoothing to a constant output rate genuinely is the goal, not an approximation of client fairness.
If queuing delay itself is unacceptable for the use case (a synchronous request-response API where a caller is blocked waiting), the queue form of leaky bucket is actively harmful — it converts "rejected immediately" into "kept waiting, then possibly still rejected," which is a worse experience for most API callers than a fast, clear rejection.

Common pitfall

Reaching for leaky bucket because it's one of the "big four" rate-limiting algorithms everyone lists together, without noticing it solves a different problem than the other three. Fixed window, sliding window, and token bucket are all answering "is this client within its fair allotment right now" — leaky bucket (queue form) is answering "how do I convert this bursty input into a steady output," which is traffic shaping, not client-level fairness enforcement. Picking it for a public API rate limiter usually means either silently degrading to the meter form (which just re-implements token/window-counter behavior with extra complexity) or introducing unwanted queuing latency that a caller never asked for and has no way to detect until requests start timing out upstream of the queue.

Engineering Lens

The leaky-bucket-vs-token-bucket confusion is a good instance of a broader design-review smell: two algorithms with a similar mental model (a bucket with a rate) can be solving genuinely different problems, and picking based on the metaphor's familiarity rather than the actual constraint (fairness vs. shaping, reject vs. delay) produces a system that technically "has rate limiting" but doesn't behave the way anyone downstream actually needs. The stronger review question isn't "which bucket algorithm did you pick" — it's "what should happen to a request that would exceed the limit: reject it now, or delay it until capacity frees up," because that answer alone determines whether leaky bucket (shaping) or token bucket (fairness) is even in the right family for the problem.