Developer/SchedulingQueueingDisciplines/WeightedRoundRobin-LeastConnections/Fundamentals/weighted-round-robin-vs-least-connections

Weighted Round Robin and Least Connections

Concept

Plain round robin assumes every server behind a load balancer is interchangeable — same capacity, same request cost — and cycles through them equally. Two refinements exist for the moment that assumption breaks, and they fix different halves of it.

Weighted round robin keeps round robin's cyclic, arrival-order-based dispatch but assigns each server a weight reflecting its relative capacity. If server A has weight 3 and server B has weight 1, the load balancer sends three requests to A for every one it sends to B — a fixed, precomputed ratio, not something that reacts to what's actually happening on either server. This fixes heterogeneous capacity: a mix of larger and smaller instances, or a gradual canary rollout where a new server's weight is dialed up over time. It does not fix uneven request cost — if requests routed to A happen to be far more expensive than the ones routed to B, weighted round robin has no way to know or react, because it never looks at actual server load.

Least connections abandons the cyclic assumption entirely and routes each new request to whichever server currently has the fewest in-flight (open) connections. This is a live, load-reactive decision rather than a precomputed schedule — it's the same choice a human operator would make with a real-time dashboard in front of them: send the next request to whoever looks least busy right now. That makes it adapt automatically to uneven request cost and uneven capacity alike, without anyone having to keep weight numbers manually accurate as capacity changes. Its own soft spot is connection duration: "fewest current connections" is a decent proxy for "least busy" only when connections are roughly similar-length; a server holding a handful of very long-lived connections (websockets, long-polling, streaming responses) can look artificially "free" by connection count while actually being saturated in the metric that matters (CPU, memory, bandwidth).

Tradeoffs

Algorithm	Adapts to uneven capacity	Adapts to uneven/live request cost	Operational cost	Weak point
Plain round robin	No — equal share regardless	No — blind to load	None — no config to maintain	Wrong the moment servers or requests aren't uniform
Weighted round robin	Yes — via manually/automatically set weights	No — still a precomputed schedule	Low, but weights need to stay accurate as capacity changes	Weights drift stale; doesn't react to transient load spikes
Least connections	Implicitly — busy servers naturally get fewer new requests	Yes — reacts to real in-flight load	None — no weights to maintain	Long-lived/streaming connections distort the "fewest connections" signal
Least connections + weights (hybrid, supported by nginx/HAProxy)	Yes, explicitly	Yes, reactively	Low	Same long-connection distortion, now combined with weight tuning

Neither pure algorithm is a strict improvement over the other — weighted round robin is simpler to reason about and cheaper to compute, but requires someone (or an autoscaler) to keep weights honest; least connections is self-adjusting but needs connection-duration uniformity to be a trustworthy signal.

When to use / when not to

Use plain round robin as the default for a fleet of identical-capacity instances handling similarly-sized, similarly-short requests — it's the simplest option and there's nothing uneven for a smarter algorithm to correct for.
Use weighted round robin when capacity is known and relatively stable — a deliberate mix of instance sizes, or a canary/blue-green rollout where you want an explicit, auditable traffic ratio rather than a load-reactive one.
Use least connections once request cost or duration varies meaningfully and you want the system to adapt automatically rather than requiring someone to keep weights accurate — it's usually the safer default the moment instances or requests stop being uniform.
Avoid least connections un-modified for workloads dominated by long-lived connections (websockets, SSE, streaming) — the connection count stops correlating with actual load; either combine it with weights or use a load-balancing signal based on actual resource utilization instead.

Common pitfall

Setting weights once during initial capacity planning and never revisiting them — weighted round robin's whole value depends on the weights staying an accurate reflection of relative capacity, and infrastructure changes (an instance resized, a new instance type mixed in, one server sharing its box with another workload) silently invalidate that assumption without producing an obvious error. The system doesn't fail loudly; it just quietly over-sends traffic to whichever server the stale weight favors, showing up later as one instance running hotter than its peers for no apparent reason.

Engineering Lens

The design-review question is which signal the algorithm is actually reacting to: a precomputed ratio (weighted round robin) or live in-flight load (least connections). Neither is universally "better" — the real judgment call is whether your traffic pattern has stable, known capacity differences (favoring weights) or unpredictable, load-reactive ones (favoring least connections), and whether long-lived connections in your workload would distort the least-connections signal enough to need a different metric (e.g. weighted least-connections, or a utilization-based algorithm) instead.