Developer/CommunicationPatterns/Protocols/ShortPolling/Fundamentals/short-polling-fundamentals-interval-choice-and-the-thundering-herd

Short Polling Fundamentals: Interval Choice and the Thundering Herd

Concept

Short polling is the simplest way to simulate real-time updates over plain request/response HTTP: the client asks "anything new?" on a fixed timer, the server answers immediately every time — with fresh data if something changed, or a response saying nothing changed if not — and the connection closes either way. No connection is ever held open, unlike long polling (server delays the response until data exists or a timeout fires) or SSE/WebSockets (server pushes over a persistent connection). That makes short polling trivial to implement, trivial to debug (every request/response pair is independent and inspectable in isolation), and trivial to reason about under load, since server-side state per client is zero between requests.

The tradeoff it makes is blunt: latency is bounded by the poll interval (a change is visible up to one full interval late, on average half an interval), and most requests are wasted — the overwhelming majority of polls, for anything that doesn't change every few seconds, return "nothing new." Two levers exist to soften that waste without abandoning the model entirely. First, conditional requests: the server returns an ETag or Last-Modified header with the data, and the client sends it back as If-None-Match/If-Modified-Since on the next poll; if nothing changed, the server responds 304 Not Modified with an empty body instead of re-sending the full payload, cutting bandwidth even though the request count stays the same. Second, adaptive intervals: back off the poll frequency after repeated empty responses (poll every 2s initially, widen to 10s, 30s, 60s if nothing changes) and reset to the fast interval the moment something does — trading a little latency on the first change after a quiet period for a large reduction in wasted requests during quiet periods.

The failure mode that's easy to miss in isolation is aggregate, not per-client: synchronized polling, sometimes called a thundering herd. If many clients start their polling timer at a correlated moment — everyone reconnects after a deploy, everyone's tab reloads after a shared outage, everyone's setInterval was started at page load with no offset — their requests land on the server in synchronized bursts every interval instead of being smoothly spread out, producing a periodic load spike that's invisible if you only ever test with one client locally. The standard fix, borrowed from the same principle AWS's retry-backoff guidance uses for retry storms, is jitter: randomize each client's actual poll interval by some percentage (e.g. base interval ± 20%) so the population's requests spread out over time instead of clustering.

Tradeoffs

Interval strategy	Latency	Server load	Implementation cost
Fixed short interval (e.g. 2s)	Low, bounded tightly	High — most requests wasted on unchanged data	Lowest
Fixed long interval (e.g. 60s)	High, bounded loosely	Low	Lowest
Fixed interval + conditional requests (ETag/304)	Same as interval chosen	Lower bandwidth per request, same request count	Low — standard HTTP feature
Adaptive/backoff interval	Low right after a change, degrades during quiet periods	Lowest sustained load	Moderate — needs state tracking per client
Adaptive interval + jitter	Same as adaptive	Lowest, and smooth rather than spiky	Moderate

Short polling's ceiling, even fully optimized with conditional requests and adaptive jitter, is still request-per-check overhead that long polling, SSE, and WebSockets don't pay — those get the server to speak only when there's something to say. Short polling never closes that gap; it only manages it.

When to use / when not to

Use for low-stakes, low-frequency status checks where a few seconds to a minute of staleness is genuinely fine and implementation simplicity outweighs efficiency — checking whether an async job finished, whether a booking confirmation has posted, whether a background export is ready to download.
Use as the fallback of last resort when neither long polling nor SSE/WebSockets are available — environments where the server can't hold connections open at all (some serverless/FaaS backends bill and time-limit by wall-clock execution, making a held-open long-poll or SSE stream expensive or outright unsupported).
Don't use for anything latency-sensitive or high-frequency — chat, live collaboration, real-time dashboards. The wasted-request cost compounds badly at high polling frequency, and the achievable latency floor is still bounded by the interval, unlike push-based approaches.
Don't add short polling to a client population without adding jitter once that population might grow past a handful of clients — the thundering-herd risk is invisible during development with one browser tab and becomes a real, periodic load spike only once enough clients exist to synchronize.

Common pitfall

Choosing a poll interval by guessing at "reasonable" rather than by how frequently the underlying data actually changes, and then never revisiting it as usage scales. A 2-second interval that was fine for ten internal users during a beta becomes a real cost line once ten thousand users share the same polling behavior — most of those requests were always wasted, but the wasted fraction only becomes a load or billing problem at volume. The fix isn't just "poll less" — it's tying the interval (and whether to use conditional requests) to the actual rate of change in the underlying data, and re-checking that assumption as usage grows rather than treating the original interval as permanent.

Engineering Lens

Short polling's real cost rarely shows up in a single client's request-rate metrics — it shows up in the aggregate load curve once many independently-timed clients happen to correlate, which is exactly the failure mode that's invisible when testing with one client locally and only visible in production traffic graphs after the fact. The engineering judgment worth applying isn't "avoid short polling" — it's cheap and appropriate for plenty of low-stakes checks — but rather building in jitter and conditional requests from the start, before the client population is large enough for synchronized bursts to matter, rather than discovering the thundering-herd pattern from an incident.