Developer/CommunicationPatterns/Protocols/WebSockets/CaseStudies/slack-real-time-messaging-channel-and-gateway-servers

Slack: Channel Servers and Gateway Servers for Real-Time Message Fan-Out

Problem + constraints

Slack's core product promise — a message posted in a channel shows up for every other member almost instantly — is a fan-out problem at a scale a naive "write to a database, have clients re-fetch" design can't meet. A message to a busy channel has to reach every currently-connected member in a few hundred milliseconds, at a scale of tens of millions of concurrently connected clients, with peak load reaching roughly 16 million channels served per host. The constraint isn't raw message throughput — it's connection state: every connected client needs a live, addressable channel (in the networking sense) back to it so the server can push, and that connection state has to be tracked, load-balanced, and kept consistent across a fleet of servers without funneling every message through a single bottleneck.

Solution

Slack built the real-time layer as a small set of specialized, stateful Java services rather than bolting push onto a generic stateless API tier. Channel Servers (CS) hold channel state and recent history in memory, each responsible for a subset of channels assigned by consistent hashing, so a given channel's fan-out logic always lives on one predictable server rather than being scattered across the fleet. Gateway Servers (GS) sit between clients and Channel Servers: each GS holds the live WebSocket connections for a set of users along with which channels those users are subscribed to, and is what a Channel Server actually pushes a new message through to reach a specific connected client. Splitting "who owns this channel's data" (CS) from "who owns this client's live connection" (GS) means one incoming message only has to reach the single CS that owns its channel, which then fans it out to exactly the GS instances holding sockets for members currently online — no broadcast to servers with no interested clients attached. Presence Servers separately track online/offline/away state, since presence tolerates staleness in a way message delivery can't, and doesn't need to sit in the same hot path.

The consistent-hashing assignment of channels to Channel Servers is what keeps this from becoming a distributed coordination problem on every single message: because the owning server for a channel is deterministic, a Gateway Server always knows which Channel Server to talk to for a given channel, with no discovery step needed before fan-out can start. The measured result of this design, per Slack's own account, is key-press-to-receipt latency under 500 milliseconds under normal load.

What to steal

Separate "who owns this piece of data" from "who owns this client's connection" as two different sharding problems. Slack's CS/GS split means a channel's fan-out logic and a user's live socket are independently placed and independently scaled — a hot channel doesn't force every server holding a socket for one of its members to also hold that channel's full state.
Consistent hashing on data ownership removes a coordination step from the hot path. Deterministic channel-to-server assignment means fan-out doesn't need a lookup or discovery phase before it can start pushing — the owning server is knowable in advance, not resolved per message.
Keep low-consistency-requirement state out of the high-consistency-requirement hot path. Presence tolerates staleness that message delivery can't, and Slack's architecture reflects that by giving it its own dedicated tier instead of folding it into the message fan-out path.
Sub-second delivery at tens of millions of live connections doesn't require a fully stateless architecture. A small number of purpose-built stateful services, sharded correctly, gets there — the state is exactly what makes the fan-out fast, not something to be engineered away in favor of statelessness by default.

Engineering Lens

The generalizable idea here isn't "use WebSockets for real-time" — that's the transport choice, and it's almost incidental. The real design decision is architecting fan-out around two orthogonal ownership questions — which server owns this channel's state, which server owns this client's live connection — and picking a sharding strategy, consistent hashing, that makes the first question answerable without a lookup. Any system with a real-time push requirement at scale — chat, live dashboards, collaborative-editing presence layers — runs into the same fan-out shape, and Slack's architecture is a concrete existence proof that a small number of specialized stateful tiers can hit sub-second delivery at tens of millions of connections without needing a fully stateless request model.

Sources