Publish/Subscribe Messaging: Topics, Fan-Out, and Delivery Guarantees
Concept
Publish/subscribe (pub/sub) is a messaging pattern where publishers send messages to a named channel — a topic — without any knowledge of who, if anyone, is listening, and subscribers independently register interest in a topic without knowing who publishes to it. This is the structural difference from a point-to-point queue: a queue's contract is "exactly one consumer gets each message," while pub/sub's contract is "every current subscriber gets its own copy of each message" (fan-out). Enterprise Integration Patterns formalizes this as the Publish-Subscribe Channel — one input channel that splits into multiple output channels, one per subscriber, with the channel itself responsible for delivering a copy of each incoming message to every output.
The decoupling is the entire value proposition: a publisher emitting an OrderPlaced event has no compile-time or runtime dependency on the number or identity of consumers reacting to it — a warehouse system, a notification service, and an analytics pipeline can all subscribe independently, and adding a fourth subscriber (say, a fraud-detection pipeline) requires zero changes to the publisher or to any existing subscriber. This is what makes pub/sub the default choice for event-driven architectures: the publish side stays a single, stable integration point even as the number of interested systems grows over the system's life.
Most managed pub/sub systems (Google Cloud Pub/Sub, AWS SNS, Kafka consumer groups) default to at-least-once delivery, not exactly-once: a subscriber has a bounded acknowledgment window (an "ack deadline") to confirm receipt, and if that window expires — because the subscriber crashed, was slow, or the ack itself was lost in transit — the message is redelivered. This means every subscriber has to be written as an idempotent consumer, one that produces the same end state whether it processes a given message once or five times, because the delivery contract explicitly allows duplicates. Ordering is a related but separate guarantee: by default, messages carry no ordering guarantee at all across a topic, and even where a system supports it — Cloud Pub/Sub's ordering keys guarantee in-order delivery to a subscriber, but only for messages sharing the same key — redelivering one message generally means every subsequent message for that same key is redelivered too, in order, rather than the stream skipping ahead. Ordering and at-least-once delivery interact; they aren't independent knobs a subscriber can reason about separately.
Tradeoffs
| Approach | Benefit | Cost |
|---|---|---|
| Point-to-point queue | Simple "exactly one consumer" semantics; natural work distribution across a pool of workers | Adding a second, independent consumer of the same message requires structural change — a second queue, explicit fan-out logic |
| Pub/sub (topic-based) | New subscribers added with zero change to the publisher or existing subscribers; natural fit for "many systems care about this event" | Broker/topic infrastructure to operate; at-least-once delivery pushes idempotency and dedup responsibility onto every subscriber |
| Direct synchronous call (no broker) | Simplest possible integration for exactly one caller and one callee | Publisher and consumer are coupled in both time (both must be up simultaneously) and code (publisher must know every consumer to call); adding a consumer means changing the publisher |
When to use / when not to
- Use when multiple, independent parts of a system need to react to the same event without the source of that event knowing or caring who's listening — the canonical case is one business event (order placed, user signed up) triggering several unrelated downstream actions.
- Especially valuable when the set of subscribers is expected to grow over the system's life — pub/sub is what keeps that growth from becoming a repeated change to the publisher.
- Don't reach for pub/sub when there's genuinely one consumer and that's expected to stay true — a plain point-to-point queue (or even a direct call) is simpler to operate and reason about, and pub/sub's fan-out infrastructure buys nothing if there's never a second subscriber.
- Skip it, or pair it carefully with an outbox/dedup layer, when strict exactly-once processing is a hard business requirement (e.g. charging a payment method exactly once) — the delivery contract itself doesn't provide that; the consumer has to build it.
Common pitfall
Writing subscriber logic that assumes each message arrives exactly once. At-least-once delivery is the default across essentially every production pub/sub system, so a subscriber that isn't idempotent — one that increments a counter or sends an email on every message received, rather than checking "have I already handled this message ID" — will double-process during the redeliveries that are a normal, expected part of operation, not a rare failure case to shrug off.
Engineering Lens
The design review question that actually matters for a pub/sub-based integration isn't "which broker" — it's "what does this specific subscriber do if it receives the same message twice," because the answer to that question is the real correctness boundary of the system, not the topic/subscription wiring. A team that can point to an idempotency key (a message ID, a natural business key) and show exactly how the consumer uses it to no-op on a duplicate has actually designed for the delivery contract; a team that hasn't is quietly relying on the broker never redelivering, which it explicitly does not promise.
Sources
- Publish-Subscribe Channel — Enterprise Integration Patterns
- Subscription overview — Pub/Sub, Google Cloud Documentation
- Publisher-Subscriber pattern — Azure Architecture Center, Microsoft Learn