Publish-Subscribe Pattern: Decoupling Producers from Consumers
Generalize Observer with an event broker in the middle — publishers and subscribers never know about each other, only about topics.
Publish-Subscribe Pattern
Publish-Subscribe (Pub-Sub) decouples message producers from consumers through an intermediary — a broker — organized around named channels called topics. Publishers publish to a topic without knowing who (if anyone) is listening. Subscribers subscribe to a topic without knowing who publishes to it. Neither side holds a reference to the other; the broker is the only thing both sides know about.
This is Observer generalized: Observer's subject is the notification mechanism; Pub-Sub extracts that mechanism into its own first-class object.
1. The Problem: Direct Coupling Doesn't Scale Past One Process (or One Producer)
Plain Observer requires the subject to hold direct references to observers and to know their interface. That breaks down once producers and consumers are independently deployed, or once multiple, unrelated producers need to feed the same consumers.
// VIOLATION: OrderService knows about every consumer directly — Observer-style,
// but now the "subject" is a business service that shouldn't own delivery logic
class OrderService {
private InventoryService inventoryService;
private EmailService emailService;
private AnalyticsService analyticsService;
// Adding a new consumer (e.g. FraudDetectionService) means OrderService
// must be changed, redeployed, and re-tested — even though OrderService's
// actual job (placing orders) hasn't changed at all.
void placeOrder(Order order) {
// save order...
inventoryService.reserve(order);
emailService.sendConfirmation(order);
analyticsService.record(order);
}
}OrderService has become a routing hub for every downstream consumer that ever needs to know about an order — a responsibility it should never have had. Worse, if InventoryService and AnalyticsService are separate deployable services, this direct-call model doesn't even compile across process boundaries.
2. Structure
The publisher only ever talks to Broker.publish(topic, message). It never imports, references, or knows the existence of InventorySubscriber or EmailSubscriber — that's the structural difference from Observer, where the subject directly manages its observer list.
3. Full Implementation
// A message on the wire — payload + minimal metadata
record Message(String topic, Object payload, long timestamp) {
Message(String topic, Object payload) {
this(topic, payload, System.currentTimeMillis());
}
}
interface Subscriber {
void onMessage(Message message);
}
interface Broker {
void publish(String topic, Object payload);
void subscribe(String topic, Subscriber subscriber);
void unsubscribe(String topic, Subscriber subscriber);
}
// A simple in-process broker. Production brokers (Kafka, RabbitMQ, SNS/SQS)
// implement the same conceptual contract with persistence, delivery guarantees,
// and cross-process transport added.
class InMemoryBroker implements Broker {
private final Map<String, List<Subscriber>> topics = new ConcurrentHashMap<>();
@Override
public void subscribe(String topic, Subscriber subscriber) {
topics.computeIfAbsent(topic, t -> new CopyOnWriteArrayList<>()).add(subscriber);
}
@Override
public void unsubscribe(String topic, Subscriber subscriber) {
List<Subscriber> subs = topics.get(topic);
if (subs != null) subs.remove(subscriber);
}
@Override
public void publish(String topic, Object payload) {
Message message = new Message(topic, payload);
List<Subscriber> subs = topics.getOrDefault(topic, List.of());
for (Subscriber subscriber : subs) {
// isolate one bad subscriber from breaking delivery to the rest
try {
subscriber.onMessage(message);
} catch (Exception e) {
System.err.println("Subscriber failed on topic " + topic + ": " + e.getMessage());
}
}
}
}
// Publisher: knows only the broker and a topic name — zero knowledge of subscribers
class OrderService {
private final Broker broker;
OrderService(Broker broker) { this.broker = broker; }
void placeOrder(Order order) {
// save order...
broker.publish("order.placed", order);
}
}
// Subscribers: know only the broker and a topic name — zero knowledge of the publisher
class InventorySubscriber implements Subscriber {
@Override
public void onMessage(Message message) {
Order order = (Order) message.payload();
System.out.println("[Inventory] Reserving stock for order " + order.id());
}
}
class EmailSubscriber implements Subscriber {
@Override
public void onMessage(Message message) {
Order order = (Order) message.payload();
System.out.println("[Email] Sending confirmation for order " + order.id());
}
}
record Order(String id) {}
class Demo {
public static void main(String[] args) {
Broker broker = new InMemoryBroker();
broker.subscribe("order.placed", new InventorySubscriber());
broker.subscribe("order.placed", new EmailSubscriber());
// Adding FraudDetectionSubscriber later touches ZERO existing code —
// just one new subscribe() call, and OrderService is never redeployed.
OrderService orderService = new OrderService(broker);
orderService.placeOrder(new Order("ORD-1001"));
}
}4. Topic Hierarchies and Filtering
Real brokers rarely stop at exact-match topic names. Two extensions worth knowing:
// Wildcard topic matching: "orders.*" matches "orders.placed", "orders.cancelled", etc.
class WildcardBroker implements Broker {
private final Map<String, List<Subscriber>> patterns = new ConcurrentHashMap<>();
@Override
public void subscribe(String topicPattern, Subscriber subscriber) {
patterns.computeIfAbsent(topicPattern, t -> new CopyOnWriteArrayList<>()).add(subscriber);
}
@Override
public void unsubscribe(String topicPattern, Subscriber subscriber) {
List<Subscriber> subs = patterns.get(topicPattern);
if (subs != null) subs.remove(subscriber);
}
@Override
public void publish(String topic, Object payload) {
Message message = new Message(topic, payload);
for (var entry : patterns.entrySet()) {
if (matches(entry.getKey(), topic)) {
entry.getValue().forEach(s -> s.onMessage(message));
}
}
}
private boolean matches(String pattern, String topic) {
String regex = pattern.replace(".", "\\.").replace("*", ".*");
return topic.matches(regex);
}
}Attribute-based filtering (subscribe to order.placed but only where amount > 1000) is the same idea applied to the message payload rather than the topic string — the subscriber (or the broker, in smarter implementations) checks a predicate before delivering.
5. When to Use vs. When It's Overkill
| Use Pub-Sub when | Skip it when |
|---|---|
| Multiple, independently deployed producers and consumers need to communicate | Everything lives in one process and one team owns both ends |
| You need to add new consumers without redeploying producers (or vice versa) | The set of consumers is small, fixed, and known at compile time — plain Observer suffices |
| Cross-service or cross-process delivery, potentially with persistence/retry | In-process notification where a direct method call or Observer is simpler and faster |
| You need delivery guarantees (at-least-once), replay, or fan-out at scale | Sub-millisecond latency matters more than decoupling — broker hop adds overhead |
Over-applying Pub-Sub: introducing a message broker for two services owned by the same team, deployed together, that will only ever have exactly one producer and one consumer, trades a simple method call for network hops, serialization, and a new failure mode (broker down = no communication) — with no decoupling benefit actually realized.
6. Pub-Sub vs. Observer
| Observer | Pub-Sub | |
|---|---|---|
| Who knows whom | Subject holds direct references to its observers | Publisher and subscriber know only the broker, not each other |
| Typical scope | In-process, same address space, synchronous | Often cross-process/cross-service, can be async |
| Adding a new consumer | Requires the subject to accept a new attach() call, still in the same codebase | Requires only a new subscribe() call — publisher's codebase is untouched entirely |
| Failure isolation | An observer exception can propagate into the subject's notify loop unless the subject guards it | Broker typically isolates publisher and subscribers from each other's failures |
| Delivery guarantees | None built in — it's a direct synchronous call | Production brokers offer at-least-once/at-most-once, persistence, retry, DLQ |
See Observer Pattern for the direct, broker-less version of this same one-to-many notification problem, including the push-vs-pull tradeoff and the classic listener-leak bug.
7. Reactive Programming as an Evolution
Reactive streams (RxJava's Observable, Project Reactor's Flux/Mono) take the Pub-Sub/Observer shape and add: composable operators (map, filter, debounce, merge), explicit backpressure (a slow subscriber can signal "send less, slower" instead of being flooded), and a formalized completion/error channel alongside the data channel. Conceptually it is still "producer emits, subscriber reacts" — reactive programming is what Pub-Sub looks like once you standardize the subscriber contract (onNext, onError, onComplete) and make composition a first-class operation.
8. Real-World / Production Examples
- Kafka — topics with partitions, publishers (producers) and subscribers (consumer groups) fully decoupled, with persistence and replay.
- RabbitMQ / AMQP — exchanges (the broker) route messages to queues based on routing keys — a more elaborate topic-matching scheme.
- AWS SNS + SQS — SNS is the pub-sub fan-out layer; each subscriber gets its own SQS queue, decoupling delivery pace per consumer.
- Redis Pub/Sub — the simplest possible broker:
PUBLISH channel message/SUBSCRIBE channel, no persistence, fire-and-forget. - Webhooks — a GitHub repo "publishes" push events; any registered webhook URL is a subscriber, the GitHub infra is the broker.
- Spring Cloud Stream / Spring Kafka — Spring services declare
@KafkaListenermethods (subscribers) against topics, with Kafka as the broker, so services never call each other directly.
Interview Questions
- What is the structural difference between Observer and Pub-Sub? Specifically, what does the subject/publisher know in each?
- Why does Pub-Sub scale better than Observer across independently deployed services?
- How would you implement topic wildcard matching (
orders.*)? What data structure/algorithm would you use for efficient matching at scale? - What delivery guarantees (at-least-once, at-most-once, exactly-once) exist in production brokers, and what does each cost you?
- How does a broker isolate a failing subscriber from affecting other subscribers or the publisher? Show where you'd add that isolation in code.
- Compare Redis Pub/Sub with Kafka. Why would you pick one over the other for a given use case?
- Explain how reactive programming (RxJava/Reactor) relates to Pub-Sub. What does it add on top of the basic publish/subscribe contract?
- When is introducing a message broker premature — what's the cost you're paying for decoupling you may not need yet?