4.2 Publisher–Subscriber (Pub–Sub, Topics)
Asynchronous one-to-many communicationThis pattern is based on the Publish-Subscribe Channel by Hohpe et al. [4, p. 113, 53], Publisher-Subscriber by Buschmann et al. [54, p. 234] and Microsoft [55] and Publish-subscribe by Deenadayalan [56].
The name for this pattern is sometimes interchangeably used for communication through message brokers in general; see § 4.2.7.
4.2.1 Context
There is a need for asynchronous communication between a publishing service and one or more consuming services (subscribers). Other supporting factors include
The need for loosely coupled or fluctuating components in the system
The need for message buffering or guaranteed delivery
4.2.2 Solution
Introduce a message broker as an intermediary between publishers and subscribers (see fig. 5). This broker manages “topics,” which consist of one input queue for the publisher and multiple output queues, one for each subscriber. The publisher sends messages to the input queue, which the broker processes and propagates to all the output queues.
The broker does not need to manage only one queue; once set up, it can handle multiple topics, and each subscriber can choose which topics to subscribe to.
Existing message brokers typically also provide guaranteed delivery, but the implementation details are challenging and vary [3, p. 135] and, as such, are outside the scope of this work. Other patterns described here may assume that the broker provides this feature.
4.2.3 Potential issues
Using a broker means introducing another component into the system, leading to increased complexity and possibly increased latency. Also, due to its nature, it can become a single point of failure and thus a possible bottleneck.
Having multiple subscribers can lead to ordering issues if the order of messages is important, and, as with any network communication, message duplication is a possibility and needs to be handled.
Furthermore, implementing a custom message broker may not be feasible due to limited development resources, which may mean trusting a third-party service.
4.2.4 Example
ExampleEshop uses a read-only database to improve their performance when serving product pages. However, they have found that even this is not enough to handle the surge in traffic during large events. To address this, they decided to replicate their read-only database. To ensure that the data is consistent across all replicas, they subscribe to the same topic to receive updates.
4.2.5 Related patterns
This pattern directly facilitates Scatter–Gather, which deals with destructuring workloads into smaller tasks and reassembling the results (see § 6.3). It is also possible to use a single messaging queue for communication instead of a topic, which still includes the benefits of message buffering.
Using it in one-to-one communication results in Queue-based Load Levelling
Using it for one-to-many communication results in load-balancing Competing Consumers
4.2.6 Further reading
The “Gang of Four” Observer pattern [24, p. 273] this pattern builds upon with asynchronous messaging
Wikipedia [57]
The Inbox and Outbox pattern can also be used for guaranteed delivery [58]
4.2.7 Message Broker
The categorisation, naming, and cross-section of messaging patterns vary. Most centre around messaging middleware and queues [4, p. 76, 14, p. 27, 15, p. 136, 20, p. 85], whilst others assume that terms such as “pub/sub” and “competing consumers” are common knowledge and do not need explaining [3, p. 445, 12, p. 117, 20, p. 92]. This work errs instead on caution and fidelity, presenting the most common communication patterns but focusing primarily on the context in which they are applied: see §§ 4.2, 4.3, 6.1, 6.3, and 7.2.