Distributed Application Architecture Patterns

7.2 Queue-Based Load Leveling

Use a messaging queue to manage and cope with peaks in demand

This pattern is based on Queue-Based Load Leveling defined by Microsoft [99], Queue-Centric Workflow by Wilder [14, p. 27], Loose Coupling by Fehling et al. [15, p. 156] and Decoupled Invocation by Rotem-Gal-Oz [13, p. 47]. However, it is just a re-interpretation of the Point-to-Point Channel pattern by Hohpe et al. [4, p. 111, 100], focusing on benefits they consider part of messaging in general [4, p. 16].

Nygard [12, p. 73] presents publish/subscribe messaging and message queues as an alternative to improve the scalability of point-to-point communication but does not elaborate on their differences1. Burns [22, p. 117] considers handling bursts an inherent part of the Work Queue pattern but instead focuses on applying it for batch processing.

Richardson mostly inherits terminology from [4] and mentions message buffering as a benefit of broker-based Messaging [20, p. 93]. So does Newman [3, p. 107], but instead embraces the topic and queue terminology used by brokers [3, p. 135].

This work uses the term Queue-Based Load Leveling as it best captures the pattern’s intent, as per guidelines defined in § 1.3.

See also Message Broker.

7.2.1 Context

The system experiences variable load and does not manage to keep up when demand peaks, leading to dropped requests.

7.2.2 Solution

Introduce a message queue into the service’s communication channels (see fig. 15). This queue can be used to buffer requests and allow the service to process them at its own pace.

Figure 15: Queue-Based Load Levelling

Since the service now does not need to be powerful enough to handle the peak load, its computational resources can be scaled down, reducing the cost of the system. Furthermore, the services now have decreased coupling as they do not necessarily need to know about each other.

7.2.3 Potential issues

See the potential issues in § 4.2, namely increased latency, additional complexity, and an additional point of failure, when compared to direct communication. Furthermore, using a queue means losing replies without additional measures (see § 4.3).

See also § 4.2.3.

7.2.4 Example

The distributed transaction used for handling orders in ExampleEshop handles business failures with the Saga pattern, but this does not handle technical failures. To address this, the system deploys a message broker to handle the communication between services. This broker persists messages until they are successfully processed and communicates with the services transactionally.

7.2.6 Further Reading

  1. And later even calls it a pub/sub bus [12, p. 211]↩︎