Distributed Application Architecture Patterns

7.6 Leader and Followers

Decentrally appoint a replaceable group leader

This pattern is based on Leader and Followers by Joshi [5, p. 85, 119], Leader Election by Microsoft [120], and Ownership Election by Burns [22, p. 93].

7.6.1 Context

A group of peer service instances with no natural leader need to coordinate their actions. However, the group needs to be resilient to the leader’s failure, and the leader is replaceable.

7.6.2 Solution

Each service has one of the following states: Leader, Follower, or Looking for Leader (sometimes called Candidate). Each service starts a leader election on startup and does not accept requests until a leader is elected.

Once a leader is elected, it sends HeartBeats [see 5, p. 93] to the followers. If the followers do not receive a heartbeat within a specific time, they start a new leader election (see fig. 19).

Figure 19: Leader and Followers

There are a number of algorithms for leader election, such as Zab [121] or Raft [122].

Alternatively, if there is an implicit order in the group, such as the service age, this can be used to determine the leader without an election, known as the Emergent Leader pattern [5, p. 375].

7.6.3 Potential issues

The added complexity of leader election may not be necessary, depending on the availability requirements of the system.

The leader may become a bottleneck if responsible for too much work. If this cannot be mitigated otherwise, one option is to have the leader not participate in the work itself, only in its coordination.

A fault in the algorithm implementation can lead to split-brain1 scenarios, where multiple leaders are elected.

7.6.4 Example

ExampleEshop uses health monitoring to check the status of its services. The system uses multiple monitors to accommodate for possible failures and handle the number of service instances they operate. However, they need to coordinate on how to handle failures so as not to result in conflicts, such as multiple restarts. Furthermore, there is no natural leader, as all monitors are equal. To address this, the monitors use leader election to select a single monitor to react to failures. If this monitor fails, other monitors will elect a new leader.

7.6.6 Further reading

  1. [13], [p. 66]↩︎