Issue No. 10 - October (2001 vol. 12)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/71.963420
<p><b>Abstract</b>—We consider the following natural model: Customers arrive as a Poisson stream of rate <tmath>\lambda n</tmath>, <tmath>\lambda < 1</tmath>, at a collection of <tmath>n</tmath> servers. Each customer chooses some constant <tmath>d</tmath> servers independently and uniformly at random from the <tmath>n</tmath> servers and waits for service at the one with the fewest customers. Customers are served according to the first-in first-out (FIFO) protocol and the service time for a customer is exponentially distributed with mean 1. We call this problem the <it>supermarket model</it>. We wish to know how the system behaves and in particular we are interested in the effect that the parameter <tmath>d</tmath> has on expected time a customer spends in the system in equilibrium. Our approach uses a limiting, deterministic model representing the behavior as <tmath>n \rightarrow \infty</tmath> to approximate the behavior of finite systems. The analysis of the deterministic model is interesting in its own right. Along with a theoretical justification of this approach, we provide simulations that demonstrate that the method accurately predicts system behavior, even for relatively small systems. Our analysis provides surprising implications: Having <tmath>d = 2</tmath> choices leads to exponential improvements in the expected time a customer spends in the system over <tmath>d = 1</tmath>, whereas having <tmath>d=3</tmath> choices is only a constant factor better than <tmath>d=2</tmath>. We discuss the possible implications for system design.</p>
Load balancing, queuing theory, distributed systems, limiting systems, choices.
Michael Mitzenmacher, "The Power of Two Choices in Randomized Load Balancing", IEEE Transactions on Parallel & Distributed Systems, vol. 12, no. , pp. 1094-1104, October 2001, doi:10.1109/71.963420