This Article 
 Bibliographic References 
 Add to: 
The Power of Two Choices in Randomized Load Balancing
October 2001 (vol. 12 no. 10)
pp. 1094-1104

Abstract—We consider the following natural model: Customers arrive as a Poisson stream of rate \lambda n, \lambda < 1, at a collection of n servers. Each customer chooses some constant d servers independently and uniformly at random from the n servers and waits for service at the one with the fewest customers. Customers are served according to the first-in first-out (FIFO) protocol and the service time for a customer is exponentially distributed with mean 1. We call this problem the supermarket model. We wish to know how the system behaves and in particular we are interested in the effect that the parameter d has on expected time a customer spends in the system in equilibrium. Our approach uses a limiting, deterministic model representing the behavior as n \rightarrow \infty to approximate the behavior of finite systems. The analysis of the deterministic model is interesting in its own right. Along with a theoretical justification of this approach, we provide simulations that demonstrate that the method accurately predicts system behavior, even for relatively small systems. Our analysis provides surprising implications: Having d = 2 choices leads to exponential improvements in the expected time a customer spends in the system over d = 1, whereas having d=3 choices is only a constant factor better than d=2. We discuss the possible implications for system design.

[1] Y. Azar, A.Z. Broder, A.R. Karlin, and E. Upfal, Balanced Allocations Proc. 26th ACM Symp. Theory of Computing, pp. 593-602, 1994.
[2] M. Adler, S. Chakrabarti, M. Mitzenmacher, and L. Rasmussen, “Parallel Randomized Load Balancing,” Proc. 27th ACM Symp. Theory of Computing, pp. 238-247, 1995.
[3] D. Achlioptas and M. Molloy, “The Analysis of a List-Coloring Algorithm on a Random Graph,” Proc. 38th IEEE Symp. Foundations of Computer Science, 1997.
[4] A. Bestavros, “Load Profiling: A Methodology for Scheduling Real-Time Tasks in a Distributed System,” Proc. ICDCS '97: IEEE Int'l Conf. Distributed Computing Systems, May 1997.
[5] M. Dahlin, Serverless Network File Systems, PhD dissertation, University of Calif. at Berkeley, Computer Science Dept., 1995.
[6] K. Deimling, “Ordinary Differential Equations in Banach Spaces,” Lecture Notes in Math., vol. 96, 1977.
[7] S.N. Ethier and T.G. Kurtz, Markov Processes: Characterization and Convergence. John Wiley and Sons, 1986.
[8] D.L. Eager, E.D. Lazowska, and J. Zahorjan, "Adaptive Load Sharing in Homogeneous Distributed Systems," IEEE Trans. Software Eng., vol. 12, no. 5, pp. 662-675, May 1986.
[9] D.L. Eager, E.D. Lazowska, and J. Zahorjan, "A Comparison of Receiver-Initiated and Sender-Initiated Adaptive Load Sharing," Performance Evaluation, Vol. 6, Mar. 1986, pp. 53-68.
[10] B. Hajek, “Asymptotic Analysis of an Assignment Problem Arising in a Distributed Communications Protocol,” Proc. 27th Conf. Decision and Control, pp. 1455-1459, 1988.
[11] R.M. Karp, M. Luby, and F. Meyer auf der Heide, Efficient PRAM Simulation on a Distributed Memory Machine Proc. 24th ACM Symp. Theory of Computing, pp. 318-326, May 1992.
[12] R.M. Karp and M. Sipser, “Maximum Matchings in Sparse Random Graphs,” Proc. 22nd IEEE Symp. Foundations of Computer Science, pp. 364-375, 1981.
[13] R.M. Karp, U.V. Vazirani, and V.V. Vazirani, “An Optimal Algorithm for On-Line Bipartite Matching,” Proc. 22nd ACM Symp. Theory of Computing, pp. 352-358, 1990.
[14] T.G. Kurtz, “Solutions of Ordinary Differential Equations as Limits of Pure Jump Markov Processes,” J. Applied Probability, vol. 7, pp. 49-58, 1970.
[15] T.G. Kurtz, “Limit Theorems for Sequences of Jump Markov Processes Approximating Ordinary Differential Processes,” J. Applied Probability, vol. 9, pp. 344-356, 1971.
[16] T.G. Kurtz, “Strong Approximation Theorems for Density Dependent Markov Chains,” Stochastic Processes and Applications, vol. 6, pp. 223-240, 1978.
[17] T.G. Kurtz, Approximation of Population Processes. SIAM Press, 1981.
[18] M. Luby, M. Mitzenmacher, M.A. Shokrollahi, D. Spielman, and V. Stemann, “Practical Loss-Resilient Codes,” Proc. 29th ACM Symp. Theory of Computing, pp. 150-159, 1997.
[19] P.D. MacKenzie, C.G. Plaxton, and R. Rajaraman, “On Contention Resolution Protocols and Associated Probabilistic Phenomena,” J. ACM, vol. 45, no. 2, pp. 324-378, 1998.
[20] R. Abraham, J. Marsden, and T. Raitu, Manifolds, Tensor Analysis, and Applications. Addison-Wesley, 1983.
[21] A.N. Michel and R.K. Miller, “Qualitative Analysis of Large Scale Dynamical Systems,” Math. Science and Eng., vol. 134, 1977.
[22] R. Mirchandaney, D. Towsley, and J.A. Stankovic, "Analysis of the Effect of Delays on Load Sharing," IEEE Trans. Computers, vol. 38, no. 11, pp. 1,513-1,525, Nov. 1989.
[23] R. Mirchandaney, D. Towsley, and J.A. Stankovic, "Adaptive Load Sharing in Heterogeneous Distributed Systems," J. Parallel and Distributed Computing, Vol. 9, 1990, pp. 331-346.
[24] M. Mitzenmacher, “Load Balancing and Density Dependent Jump Markov Processes,” Proc. 37th IEEE Symp. Foundations of Computer Science, pp. 213-222, 1996.
[25] M. Mitzenmacher, The Power of Two Choices in Randomized Load Balancing PhD thesis, Univ. of California Berkeley, 1996.
[26] M. Mitzenmacher, “On the Analysis of Randomized Load Balancing Schemes,” Theory of Computing Systems, vol. 32, pp. 361-386, 1999.
[27] M. Michael, How Useful Is Old Information? IEEE Trans. Parallel and Distributed Systems, vol. 11, no. 1, pp. 6-20, Jan. 2000.
[28] M. Mitzenmacher, A. Richa, and R. Sitaraman, “The Power of Two Random Choices: A Survey of Techniques and Results,” Handbook of Randomized Computing, vol. 1, June 2001.
[29] B. Pittel, J. Spencer, and N. Wormald, “Sudden Emergence of a Giant k-Core in a Random Graph,” J. Combinatorial Theory, Series B, vol. 67, pp. 111-151, 1996.
[30] J.R. Santos and R. Muntz, “Design of the RIO (Randomized I/O) Storage Server,” technical report, Computer Science Dept., Univ. of Calif., Los Angeles, May 1997.
[31] A. Shwartz and A. Weiss, Large Deviations for Performance Analysis. Chapman and Hall, 1995.
[32] V. Stemann, “Parallel Balanced Allocations,” Proc. Eighth Ann. ACM Symp. Parallel Algorithms and Architectures, pp. 261-269, 1996.
[33] N.D. Vvedenskaya, R.L. Dobrushin, and F.I. Karpelevich, “Queueing System with Selection of the Shortest of Two Queues: An Asymptotic Approach,” Problems of Information Transmission, vol. 32, pp. 15-27, 1996.
[34] R. Weber, “On the Optimal Assignment of Customers to Parallel Servers,” J. Applied Probability, vol. 15, pp. 406-413, 1978.
[35] W. Winston, “Optimality of the Shortest Line Discipline” J. Applied Probability, vol. 14, pp. 181-189, 1977.
[36] N.C. Wormald, “Differential Equations for Random Processes and Random Graphs,” Annals of Applied Probability, vol. 5, pp. 1217-1235, 1995.
[37] S. Zhou, "A Trace-Driven Simulation Study of Dynamic Load Balancing," IEEE Trans. Software Eng., vol. 14, no. 9, pp. 1,327-1,341, Sept. 1988.

Index Terms:
Load balancing, queuing theory, distributed systems, limiting systems, choices.
Michael Mitzenmacher, "The Power of Two Choices in Randomized Load Balancing," IEEE Transactions on Parallel and Distributed Systems, vol. 12, no. 10, pp. 1094-1104, Oct. 2001, doi:10.1109/71.963420
Usage of this product signifies your acceptance of the Terms of Use.