This Article 
 Bibliographic References 
 Add to: 
Bounding the Mean Response Time of the Minimum Expected Delay Routing Policy: An Algorithmic Approach
December 1995 (vol. 44 no. 12)
pp. 1371-1382

Abstract—Balancing loads in a multi-server system can have a significant impact on performance. In this paper, we model such a system as a heterogeneous multi-server queueing system. We study the behavior of such a system operating under the minimum expected delay (MED) routing policy, i.e., an arriving customer is assigned to the queue which has the minimal expected value of unfinished work. This routing discipline can be viewed as a generalization of the join-the-shortest queue (SQ) discipline for homogeneous servers. There is no closed-form solution for this class of queueing problem. In this paper, we provide a methodology to compute upper and lower bounds on the mean response time of the system. This methodology allows one to tradeoff the tightness of the bounds and computational cost. Applications and numerical examples are presented which show how to use this methodology for deriving performance measures and also illustrating that the excellent accuracy of the computational algorithm which is achievable with modest computational cost.

[1] I.J.B.F. Adan,J. Wessels,, and W.H.M. Zijm,“Analysis of the symmetric shortest queue problem,” Stochastic Models, vol.6, pp. 691-713, 1990.
[2] I.J.B.F. Adan, J. Wessels, and W.H.M. Zijm, "Analysis of the Asymmetric Shortest Queue Problem," Queueing Systems, vol. 8, pp. 1-58, 1991.
[3] I. Adan,G.-J. van Houtum,, and J. van der Wal,“Upper and lower bounds for the waiting time in the symmetric shortest queue system,” Technical Report COSOR 92-09, Eindhoven Univ. of Technology.
[4] N.R. Baker,“Optimal user search sequences and implications for information system operation,” J. Applied Probability, vol.24, pp. 540-546, 1987.
[5] J.P.C. Blanc,“A note on waiting times in systems with queues in parallel,” J. Applied Probability, vol.24, pp. 540-546, 1987.
[6] B.W. Conolly,“The autostrada queueing problem,” J. Applied Probability, vol. 21, pp. 394-403, 1984.
[7] J.W. Cohen and O.J. Boxma,Boundary Value Problems in Queueing System Analysis. NorthHolland, 1983.
[8] P.J. Courtois,Decomposability—Queueing and Computer System Applications.New York: Academic Press, 1977.
[9] P.-J. Courtois and P. Semal, Computable Bounds for Conditional Steady-State Probabilities in Large Markov Chains and Queueing Models IEEE J. Selected Areas in Comm., vol. 4, no. 6, pp. 926-937, Sept. 1986.
[10] A. Ephremides,P. Varaiya,, and J. Walrand,“A simple dynamic routing problem,” IEEE Trans. Automatic Control, vol. 25, 1980.
[11] L. Flatto and H.P McKean,“Two queues in parallel,” Comm. Pure and Applied Mathematics, vol. 30, pp. 255-263, 1977.
[12] S. Halfin,“The shortest queue problem,” J. Applied Probability, vol. 22, pp. 865-878, 1985.
[13] W.K. Grassmann,“Transient and steady state results for two parallel queues,” Omega, vol. 8, pp. 105-112, 1980.
[14] J.F.C Kingman,“Two similar queues in parallel,” Annals of Mathematical Statistics, vol 32, pp. 1,314-1,323, 1961.
[15] C. Knessl,B.J. Matkowsky,Z. Schuss,, and C. Tier,“Two parallel queues with dynamic routing,” IEEE Trans. Communications, vol. 34, pp. 1,170-1,175, 1986.
[16] L. Kleinrock,Queueing Systems: Volume I: Theory.New York: Wiley-Interscience Publication, 1975.
[17] J.D.C Little,“A proof of the queueing formula L =λW,” Operations Research, vol 9, pp. 383-387, 1967.
[18] H.C. Lin and C.S. Raghavendra,“An analysis of the join the shortest queue policy,” Electrical Eng. technical report, Univ. of Southern California, 1991.
[19] J.C.S. Lui and R.R. Muntz,“Algorithmic approach to bounding the response time of a minimum expected delay routing system,” Proc. 1992 ACM SIGMETRICS/Performance’92 Conf., pp. 140-152.
[20] A.W. Marshall and I. Olkin., Inequalities: Theory of Majorization and Applications.New York: Academic Press, 1979.
[21] R.D. Nelson and T.K. Philips, "An Approximation to the Response Time for Shortest Queue Routing," ACM Performance Evaluation Review, vol. 17, pp. 181-189, May 1989.
[22] R.D. Nelson and T.K. Philips,“An approximation for the mean response time for shortest queue routing with general interarrival and service times,” Technical Report RC15429, IBM T.J. Watson Research Lab, 1990.
[23] M.F. Neuts,Matrix-Geometric Solutions in Stochastic Models—An Algorithmic Approach.Baltimore: Johns Hopkins Univ. Press, 1981.
[24] B.M. Rao and M.J.M. Posner,“Algorithmic and approximate analysis of the shorter queue model,” Naval Research Logistics, vol.34, pp. 381-398, 1987.
[25] W.J. Stewart,“MARCA: Markov chain analyzer, a software package for Markov modeling,” Numerical Solution of Markov Chains. Dekker Press, 1991.
[26] D. Towsley and S. Chen,“Design and modeling policies for two server fork/join queueing systems,” COINS Technical Report 91-39, Univ. of Massachusetts.
[27] D. Towsley,P. Sparaggis,, and C. Cassandras,“Stochastic ordering properties and optimal routing control for a class of finite capacity queueing systems,” IEEE Trans. Automatic Control, vol. 37, no. 9, pp. 1,446-1,451, Septe. 1992.
[28] Y.T. Wang and R.J.T. Morris,“Load sharing in distributed systems,” IEEE Trans. Computers, vol. 34, no. 3, pp. 204-217, Mar. 1985.
[29] W. Winston,“Optimality of the shortest line discipline,” J. Applied Probability, vol 15, pp. 181-189, 1977.
[30] Y. Zhao and W.K. Grassmann,“The shortest queue model with jockeying,” Naval Research Logistics, vol. 37, pp. 773-787, 1990.
[31] Y. Zhao and W.K. Grassmann, "A Numerically Stable Algorithm for Two Server Queue Models," Queueing Systems, vol. 8, pp. 59-79, 1991.

Index Terms:
Load balancing, parallel systems, scheduling, queueing models, shortest delay routing.
Richard R. Muntz, John C.S. Lui, Don Towsley, "Bounding the Mean Response Time of the Minimum Expected Delay Routing Policy: An Algorithmic Approach," IEEE Transactions on Computers, vol. 44, no. 12, pp. 1371-1382, Dec. 1995, doi:10.1109/12.477243
Usage of this product signifies your acceptance of the Terms of Use.