Issue No.10 - October (2010 vol.9)
pp: 1391-1406
Nitin Salodkar , General Motors Indian Science Lab, ITPL, Bangalore
Abhay Karandikar , Indian Institute of Technology Bombay, Mumbai
Vivek S. Borkar , Tata Institute of Fundamental Research, Mumbai
In this paper, we consider the problem of energy-efficient uplink scheduling with delay constraint for a multiuser wireless system. We address this problem within the framework of constrained Markov decision processes (CMDPs) wherein one seeks to minimize one cost (average power) subject to a hard constraint on another (average delay). We do not assume the arrival and channel statistics to be known. To handle state-space explosion and informational constraints, we split the problem into individual CMDPs for the users, coupled through their Lagrange multipliers; and a user selection problem at the base station. To address the issue of unknown channel and arrival statistics, we propose a reinforcement learning algorithm. The users use this learning algorithm to determine the rate at which they wish to transmit in a slot and communicate this to the base station. The base station then schedules the user with the highest rate in a slot. We analyze convergence, stability, and optimality properties of the algorithm. We also demonstrate the efficacy of the algorithm through simulations within IEEE 802.16 system.
Multiuser fading channel, constrained Markov decision process, energy-efficient scheduling, learning algorithm.
Nitin Salodkar, Abhay Karandikar, Vivek S. Borkar, "A Stable Online Algorithm for Energy-Efficient Multiuser Scheduling", IEEE Transactions on Mobile Computing, vol.9, no. 10, pp. 1391-1406, October 2010, doi:10.1109/TMC.2010.106
[1] LAN/MAN Committee, IEEE 802.16-2004: IEEE Standard for Local and Metropolitan Area Networks—Part 16: Air Interface for Fixed Broadband Wireless Access Systems, IEEE, May 2004.
[2] "Third Generation Partnership Project," http:/, 2010.
[3] D. Tse and P. Viswanath, Fundamentals of Wireless Communication. Cambridge Univ. Press, 2005.
[4] D. Tse and S. Hanly, "Multi-Access Fading Channels—Part I: Polymatroid Structure, Optimal Resource Allocation and Throughput Capacities," IEEE Trans. Information Theory, vol. 44, no. 7, pp. 2796-2815, Nov. 1998.
[5] R. Knopp and P.A. Humblet, "Information Capacity and Power Control in Single-Cell Multiuser Communications," Proc. IEEE Int'l Conf. Comm. (ICC), pp. 331-335, June 1995.
[6] S. Shakkottai, T.S. Rappaport, and P.C. Karlsson, "Cross-Layer Design for Wireless Networks," IEEE Comm. Magazine, vol. 41, no. 10, pp. 74-80, Oct. 2003.
[7] R.A. Berry and R.G. Gallager, "Communication over Fading Channels with Delay Constraints," IEEE Trans. Information Theory, vol. 48, no. 5, pp. 1135-1149, May 2002.
[8] H. Wang and N. Mandayam, "A Simple Packet Transmission Scheme for Wireless Data over Fading Channels," IEEE Trans. Comm., vol. 52, no. 7, pp. 1055-1059, July 2004.
[9] M. Goyal, A. Kumar, and V. Sharma, "Power Constrained and Delay Optimal Policies for Scheduling Transmissions over a Fading Channel," Proc. IEEE INFOCOM, vol. 1, pp. 311-320, Mar. 2003.
[10] M. Agarwal, V.S. Borkar, and A. Karandikar, "Structural Properties of Optimal Transmission Policies over a Randomly Varying Channel," IEEE Trans. Automatic Control, vol. 53, no. 6, pp. 1476-1491, July 2008.
[11] D.V. Djonin and V. Krishnamurthy, Structural Results on Optimal Transmission Scheduling: A Constrained Markov Decision Process Approach. Springer Verlag, 2007.
[12] B. Prabhakar, E.U. Biyikoglu, and A.E. Gamal, "Energy-Efficient Transmission over a Wireless Link via Lazy Packet Scheduling," Proc. IEEE INFOCOM, vol. 1, pp. 386-394, Apr. 2001.
[13] B.E. Collins and R.L. Cruz, "Transmission Policies for Time Varying Channels with Average Delay Constraints," Proc. Allerton Conf. Comm. Control, and Computation, 1999.
[14] G. Rajadhyaksha and V.S. Borkar, "Transmission Rate Control over Randomly Varying Channels," Probability in the Eng. and Informational Sciences, vol. 19, no. 1, pp. 73-82, Jan. 2005.
[15] R. Berry, "Power and Delay Trade-Offs in Fading Channels," PhD thesis, Massachusetts Inst. of Tech nology, June 2000.
[16] M.J. Neely, "Optimal Energy and Delay Tradeoffs for Multi-User Wireless Downlinks," Proc. IEEE INFOCOM, pp. 1-13, 2006.
[17] N. Salodkar, A. Bhorkar, A. Karandikar, and V.S. Borkar, "An On-Line Learning Algorithm for Energy Efficient Delay Constrained Scheduling over a Fading Channel," IEEE J. Selected Areas in Comm., special issue on control and communications, vol. 26, no. 4, pp. 732-742, May 2008.
[18] R. Berry and E.M. Yeh, "Cross-Layer Wireless Resource Allocation," IEEE Signal Processing Magazine, vol. 21, no. 5, pp. 59-68, Sept. 2004.
[19] E. Yeh and A. Cohen, "Throughput and Delay Optimal Resource Allocation in Multiaccess Fading Channels," Proc. IEEE Int'l Symp. Information Theory (ISIT), p. 245, 2003.
[20] M.J. Neely, E. Modiano, and C. Rohrs, "Power and Server Allocation in Multi-Beam Satellite with Time Varying Channels," Proc. IEEE INFOCOM, pp. 138-152, 2002.
[21] L. Tassiulas and A. Ephremides, "Dynamic Server Allocation to Parallel Queues with Randomly Varying Connectivity," IEEE Trans. Information Theory, vol. 39, no. 2, pp. 466-478, Mar. 1993.
[22] S. Shakkottai and A. Stolyar, "Scheduling for Multiple Flows Sharing a Time-Varying Channel: The Exponential Rule," Analytic Methods in Applied Probability, pp. 185-201, Am. Math. Soc., 2002.
[23] E. Yeh and A. Cohen, "Delay Optimal Rate Allocation in Multiaccess Fading Communications," Proc. IEEE Workshop Multimedia Signal Processing, pp. 404-407, Oct. 2002.
[24] M. Andrews, K. Kumaran, K. Ramanan, A. Stolyar, P. Whiting, and R. Vijayakumar, "Providing Quality of Service over a Shared Wireless Link," IEEE Comm. Magazine, vol. 39, no. 2, pp. 150-154, Feb. 2001.
[25] E. Yeh, "Multiaccess and Fading in Communication Networks," PhD thesis, Massachusetts Inst. of Tech nology, Sept. 2001.
[26] E.F. Chaponniere, P. Black, J.M. Holtzman, and D. Tse, "Transmitter Directed Multiple Receiver System Using Path Diversity to Equitably Maximize Throughput," US Patent No. 6449490, Sept. 2002.
[27] X. Liu, E. Chong, and N. Shroff, "Opportunistic Transmission Scheduling with Resource-Sharing Constraints in Wireless Networks," IEEE J. Selected Areas in Comm., vol. 19, no. 10, pp. 2053-2064, Oct. 2001.
[28] E. Altman, Constrained Markov Decision Processes. Chapman and Hall/CRC Press, 1999.
[29] H.S. Wang and N. Moayeri, "Finite-State Markov Channel—A Useful Model for Radio Communication Channels," IEEE Trans. Vehicular Technology, vol. 44, no. 1, pp. 163-171, Feb. 1995.
[30] M. Puterman, Markov Decision Processes. Wiley, 1994.
[31] D.P. Bertsekas and J.N. Tsitsiklis, Neuro-Dynamic Programming. Athena Scientific, 1996.
[32] D.P. Bertsekas and R. Gallager, Data Networks. Prentice Hall, 1987.
[33] D.P. Bertsekas, Nonlinear Programming. Athena Scientific, 1999.
[34] W.B. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley-Interscience, 2007.
[35] V.S. Borkar and P.P. Varaiya, "Identification and Adaptive Control of Markov Chains," SIAM J. Control and Optimization, vol. 20, no. 4, pp. 470-489, 1982.
[36] F.P. Kelly, A.K. Maulloo, and D.K.H. Tan, "Rate Control for Communication Networks: Shadow Prices, Proportional Fairness and Stability," The J. Operational Research Soc., vol. 49, no. 3, pp. 237-252, 1998.
[37] V.S. Borkar, Stochastic Approximation—A Dynamical Systems Viewpoint. Hindustan Publishing Agency and Cambridge Univ. Press, 2008.
[38] P. Milgrom and I. Segal, "Envelope Theorems for Arbitrary Choice Sets," Econometrica, vol. 70, pp. 583-601, 2002.
[39] S.P. Meyn and R.J. Tweedie, Markov Chains and Stochastic Stability, second ed. Cambridge Univ. Press, 2009.
[40] M.W. Hirsch, "Systems of Differential Equations That Are Competitive or Cooperative II: Convergence Almost Everywhere," SIAM J. Math. Analysis, vol. 16, pp. 423-439, 1985.
[41] P. Whittle, "Restless Bandits; Activity Allocation in a Changing World," J. Applied Probability, vol. 25A, pp. 301-313, 1988.
[42] C.H. Papadimitriou and J.N. Tsitsiklis, "The Complexity of Optimal Queueing Network Control," Math. of Operations Research, vol. 24, no. 2, pp. 293-305, 1999.
[43] D. Fudenberg and J. Tirole, Game Theory. MIT Press, 1995.
[44] C. Cicconeti, L. Lenzini, E. Mingozzi, and C. Eklund, "Quality of Service Support in IEEE 802.16 Networks," IEEE Network, vol. 20, no. 2, pp. 50-55, Mar./Apr. 2006.
[45] T.G. Neame, M. Zukerman, and R.G. Addie, "Modeling Broadband Traffic Streams," Proc. IEEE Global Telecomm. Conf. (GLOBECOM), pp. 1048-1052, 1999.