Subscribe

Issue No.09 - Sept. (2013 vol.12)

pp: 1737-1749

Jongwook Lee , Samsung Advanced Institute of Technology, Yongin

Saewoong Bahk , Seoul National University, Seoul

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TMC.2012.139

ABSTRACT

In this paper, we deal with a cost minimization problem for a multihomed mobile terminal that downloads and plays a video-on-demand (VoD) stream. The cost consists of the user's dissatisfaction due to playback disruptions and communication cost for downloading the VoD stream. There are three components in our approach: parameter estimation, threshold adjustment, and threshold compensation. Since we do not assume any a priori knowledge about underlying random variables, necessary parameter values are estimated online. Using the resultant estimates, we formulate the problem as a Markov decision process (MDP) problem considering as if the random variables are exponentially distributed. To solve the MDP problem efficiently, we propose a threshold adjustment algorithm that exploits some structural properties of any optimal solution that are specific to our problem. Finally, to handle the cases where the random variables are not exponentially distributed, we propose a threshold compensation algorithm to compensate for the modeling error. Through extensive simulations, we compare the performance of our scheme with those of static threshold schemes.

INDEX TERMS

Random variables, Mobile computing, Markov processes, Minimization, Wireless networks, Mobile communication, Servers, Markov decision process, Multihoming, video-on-demand

CITATION

Jongwook Lee, Saewoong Bahk, "On the MDP-Based Cost Minimization for Video-on-Demand Services in a Heterogeneous Wireless Network with Multihomed Terminals",

*IEEE Transactions on Mobile Computing*, vol.12, no. 9, pp. 1737-1749, Sept. 2013, doi:10.1109/TMC.2012.139REFERENCES

- [1] B. Liu, N. Boukhatem, P. Martins, and P. Bertin, "Multihoming at Layer-2 for Inter-RAT Handover,"
Proc. IEEE Int'l Symp. Personal Indoor and Mobile Radio Comm. (PIMRC), pp. 1173-1178, Sept. 2010.- [2] G. Tsirtsis, H. Soliman, N. Montavont, G. Giaretta, and K. Kuladinithi, "Flow Bindings in Mobile IPv6 and NEMO Basic Support," IETF Internet draft, Aug. 2010.
- [3] Y. Choi, K. Sohaib, H. Kim, K. Chang, S. Kang, and Y. Han, "A Distributed Multiple Spectrum Pricing Scheme for Optimality Support in Multiaccess Systems,"
J. Comm. and Networks, vol. 11, no. 4, pp. 368-374, 2009.- [4] F. Davoli, M. Marchese, and M. Mongelli, "A Measurement-Based Adaptive Control Mechanism for Pricing in Telecommunication Networks,"
J. Comm. and Networks, vol. 12, no. 3, pp. 253-265, 2010.- [5] M. Frey and N.-Q. Son, "A Gamma-Based Framework for Modeling Variable-Rate MPEG Video Sources: The GOP GBAR Model,"
IEEE/ACM Trans. Networking, vol. 8, no. 6, pp. 710-719, Dec. 2000.- [6] K. Kim and H.A. Latchman, "Statistical Traffic Modeling of MPEG Frame Size: Experiments and Analysis,"
IIIC J. Systemics, Cybernetics and Informatics, vol. 7, no. 5, pp. 54-59, 2009.- [7] E. Stevens-Navarro, Y. Lin, and V.W.S. Wong, "An MDP-Based Vertical Handoff Decision Algorithm for Heterogeneous Wireless Networks,"
IEEE Trans. Vehicular Technology, vol. 57, no. 2, pp. 1243-1254, Mar. 2008.- [8] S.J. Russell and P. Norvig,
Artificial Intelligence: A Modern Approach, second ed. Prentice-Hall, 2005.- [9] A.R. Cassandra, "Optimal Policies for Partially Observable Markov Decision Processes," Technical Report CS-94-14, Brown Univ., Dept. of Computer Science, 1994.
- [10] "Curse of Dimensionality," http://en.wikipedia.org/wiki Curse_of_dimensionality , 2013.
- [11] D.P. Bertsekas,
Dynamic Programming and Optimal Control, third ed., vol. 2. Athena Scientific, 2007.- [12] "Introduction to lp_solve 5.5.2.0," http://lpsolve.sourceforge.net5.5/, 2013.
- [13] D. Kaspar, "Enhancing Video-on-Demand Playout over Multiple Heterogeneous Access Networks,"
Proc. IEEE Consumer Comm. and Networking Conf. (CCNC), pp. 1-5, Jan. 2010.- [14] E.A. Feinberg and A. Shwartz,
Handbook of Markov Decision Processes: Methods and Applications. Kluwer Academic, 2002.- [15] P.A. Chou and Z. Miao, "Rate-Distortion Optimized Streaming of Packetized Media,"
IEEE Trans. Multimedia, vol. 8, no. 2, pp. 390-404, Apr. 2006.- [16] D. Nguyen, T. Nguyen, and X. Yang, "Joint Network Coding and Scheduling for Media Streaming over Multiuser Wireless Networks,"
IEEE Trans. Vehicular Technology, vol. 60, no. 3, pp. 1086-1098, Mar. 2011.- [17] J. Wang, R.V. Prasad, and I.G.M.M. Niemegeers, "Solving the Incertitude of Vertical Handovers in Heterogeneous Mobile Wireless Network Using MDP,"
Proc. IEEE Int'l Conf. Comm. (ICC), pp. 2187-2192, May 2008.- [18] C. Sun, E. Stevens-Navarro, and V.W.S. Wong, "A Constrained MDP-Based Vertical Handoff Decision Algorithm for 4G Wireless Networks,"
Proc. IEEE Int'l Conf. Comm. (ICC), pp. 2169-2174, May 2008.- [19] Y. Chen, H. Chen, L. Xie, and K. Wang, "An MDP-Based Handoff Decision Algorithm for Multi-Domain Heterogeneous Wireless Access Networks,"
Proc. Int'l Conf. Comm., Circuits and Systems (ICCCAS), pp. 163-167, July 2010.- [20] B.-J. Chang, J.-F. Chen, C.-H. Hsieh, and Y.-H. Liang, "Markov Decision Process-Based Adaptive Vertical Handoff with RSS Prediction in Heterogeneous Wireless Networks,"
Proc. IEEE Wireless Comm. and Networking Conf. (WCNC), pp. 1-6, Apr. 2009.- [21] X. Gelabert, J. Pérez-Romero, and R. Agustí, "A Markovian Approach to Radio Access Technology Selection in Heterogeneous Multiaccess/Multiservice Wireless Networks,"
IEEE Trans. Mobile Computing, vol. 7, no. 10, pp. 1257-1270, Oct. 2008.- [22] J. Iyengar, P. Amer, and R. Stewart, "Concurrent Multipath Transfer Using SCTP Multihoming over Independent End-to-End Paths,"
IEEE/ACM Trans. Networking, vol. 14, no. 5, pp. 951-964, Oct. 2006.- [23] V. Bui, W. Zhu, A. Botta, and A. Pescapé, "A Markovian Approach to Multipath Data Transfer in Overlay Networks,"
IEEE Trans. Parallel and Distributed Systems, vol. 21, no. 10, pp. 1398-1411, Oct. 2010.- [24] T. Alpcan, J.P. Singh, and T. Başar, "Robust Rate Control for Heterogeneous Network Access in Multihomed Environments,"
IEEE Trans. Mobile Computing, vol. 8, no. 1, pp. 41-51, Jan. 2009.- [25] K.-D. Lee and V.C.M. Leung, "Utility-Based Rate-Controlled Parallel Wireless Transmission of Multimedia Streams with Multiple Importance Levels,"
IEEE Trans. Mobile Computing, vol. 8, no. 1, pp. 81-92, Jan. 2009.- [26] T. Taleb, "A Cross-Layer Approach for an Efficient Delivery of TCP/RTP-Based Multimedia Applications in Heterogeneous Wireless Networks,"
IEEE Trans. Vehicular Technology, vol. 57, no. 6, pp. 3801-3814, Nov. 2008.- [27] L. Zhou, H. Wang, S. Lian, Y. Zhang, and A. Vasilakos, "Availability-Aware Multimedia Scheduling in Heterogeneous Wireless Networks,"
IEEE Trans. Vehicular Technology, vol. 60, no. 3, pp. 1161-1170, Mar. 2011.- [28] X. Zhu, P. Agrawal, J.P. Singh, T. Alpcan, and B. Girod, "Distributed Rate Allocation Policies for Multihomed Video Streaming over Heterogeneous Access Networks,"
IEEE Trans. Multimedia, vol. 11, no. 4, pp. 752-764, June 2009.- [29] S.-B. Lee, A.F. Smeaton, and G.-M. Muntean, "Quality-Oriented Multiple-Source Multimedia Delivery over Heterogeneous Wireless Networks,"
IEEE Trans. Broadcasting, vol. 57, no. 2, pp. 216-230, June 2011.- [30] S.I. Lee, H. Park, and M. van der Schaar, "Foresighted Joint Resource Reciprocation and Scheduling Strategies for Real-Time Video Streaming over Peer-to-Peer Networks,"
Proc. Int'l Packet Video Workshop, pp. 1-10, May 2009.- [31] C. Liang, Z. Fu, Y. Liu, and C.W. Wu, "Incentivized Peer-Assisted Streaming for On-Demand Services,"
IEEE Trans. Parallel and Distributed Systems, vol. 21, no. 9, pp. 1354-1367, Sept. 2010.- [32] X. Chu, K. Zhao, Z. Li, and A. Mahanti, "Auction-Based On-Demand P2P Min-Cost Media Streaming with Network Coding,"
IEEE Trans. Parallel and Distributed Systems, vol. 20, no. 12, pp. 1816-9219, Dec. 2009.- [33] Y. He, G. Shen, Y. Xiong, and L. Guan, "Optimal Prefetching Scheme in P2P VoD Applications with Guided Seeks,"
IEEE Trans. Multimedia, vol. 11, no. 1, pp. 138-151, Jan. 2009.- [34] K.-J. Peng and Z. Tsai, "Distortion and Cost Controlled Video Streaming in a Heterogeneous Wireless Network Environment,"
Proc. IEEE Int'l Symp. Personal, Indoor and Mobile Radio Comm. (PIMRC), pp. 1-5, Sept. 2006.- [35] A.H. Zahran and C.J. Sreenan, "Threshold-Based Media Streaming Optimization for Heterogeneous Wireless Networks,"
IEEE Trans. Mobile Computing, vol. 9, no. 6, pp. 753-764, June 2010.- [36] C.G. Cassandras and S. Lafortune,
Introduction to Discrete Event Systems. Kluwer Academic, 1999. |