The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - November (2009 vol.8)
pp: 1469-1479
Yung-Han Chen , Industrial Technology Research Institute
Chung-Ju Chang , National Chiao Tung University, Hsinchu
Ching Yao Huang , National Chiao Tung University, Hsinchu
ABSTRACT
In this paper, admission control by a fuzzy Q-learning technique is proposed for WCDMA/WLAN heterogeneous networks with multimedia traffic. The fuzzy Q-learning admission control (FQAC) system is composed of a neural-fuzzy inference system (NFIS) admissibility estimator, an NFIS dwelling estimator, and a decision maker. The NFIS admissibility estimator takes essential system measures into account to judge how each reachable subnetwork can support the admission request's required QoS and then output admissibility costs. The NFIS dwelling estimator considers the Doppler shift and the power strength of the requested user to assess his/her dwell time duration in each reachable subnetwork and then output dwelling costs. Also, in order to minimize the expected maximal cost of the user's admission request, a minimax theorem is applied in the decision maker to determine the most suitable subnetwork for the user request or to reject. Simulation results show that FQAC can always maintain the system QoS requirement up to a traffic intensity of 1.1 because it can appropriately admit or reject the users' admission requests. Also, the FQAC can achieve lower blocking probabilities than conventional JSAC proposed in [20] and can significantly reduce the handoff rate by 15-20 percent.
INDEX TERMS
Fuzzy Q-learning, admission control, handoff, heterogeneous network.
CITATION
Yung-Han Chen, Chung-Ju Chang, Ching Yao Huang, "Fuzzy Q-Learning Admission Control for WCDMA/WLAN Heterogeneous Networks with Multimedia Traffic", IEEE Transactions on Mobile Computing, vol.8, no. 11, pp. 1469-1479, November 2009, doi:10.1109/TMC.2009.65
REFERENCES
[1] IEEE Std. 802.11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specification, IEEE, 1999.
[2] 3rd Generation Partnership Project; 3GPP System to Wireless Local Area Network (WLAN) Interworking; System Description (Release 7), 3GPP Std. TS23.234 V7.6.0, www.3gpp.org, Dec. 2007.
[3] IEEE Std. 802.11u/D2.0 Amendment to Standard Information Technology: IEEE 802.11 Interworking with External Networks, IEEE, Jan. 2008.
[4] S. Shen, C.-J. Chang, C.Y. Huang, and Q. Bi, “Intelligent Call Admission Control for Wideband CDMA Cellular Systems,” IEEE Trans. Wireless Comm., vol. 3, no. 5, pp. 1810-1821, Sept. 2004.
[5] D. Gao, J. Cai, and K.N. Ngan, “Admission Control in IEEE 802.11e Wireless LANs,” IEEE Network, vol. 19, no. 4, pp. 6-13, July/Aug. 2005.
[6] F. Zhu and J. McNair, “Optimizations for Vertical Handoff Decision Algorithms,” Proc. IEEE Wireless Comm. and Networking Conf. (WCNC), vol. 2, pp. 867-872, 2004.
[7] W. Zhang, “Handover Decision Using Fuzzy MADM in Heterogeneous Networks,” Proc. IEEE Wireless Comm. and Networking Conf. (WCNC), vol. 2, pp. 653-658, 2004.
[8] W.-T. Chen and Y.-Y. Shu, “Active Application Oriented Vertical Handoff in Next-Generation Wireless Networks,” Proc. IEEE Wireless Comm. and Networking Conf. (WCNC), vol. 3, pp. 1383-1388, 2005.
[9] Q. Song and A. Jamalipour, “A Network Selection Mechanism for Next Generation Networks,” Proc. IEEE Int'l Conf. Comm. (ICC), vol. 2, pp. 1418-1422, 2005.
[10] E. Stevens-Navarro and V.W.S. Wong, “Comparison Between Vertical Handoff Decision Algorithms for Heterogeneous Wireless Networks,” Proc. IEEE 63rd Vehicular Technology Conf. (VTC '06-Spring), vol. 2, pp. 947-951, 2006.
[11] J. Ye, J. Hou, and S. Papavassiliou, “A Comprehensive Resource Management Framework for Next Generation Wireless Networks,” IEEE Trans. Mobile Computing, vol. 1, no. 4, pp. 249-264, Oct. 2002.
[12] D. Maldonado, B. Le, A. Hugine, T.W. Rondeau, and C.W. Bostian, “Cognitive Radio Applications to Dynamic Spectrum Allocation: A Discussion and an Illustrative Example,” Proc. First IEEE Int'l Symp. New Frontiers in Dynamic Spectrum Access Networks (DySPAN '05), pp. 597-600, 2005.
[13] H. Chan, P. Fan, and Z. Cao, “A Utility-Based Network Selection Scheme for Multiple Services in Heterogeneous Networks,” Proc. Int'l Conf. Wireless Networks, Comm., and Mobile Computing, vol. 2, pp. 1175-1180, 2005.
[14] W. Song, H. Jiang, W. Zhuang, and A. Saleh, “Call Admission Control for Integrated Voice/Data Services in Cellular/WLAN Interworking,” Proc. IEEE Int'l Conf. Comm. (ICC), vol. 12, pp.5480-5485, 2006.
[15] N.R. Suri and Y. Narahari, “An Auction Algorithm for Procuring Wireless Channel in a Heterogeneous Wireless Network,” Proc. IEEE Int'l Federation for Information Processing (IFIP) Int'l Conf. Wireless and Optical Comm. Networks, pp. 1-5, 2006.
[16] Y.-C. Lai and S.-F. Tsai, “Unfairness of Measurement-Based Admission Controls in a Heterogeneous Environment,” Proc. Int'l Conf. Parallel and Distributed Systems (ICPADS), pp. 667-674, 2001.
[17] Y.-R. Huang and J.-M. Ho, “Distributed Call Admission Control for a Heterogeneous PCS Network,” IEEE Trans. Computers, vol. 51, no. 12, pp. 1400-1409, Dec. 2002.
[18] W. Song and W. Zhuang, “QoS Provisioning via Admission Control in Cellular/Wireless LAN Interworking,” Proc. Int'l Conf. Broadband Networks, vol. 1, pp. 543-550, 2005.
[19] D. Niyato and E. Hossain, “A Cooperative Game Framework for Bandwidth Allocation in 4G Heterogeneous Wireless Networks,” Proc. IEEE Int'l Conf. Comm. (ICC), vol. 9, pp. 4357-4362, 2006.
[20] F. Yu and V. Krishnamurthy, “Optimal Joint Session Admission Control in Integrated WLAN and CDMA Cellular Networks with Vertical Handoff,” IEEE Trans. Mobile Computing, vol. 6, no. 1, pp.126-139, Jan. 2007.
[21] C.J.C.H. Watkins and P. Dayan, “Technical Note: Q-Learning,” Machine Learning, vol. 8, no. 3, pp. 279-292, 1992.
[22] L. Jouffe, “Fuzzy Inference System Learning by Reinforcement Methods,” IEEE Trans. Systems, Man, and Cybernetics—Part C: Applications and Rev., vol. 28, no. 3, pp. 338-355, Aug. 1998.
[23] M. Willem, Minimax Theorems. Birkhäuser Boston, 1996.
[24] C.T. Lin and C.S.G. Lee, Neural Fuzzy Systems. Prentice-Hall PTR, 1996.
[25] S. Haykin, Neural Networks, A Comprehensive Foundation. Prentice-Hall PTR, 1994.
[26] T.S. Rappaport, Wireless Communications, Principles, and Practice. Prentice-Hall PTR, 1996.
[27] G.L. Stüber, Principles of Mobile Communication. Kluwer Academic Publishers, 1996.
[28] I.F. Akyildiz, Y.-B. Lin, W.-R. Lai, and R.-J. Chen, “A New Random Walk Model for PCS Networks,” IEEE J. Selected Areas in Comm., vol. 18, no. 7, pp. 1254-1260, July 2000.
[29] 3rd Generation Partnership Project; Physical Channels and Mapping of Transport Channels onto Physical Channels (FDD), 3GPP Std. TS25.211, www.3gpp.org, Dec. 2005.
[30] 3rd Generation Partnership Project; MAC Protocol Specification, 3GPP Std. TS25.321, www.3gpp.org, Oct. 2005.
[31] 3rd Generation Partnership Project; Technical Specification Group Radio Access Network; Physical Layer—Measurements (FDD) (Release8), 3GPP Std. TS25.215 V8.0.0, www.3gpp.org, Mar. 2008.
[32] 3rd Generation Partnership Project; Technical Specification Group Radio Access Network; Physical Layer; Measurements (TDD) (Release8), 3GPP Std. TS25.225 V8.0.0, www.3gpp.org, Mar. 2008.
[33] 3rd Generation Partnership Project; Technical Specification Group Radio Access Network; Radio Resource Management Strategies (Release7), 3GPP Std. TS25.922 V7.1.0, www.3gpp.org, Mar. 2007.
[34] IEEE Std. 802.11e Amendment 8: Medium Access Control (MAC) Quality of Service Enhancements, IEEE, Nov. 2005.
[35] IEEE Std. 802.11a: High-Speed Physical Layer in the 5 GHz Band, IEEE, 1999.
[36] 3rd Generation Partnership Project; Quality of Service (QoS) and Policy Aspects of 3GPP—Wireless Local Area Network (WLAN) Interworking (Release 7), 3GPP Std. TS23.836 V1.1.0, www.3gpp.org, Nov. 2005.
[37] IEEE Std. 802.11k/D13.0 Amendment 1: Radio Resource Measurement of Wireless LANs, IEEE, Mar. 2008.
[38] IEEE Std. P802.11.2/D1.01 Draft Recommended Practice for the Evaluation of 802.11 Wireless Performance, IEEE, Feb. 2008.
[39] A. Mehbodniya and J. Chitizadeh, “An Intelligent Vertical Handoff Algorithm for Next Generation Wireless Networks,” Proc. Wireless and Optical Comm. Networks (WOCN '05), pp. 244-249, 2005.
[40] H.R. Berenji, “Fuzzy Q-Learning: A New Approach for Fuzzy Dynamic Programming,” Proc. Third IEEE Conf. Fuzzy Systems, vol. 1, pp. 486-491, 1994.
[41] L.A. Zadeh, “Outline of a New Approach to the Analysis of Complex Systems and Decision Processes,” IEEE Trans. Systems, Man, and Cybernetics, vol. SMC-3, no. 1, pp. 28-44, Jan. 1973.
[42] S.B. Thrun, Efficient Exploration in Reinforcement Learning, Technical Report CMU-CS-92-102, 1992.
[43] Y.-H. Chen, “Call Admission Control for WCDMA/WLAN Heterogeneous Networks Using Fuzzy Logic Theorem,” PhD dissertation, Nat'l Chiao Tung Univ., 2007.
[44] Universal Mobile Telecommunications System (UMTS) Std. TR 101 112 v3.2.0 (1998-04); Selection Procedures for the Choice of Radio Transmission Technologies of the UMTS (UMTS 30.03 Version 3.2.0), UMTS, 1998.
[45] 3rd Generation Partnership Project; Technical Specification Group Radio Access Network; User Equipment (UE) Radio Transmission and Reception (FDD) (Release 6), 3GPP Std. TS 25.101 v6.3.0 (2003-12), 2003.
[46] Y.-H. Chen and C.-J. Chang, “A Fast Handoff Protocol for Cellular IEEE 802.11e WLAN Systems,” IEEE Comm. Letters, vol. 11, no. 2, pp. 140-142, Feb. 2007.
[47] G. Neumann, “Reinforcement Learning Toolbox,” www.igi. tugraz.at/ril-toolbox/generaloverview.html , 2009.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool