Issue No. 11 - Nov. (2013 vol. 12)
ISSN: 1536-1233
pp: 2155-2166
Xianfu Chen , VTT Technical Research Centre of Finland, Oulu
Zhifeng Zhao , Zhejiang University, Hangzhou
Honggang Zhang , Zhejiang University, Hangzhou
ABSTRACT
As the scarce spectrum resource is becoming overcrowded, cognitive radio indicates great flexibility to improve the spectrum efficiency by opportunistically accessing the authorized frequency bands. One of the critical challenges for operating such radios in a network is how to efficiently allocate transmission powers and frequency resource among the secondary users (SUs) while satisfying the quality-of-service constraints of the primary users. In this paper, we focus on the noncooperative power allocation problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of energy efficiency. Due to the SUs' dynamic and spontaneous properties, the problem is modeled as a stochastic learning process. We first extend the single-agent $(Q)$-learning to a multiuser context, and then propose a conjecture-based multiagent $(Q)$-learning algorithm to achieve the optimal transmission strategies with only private and incomplete information. An intelligent SU performs $(Q)$-function updates based on the conjecture over the other SUs' stochastic behaviors. This learning algorithm provably converges given certain restrictions that arise during the learning procedure. Simulation experiments are used to verify the performance of our algorithm and demonstrate its effectiveness of improving the energy efficiency.
INDEX TERMS
Resource management, Interference, Games, Stochastic processes, Signal to noise ratio, Algorithm design and analysis, Wireless communication,reinforcement learning, Cognitive radio, resource allocation, algorithm/protocol design and analysis
CITATION
Xianfu Chen, Zhifeng Zhao, Honggang Zhang, "Stochastic Power Adaptation with Multiagent Reinforcement Learning for Cognitive Wireless Mesh Networks", IEEE Transactions on Mobile Computing, vol. 12, no. , pp. 2155-2166, Nov. 2013, doi:10.1109/TMC.2012.178