This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Stochastic Power Adaptation with Multiagent Reinforcement Learning for Cognitive Wireless Mesh Networks
Nov. 2013 (vol. 12 no. 11)
pp. 2155-2166
Xianfu Chen, VTT Technical Research Centre of Finland, Oulu
Zhifeng Zhao, Zhejiang University, Hangzhou
Honggang Zhang, Zhejiang University, Hangzhou
As the scarce spectrum resource is becoming overcrowded, cognitive radio indicates great flexibility to improve the spectrum efficiency by opportunistically accessing the authorized frequency bands. One of the critical challenges for operating such radios in a network is how to efficiently allocate transmission powers and frequency resource among the secondary users (SUs) while satisfying the quality-of-service constraints of the primary users. In this paper, we focus on the noncooperative power allocation problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of energy efficiency. Due to the SUs' dynamic and spontaneous properties, the problem is modeled as a stochastic learning process. We first extend the single-agent $(Q)$-learning to a multiuser context, and then propose a conjecture-based multiagent $(Q)$-learning algorithm to achieve the optimal transmission strategies with only private and incomplete information. An intelligent SU performs $(Q)$-function updates based on the conjecture over the other SUs' stochastic behaviors. This learning algorithm provably converges given certain restrictions that arise during the learning procedure. Simulation experiments are used to verify the performance of our algorithm and demonstrate its effectiveness of improving the energy efficiency.
Index Terms:
Resource management,Interference,Games,Stochastic processes,Signal to noise ratio,Algorithm design and analysis,Wireless communication,reinforcement learning,Cognitive radio,resource allocation,algorithm/protocol design and analysis
Citation:
Xianfu Chen, Zhifeng Zhao, Honggang Zhang, "Stochastic Power Adaptation with Multiagent Reinforcement Learning for Cognitive Wireless Mesh Networks," IEEE Transactions on Mobile Computing, vol. 12, no. 11, pp. 2155-2166, Nov. 2013, doi:10.1109/TMC.2012.178
Usage of this product signifies your acceptance of the Terms of Use.