Web Intelligence and Intelligent Agent Technology, IEEE/WIC/ACM International Conference on (2009)
Sept. 15, 2009 to Sept. 18, 2009
Distributed Partially Observable Markov Decision Problems (Dis-POMDPs) are emerging as a popular approach for modeling sequential decision making in teams operating under uncertainty. To achieve coherent behaviors of agents, it is essential to perform appropriate run-time communication. Thus, there have been many works on the run-time communication schemes in Dis-POMDPs. Also, a Finite State Machine (FSM) is a popular representation for describing a local policy that works in a very long or infinite time horizon. In this paper, we examine a run-time communication scheme when the local policy of each agent is represented as an FSM. In this scheme, the meaning of each message is not predefined; it is given implicitly by the interaction between local policies. We propose an iterative-improvement type algorithm that searches for a joint policy where run-time communication incurs some cost. Thus, agents use runtime communication only when doing so is cost-effective. Interestingly, our algorithm can find a joint policy that obtains a better expected reward than a hand-crafted joint policy, and it requires fewer nodes in the local FSM and fewer message types. Furthermore, we experimentally show that our algorithm can obtain a joint policy that consists of sufficiently complex local FSMs within a reasonable amount of time.
A. Iwasaki, Y. Sakurai, M. Tasaki, M. Yokoo and Y. Iwanari, "Introducing Communication in Dis-POMDPs with Finite State Machines," 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), Milan, Italy, 2009, pp. 267-270.