Issue No. 02 - Feb. (2015 vol. 37)
Finale Doshi-Velez , Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
David Pfau , Center for Theoretical Neuroscience, Columbia University, New York, NY, USA
Frank Wood , Department of Engineering, University of Oxford, Oxford, U.K.
Nicholas Roy , Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA
Making intelligent decisions from incomplete information is critical in many applications: for example, robots must choose actions based on imperfect sensors, and speech-based interfaces must infer a user’s needs from noisy microphone inputs. What makes these tasks hard is that often we do not have a natural representation with which to model the domain and use for choosing actions; we must learn about the domain’s properties while simultaneously performing the task. Learning a representation also involves trade-offs between modeling the data that we have seen previously and being able to make predictions about new data. This article explores learning representations of stochastic systems using Bayesian nonparametric statistics. Bayesian nonparametric methods allow the sophistication of a representation to scale gracefully with the complexity in the data. Our main contribution is a careful empirical evaluation of how representations learned using Bayesian nonparametric methods compare to other standard learning approaches, especially in support of planning and control. We show that the Bayesian aspects of the methods result in achieving state-of-the-art performance in decision making with relatively few samples, while the nonparametric aspects often result in fewer computations. These results hold across a variety of different techniques for choosing actions given a representation.
History, Hidden Markov models, Bayes methods, Computational modeling, Learning (artificial intelligence), Markov processes, Knowledge representation,HDP-HMM, Reinforcement Learning, POMDP,hierarchial Dirichlet process hidden Markov model, Artificial intelligence, machine learning, reinforcement learning, partially-observable Markov decision process
Finale Doshi-Velez, David Pfau, Frank Wood, Nicholas Roy, "Bayesian Nonparametric Methods for Partially-Observable Reinforcement Learning", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 37, no. , pp. 394-407, Feb. 2015, doi:10.1109/TPAMI.2013.191