Intelligent Agent Technology, IEEE / WIC / ACM International Conference on (2007)
Silicon Valley, California, USA
Nov. 2, 2007 to Nov. 5, 2007
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/IAT.2007.74
In the Q-Learning framework, the exploration of large environment is influenced by the time credit assignment problem. In this context, abstraction techniques may be used. Thus, multi-step actions (MSA) Q-Learning has been proposed to take advantage of the fact that few action switches are usually required in optimal policies. In this article, we propose the concept of inertial exploration, we apply a log-selection of the scales to MSA Q-Learning and we go further by proposing a dynamic time scale approach. We demonstrate that the same improvement in learning speed can be achieved without the full scales set. This improvement is shown on the mountain car problem and on a more realistic application of vehicle control.
J. Laumonier, D. Bergeron, C. Desjardins and B. Chaib-draa, "Reinforcement Learning with Inertial Exploration," Intelligent Agent Technology, IEEE / WIC / ACM International Conference on(IAT), Silicon Valley, California, USA, 2007, pp. 277-280.