The information available to robots in real tasks is widely distributed both in time and space, requiring the agent to search for relevant information. In this work, we implement a solution that uses qualitative and quantitative knowledge to turn robot tasks able to be treated by Reinforcement Learning (RL) algorithms. The steps of this procedure include: 1) to decompose the overall task into smaller ones, using abstraction and macrooperators, thus achieving a discrete action space; 2) to use observation functions of the environment ? here called features - to achieve both time and state space discretisation; 3) to use quantitative knowledge to design controllers that are able to solve the subtasks; 4) to learn the coordination of these behaviours using RL, more specifically Q-learning. The approach was verified on an increasingly complex set of robot tasks using a Khepera robot simulator. Two approaches for space discretisation were used, one based on features and the other on states. The learned policies over these two models were compared to a predefined hand-crafted one. It was found that the learned policy over the state-based discretisation leads quickly to good results, although it can not be applied to complex tasks, where the state space representation becomes computationally unfeasible.