This paper concerns the concept of using a Reinforcement Learning approach for PID adaptive auto tuning, which may greatly increase possible application of the controller. The proposed idea can be utilised for off-line and on-line control systems. Though it becomes really powerful in the practical on-line applications, where the environment is unknown and all information is gained only by interactions. In the paper the algorithm based on Q-learning is proposed. The properties of the algorithm are illustrated on a simple examples.