The advantage of parallel computing of artificial neural networks can be combined with the potentials of VLSI circuits in order to design a real time detection and tracking system applied to video images. Based on these facts, a real-time localization and tracking algorithm has been developed for detecting human hands in video images. Due to the real time aspect, a single-pixel-based classification is aspired, so that a continuous data stream can be processed. Consequently, no storage of full images or parts of them is necessary. The classification, whether a pixel belongs to a hand or to the background, is done by analyzing the RGB-values of a single pixel by means of an artificial neural network. To obtain the full processing speed of this neural network a hardware solution is realized in a Field Programmable Gate Array (FPGA).