Security access control systems and automatic video surveillance systems are becoming increasingly important recently. And tracking human faces is one of the indispensable processes. In this paper, we propose a real-time multiple-person tracking system. Firstly, both the skin colour and motion components are extracted to generate face-like regions. These regions regarded as face candidates will be further processed by a silhouette analyser to redefine their boundary and verified by our rule-based verification algorithm. Finally a simple method which utilize the relationship of skin colour and face history is applied to track multiple persons. And experiment results demonstrate that both the accuracy and processing speed are very promising and can be applied for the practical use.