This paper proposes a framework of face tracking with classification, which can better meet the real requirements in the surveillance systems. Face tracking is performed by a novel constrained CAMShift algorithm, namely CAMShift- C, by posing three restrict conditions, including evaluation of location accuracy, scale of face area and dynamic histogram updating. The advantages of LBP-based face classification include: 1) solving the occlusion problem by given each face a fixed label; 2) reducing the space complexity due to non-repeating storage of the face; 3) shortening the runtime since only the new face is needed to match with the template. Extensive experimental results demonstrate that, not only face tracking can provide face-of-interest for classification, but simultaneously the accuracy of face tracking is enhanced by face classification, especially in the cases of clutter background and the occurrence of occlusion. More encouragingly, beyond the high performance, the framework also can achieve real-time monitoring.