Support vector machine (SVM) has been proved to be a powerful tool for solving practical pattern recognition problems based on learning from data. Due to large number of support vectors learnt from huge amount of training data the SVM becomes too computational intensive to many critical problems. In this paper we develop a reliable reduced set vectors method to speed up the SVM with Gaussian kernel. A set of reduced vector pairs (RVPs) are calculated from the support vectors. In the case of face detection, by considering the RVPs sequentially, if at any point a window is deemed too unlikely to cease the sequential evaluation, obviating the need to evaluate the remaining RVPs so that we only need to apply a subset of the RVPs to eliminate things that are obviously not a face.