In order to speed up support vector classification, a novel algorithm by the names of SVM Decision Tree is proposed in this paper. In the decision tree, several linear SVM are constructed which can achieve the highest detection rate on the negative samples, the negative samples which can be correctly classified by the hyperplane are removed from the original samples, and train one nonlinear SVM using the rest samples. In the test step, the root of tree is used as the first classification. We apply this algorithm to face detection, experiment results show that the speed up factor is large and with no loss in generalization performance.