Support Vector Machine (SVM) has become a popular classification tool but one of its disadvantages is large memory requirement and computation time when dealing with large datasets. Parallel methods have been proposed to speed up the process of training SVM. An improved cascade SVM training algorithm is proposed, in which multiple SVM classifiers are applied. The support vectors are obtained by feeding back in a crossed way, alternating to avoid the problem that the learning results are subject to the distribution state of the data samples in different subsets. The experiment results on UCI dataset show that this parallel SVM training algorithm is efficient and has more satisfying accuracy compared with standard cascade SVM algorithm in classification precision.