In this paper we discuss how to use a one-dimensional classifier for solving high dimensional classification problems. We propose Spin Discriminant Analysis (SDA), which enables us to construct a family of new classifiers. We prove that SDA is equivalent to ridged Linear Discriminant Analysis (LDA) when two classes are Gaussians with common covariance matrices. Moreover, we prove that classification based on Parzen?s window is a special case of SDA. In addition to theoretical investigations, we conduct extensive empirical studies, implementing SDA using Support Vector Machines (SVMs) as its one-dimensional classifiers. This SVM-based SDA implementation is named SpinSVM. Our experiments show that SpinSVM outperforms traditional high dimensional classifiers like SVMs, Classification Using Spline (CUS), classification-based Parzen?s window, and LDA on most standard and synthetic datasets we tested.