Issue No. 01 - January (2008 vol. 20)
ISSN: 1041-4347
pp: 1-12
ABSTRACT
Linear Discriminant Analysis (LDA) has been a popular method for extracting features which preserve class separability. It has been widely used in many fields of information processing. However, the computation of LDA involves dense matrices eigen-decomposition which can be computationally expensive both in time and memory. Specifically, LDA has $O(mnt+t^3)$ time complexity and requires $O(mn+mt+nt)$ memory, where $m$ is the number of samples, $n$ is the number of features and $t=\min(m,n)$. When both $m$ and $n$ are large, it is infeasible to apply LDA. In this paper, we propose a novel algorithm for discriminant analysis, called {\em Spectral Regression Discriminant Analysis} (SRDA). By using spectral graph analysis, SRDA casts discriminant analysis into a regression framework which facilitates both efficient computation and the use of regularization techniques. Specifically, SRDA only needs to solve a set of regularized least squares problems and there is no eigenvector computation involved, which is a huge save of both time and memory. Our theoretical analysis shows that SRDA can be computed with $O(ms)$ time and $O(ms)$ memory, where $s (\leq n)$ is the average number of non-zero features in each sample. Extensive experimental results on four real world data sets demonstrate the effectiveness and efficiency of our algorithm.
INDEX TERMS
Data mining, Feature evaluation and selection
CITATION

J. Han, X. He and D. Cai, "SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis," in IEEE Transactions on Knowledge & Data Engineering, vol. 20, no. , pp. 1-12, 2007.
doi:10.1109/TKDE.2007.190669