This Article 
 Bibliographic References 
 Add to: 
Rotational Linear Discriminant Analysis Technique for Dimensionality Reduction
October 2008 (vol. 20 no. 10)
pp. 1336-1347
Alok Sharma, Griffith University, Brisbane and University of the South Pacific, Fiji
Kuldip K. Paliwal, Griffith University, Brisbane
The linear discriminant analysis (LDA) technique is very popular in pattern recognition for dimensionality reduction. It is a supervised learning technique that finds a linear transformation such that the overlapping between the classes is minimum for the projected feature vectors in the reduced feature space. This overlapping, if present, adversely affects the classification performance. In this paper, we introduce prior to dimensionality-reduction transformation an additional rotational transform that rotates the feature vectors in the original feature space around their respective class centroids in such a way that the overlapping between the classes in the reduced feature space is further minimized. As a result, the classification performance significantly improves which is demonstrated using several data corpuses.

[1] H. Anton, Calculus. John Wiley & Sons, 1995.
[2] P.N. Belhumeur, J.P. Hespanha, and D.J. Kriegman, “Eigenfaces versus Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[3] C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
[4] C.L. Blake and C.J. Merz, UCI Repository of Machine Learning Databases, Dept. of Information and Computer Science, Univ. of California, Irvine,, 1998.
[5] H. Cevikalp, M. Neamtu, M. Wlkes, and A. Barkana, “Discriminative Common Vectors for Face Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 4-13, Jan. 2005.
[6] L.-F. Chen, H.-Y.M. Liao, M.-T. Ko, J.-C. Lin, and G.-J. Yu, “A New LDA-Based Face Recognition System Which Can Solve the Small Sample Size Problem,” Pattern Recognition, vol. 33, pp. 1713-1726, 2000.
[7] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. John Wiley & Sons, 1973.
[8] J.H. Friedman, “Regularized Discriminant Analysis,” J. Am. Statistical Assoc., vol. 84, pp. 165-175, 1989.
[9] K. Fukunaga, Introduction to Statistical Pattern Recognition. Academic Press, 1990.
[10] S.G. Garofalo, L.F. Lori, F.M. William, F.G. Jonathan, P.S. David, and D.L. Nancy, “The DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus CDROM,” NIST, 1986.
[11] Y. Guo, T. Hastie, and R. Tinshirani, “Regularized Discriminant Analysis and Its Application in Microarrays,” Biostatistics, vol. 8, no. 1, pp. 86-100, 2007.
[12] X. Huang, A. Acero, and H.-W. Hon, Spoken Language Processing. Prentice Hall, 2001.
[13] A. Hyvärinen, “Fast and Robust Fixed-Point Algorithms for Independent Component Analysis,” IEEE Trans. Neural Networks, vol. 10, no. 3, pp. 626-634, 1999.
[14] A. Hyvärinen and E. Oja, “A Fast Fixed-Point Algorithm for Independent Component Analysis,” Neural Computation, vol. 9, no. 7, pp. 1483-1492, 1997.
[15] A.K. Jain, R.P.W. Duin, and J. Mao, “Statistical Pattern Recognition: A Review,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4-37, Jan. 2000.
[16] W.J. Krzanowski, P. Jonathan, W.V. McCarthy, and M.R. Thomas, “Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data,” Applied Statistics, vol. 44, pp. 101-115, 1995.
[17] R. Lotlikar and R. Kothari, “Fractional-Step Dimensionality Reduction,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 623-627, June 2000.
[18] J. Lu, K.N. Plataniotis, and A.N. Venetsanopoulos, “Face Recognition Using LDA-Based Algorithms,” IEEE Trans. Neural Networks, vol. 14, no. 1, pp. 195-200, 2003.
[19] S. Raudys and R.P.W. Duin, “On Expected Classification Error of the Fisher Linear Classifier with Pseudo-Inverse Covariance Matrix,” Pattern Recognition Letters, vol. 19, nos. 5-6, pp. 385-392, 1998.
[20] D.A. Reynolds, T.F. Quatieri, and R.B. Dunn, “Speaker Verification Using Adapted Gaussian Mixture Models,” Digital Signal Processing, vol. 10, no. 1-3, pp. 19-41, 2000.
[21] C. Sanderson and K.K. Paliwal, “Identity Verification Using Speech and Face Information,” Digital Signal Processing, vol. 14, no. 5, pp. 449-480, 2004.
[22] A. Sharma, K.K. Paliwal, and G.C. Onwubolu, “Class-Dependent PCA, LDA and MDC: A Combined Classifier for Pattern Classification,” Pattern Recognition, vol. 39, no. 7, pp. 1215-1229, 2006.
[23] A. Sharma and K.K. Paliwal, “A Gradient Linear Discriminant Analysis for Small Sample Sized Problem,” Neural Processing Letters, vol. 27, no. 1, pp. 17-24, 2008.
[24] D.L. Swets and J. Weng, “Using Discriminative Eigenfeatures for Image Retrieval,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 831-836, Aug. 1996.
[25] Q. Tian, M. Barbero, Z.H. Gu, and S.H. Lee, “Image Classification by the Foley-Sammon Transform,” Optical Eng., vol. 25, no. 7, pp.834-840, 1986.
[26] S. Young, G. Evermann, T. Hain, D. Kershaw, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. Woodland, The HTK Book Version 3.2. Cambridge Univ., 2002.
[27] J. Ye, “Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems,” J. Machine Learning Research, vol. 6, pp. 483-502, 2005.
[28] H. Yu and J. Yang, “A Direct LDA Algorithm for High-Dimensional Data—With Application to Face Recognition,” Pattern Recognition, vol. 34, pp. 2067-2070, 2001.

Index Terms:
Pattern Recognition, Statistical
Alok Sharma, Kuldip K. Paliwal, "Rotational Linear Discriminant Analysis Technique for Dimensionality Reduction," IEEE Transactions on Knowledge and Data Engineering, vol. 20, no. 10, pp. 1336-1347, Oct. 2008, doi:10.1109/TKDE.2008.101
Usage of this product signifies your acceptance of the Terms of Use.