This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Discriminant Subspace Analysis: A Fukunaga-Koontz Approach
October 2007 (vol. 29 no. 10)
pp. 1732-1745
The Fisher Linear Discriminant (FLD) is commonly used in pattern recognition. It finds a linear subspace that maximally separates class patterns according to the Fisher Criterion. Several methods of computing the FLD have been proposed in the literature, most of which require the calculation of the so-called scatter matrices. In this paper, we bring a fresh perspective to FLD via the Fukunaga-Koontz Transform (FKT). We do this by decomposing the whole data space into four subspaces with different discriminability, as measured by eigenvalue ratios. By connecting the eigenvalue ratio with the generalized eigenvalue, we show where the Fisher Criterion is maximally satisfied. We prove the relationship between FLD and FKT analytically, and propose a unified framework to understanding some existing work. Furthermore, we extend our our theory to Multiple Discriminant Analysis (MDA). This is done by transforming the data into intra- and extra-class spaces, followed by maximizing the Bhattacharyya distance. Based on our FKT analysis, we identify the discriminant subspaces of MDA/FKT, and propose an efficient algorithm, which works even when the scatter matrices are singular, or too large to be formed. Our method is general and may be applied to different pattern recognition problems. We validate our method by experimenting on synthetic and real data.

[1] E. Bailly-Bailliére, S. Bengio, F. Bimbot, M. Hamouz, J. Kittler, J. Mariéthoz, J. Matas, K. Messer, V. Popovici, F. Porée, B. Ruíz, and J.P. Thiran, “The Banca Database and Evaluation Protocol,” Proc. Fourth Int'l Conf. Audio and Video-Based Biometric Person Authentication, pp. 625-638, 2003.
[2] P. Belhumeur, J. Hespanha, and D. Kriegman, “Eigenfaces versus Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, July 1997.
[3] H. Cevikalp, M. Neamtu, M. Wilkes, and A. Barkana, “Discriminant Common Vectors for Face Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 4-13, Jan. 2005.
[4] S. Chakrabarti, S. Roy, and M. Soundalgekar, “Fast and Accurate Text Classification via Multiple Linear Discriminant Projections,” Proc. Int'l Conf. Very Large Data Bases, pp. 658-669, 2002.
[5] R. Duda, P. Hart, and D. Stork, Pattern Classification, second ed. John Wiley & Sons, 2000.
[6] F. de la Torre Frade and T. Kanade, “Multimodal Oriented Discriminant Analysis,” Proc. Int'l Conf. Machine Learning, Aug. 2005.
[7] K. Fukunaga, Introduction to Statistical Pattern Recognition, second ed. Academic Press, 1990.
[8] G.H. Golub and C.F. Van Loan, Matrix Computations, third ed. Johns Hopkins Univ. Press, 1996.
[9] P. Howland and H. Park, “Generalizing Discriminant Analysis Using the Generalized Singular Value Decomposition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp.995-1006, Aug. 2004.
[10] R. Huang, Q. Liu, H. Lu, and S. Ma, “Solving the Small Sample Size Problem of LDA,” Proc. IEEE Int'l Conf. Pattern Recognition, vol. 3, pp. 29-32, 2002.
[11] X. Huo, “A Statistical Analysis of Fukunaga-Koontz Transform,” IEEE Signal Processing Letters, vol. 11, no. 2, pp. 123-126, Feb. 2004.
[12] A.K. Jain, R.P.W. Duin, and J. Mao, “Statistical Pattern Recognition: A Review,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4-37, Jan. 2000.
[13] I.T. Jolliffe, Principal Component Analysis. Springer, 1986.
[14] C.F. Van Loan, “Generalizing the Singular Value Decomposition,” SIAM J. Numerical Analysis, vol. 13, no. 1, pp. 76-83, 1976.
[15] B. Moghaddam, “Principal Manifolds and Probabilistic Subspaces for Visual Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 780-788, June 2002.
[16] C.C. Paige and M.A. Saunders, “Towards a Generalized Singular Value Decomposition,” SIAM J. Numerical Analysis, vol. 18, no. 3, pp. 398-405, 1981.
[17] T. Sim, S. Baker, and M. Bsat, “The CMU Pose, Illumination, and Expression Database,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[18] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J.Cognitive Neuroscience, vol. 3, no. 1, 1991.
[19] X. Wang and X. Tang, “Dual-Space Linear Discriminant Analysis for Face Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 564-569, June 2004.
[20] J. Ye and Q. Li, “A Two-Stage Linear Discriminant Analysis via QR-Decomposition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 929-942, June 2005.
[21] H. Yu and H. Yang, “A Direct LDA Algorithm for High-Dimensional Data—with Application to Face Recognition,” Pattern Recognition, vol. 34, no. 10, pp. 2067-2070, 2001.
[22] S. Zhang and T. Sim, “When Fisher Meets Fukunaga-Koontz: A New Look at Linear Discriminants,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 323-329, June 2006.

Index Terms:
discriminant subspace analysis, Fukunaga-Koontz transform, pattern classification
Citation:
Sheng Zhang, Terence Sim, "Discriminant Subspace Analysis: A Fukunaga-Koontz Approach," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1732-1745, Oct. 2007, doi:10.1109/TPAMI.2007.1089
Usage of this product signifies your acceptance of the Terms of Use.