This Article 
 Bibliographic References 
 Add to: 
Multispace KL for Pattern Representation and Classification
September 2001 (vol. 23 no. 9)
pp. 977-996

—This work introduces the Multispace KL (MKL) as a new approach to unsupervised dimensionality reduction for pattern representation and classification. The training set is automatically partitioned into disjoint subsets, according to an optimality criterion; each subset then determines a different KL subspace which is specialized in representing a particular group of patterns. The extension of the classical KL operators and the definition of ad hoc distances allow MKL to be effectively used where KL is commonly employed. The limits of the standard KL transform are pointed out, in particular, MKL is shown to outperform KL when the data distribution is far from a multidimensional Gaussian and to better cope with large sets of patterns, which could cause a severe performance drop in KL.

[1] R. Cappelli, D. Maio, and D. Maltoni, “Similarity Search Using Multi-Space KL,” Proc. First Int'l Workshop Similarity Search- Database and Expert Systems Applications, pp. 155-160, 1999.
[2] R. Cappelli, D. Maio, and D. Maltoni, “Fingerprint Classification Based on Multi-Space KL,” Proc. Workshop Automatic Identification Advanced Technologies '99, pp. 117-120, 1999.
[3] R. Cappelli, D. Maio, and D. Maltoni, “Combining Fingerprint Classifiers,” Proc. First Int'l Workshop Multiple Classifier Systems (MCS 2000), pp. 351-361, 2000.
[4] B. Chalmond and S.C. Girard, “Nonlinear Modeling of Scattered Multivariate Data and Its Application to Shape Change,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 422- 432, May 1999.
[5] S. Chandrasekaran, B.S. Manjunath, Y.F. Wang, J. Winkeler, and H. Zhang, “An Eigenspace Update Algorithm for Image Analysis,” CVGIP: Graphical Models and Image Processing, no. 6, Apr. 1996.
[6] H. Frigui and R. Krishnapuram, “A Robust Competitive Clustering Algorithm with Applications in Computer Visions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 450- 465, May 1999.
[7] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[8] Y. Hamamoto, Y. Fujimoto, and S. Tomita, “On the Estimation of a Covariance Matrix in Designing Parzen Classifiers,” Pattern Recognition, vol. 29, no. 10, pp. 1751-1759, 1996.
[9] J.A Hartigan,Clustering Algorithms, John Wiley and Sons, New York, N.Y., 1975.
[10] T. Hastie and W. Stuetzle, “Principal Curves,” J. Am. Statistical Assoc., vol. 84, no. 406, pp. 502-516, 1989.
[11] H. Hotelling, “Analysis of a Complex of Statistical Variables into Principal Components,” J. Educational Psychology, 1933.
[12] J. Illingworth, E.J. Kittler, “The Adaptive Hough Transform,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 5, pp. 690-697, May 1987.
[13] J. Illingworth and J. Kitter, "A survey of Hough transform," CVGIP, vol. 44, pp. 87-116, 1988.
[14] A.K. Jain and R.C. Dubes, Algorithms for Clustering Data. Englewood Cliffs, N.J.: Prentice Hall, 1988.
[15] I.T. Jolliffe, Principle Component Analysis. New York: Springer-Verlag, 1986.
[16] K. Karhunen, “Uber Lineare Methoden in der Wahrscheinlichkeitsrechnung,” Ann. Academy Science Fennicae, vol. 37 pp. 3-79, 1946.
[17] M. Kirby and L. Sirovich,“Application of Karhunen-Loève procedure for the characterization of human faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 103-108, Jan. 1990.
[18] M.M. Loève, Probability Theory. Princeton, N.J.: Van Nostrand, 1955.
[19] G.J. McLachlan and T. Krishnan, The EM Algorithm and Extensions. New York: Wiley, 1997.
[20] H. Murakami and B.V.K. Kumar, “Efficient Calculation of Primary Images from a Set of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 4, no. 5, pp. 511-515, May 1982.
[21] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[22] A. Pentland, B. Moghaddam, and Starner, "View-Based and Modular Eigenspaces for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994, pp. 84-91.
[23] A. Pentland, R. Picard, and S. Sclaroff, “Photobook: Tools for Content-Based Manipulation of Image Databases,” Proc. SPIE Storage and Retrieval of Image&Video Databases II, 1994.
[24] R.A. Render and H.F. Walker, “Mixture Densities, Maximum Likelihood and the EM Algorithm,” SIAM Rev., vol. 26, no. 2, pp. 195-239, 1984.
[25] L. Sirovich and M. Kirby, “A Low-Dimensional Procedure for the Characterization of Human Faces,” J. Optical Soc. Am. A, vol. 4, no. 3, pp. 519-524, 1987.
[26] D.L. Swets and J. Weng, Using Discriminant Eigenfeatures for Image Retrieval IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 8, pp. 831-836, Aug. 1996.
[27] D.J. Swets and J. Weng, “Hierarchical Discriminant Analysis for Image Retrieval,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 386-401, May 1999.
[28] M. Uenohara and T. Kanade, "Use of Fourier and Karhunen-Loeve Decomposition for Fast Pattern Matching with a Large Set of Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 8, pp.891-898, Aug. 1997.
[29] L. Zhao and Y.-H. Yang, “An Efficient Agorithm to Compute Eigenimages in PCA-Based Vision Systems,” Pattern Recognition, no. 32, pp. 851-864, 1999.

R. Cappelli, D. Maio, D. Maltoni, "Multispace KL for Pattern Representation and Classification," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 9, pp. 977-996, Sept. 2001, doi:10.1109/34.955111
Usage of this product signifies your acceptance of the Terms of Use.