The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - April (2011 vol.23)
pp: 600-611
Wei Zhang , The Chinese University of Hong Kong, Hong Kong
Zhouchen Lin , Microsoft Research Asia, Beijing
Xiaoou Tang , The Chinese University of Hong Kong, Beijing
ABSTRACT
Discriminant feature extraction plays a central role in pattern recognition and classification. Linear Discriminant Analysis (LDA) is a traditional algorithm for supervised feature extraction. Recently, unlabeled data have been utilized to improve LDA. However, the intrinsic problems of LDA still exist and only the similarity among the unlabeled data is utilized. In this paper, we propose a novel algorithm, called Semisupervised Semi-Riemannian Metric Map ({\rm S}^3RMM), following the geometric framework of semi-Riemannian manifolds. {\rm S}^3RMM maximizes the discrepancy of the separability and similarity measures of scatters formulated by using semi-Riemannian metric tensors. The metric tensor of each sample is learned via semisupervised regression. Our method can also be a general framework for proposing new semisupervised algorithms, utilizing the existing discrepancy-criterion-based algorithms. The experiments demonstrated on faces and handwritten digits show that {\rm S}^3RMM is promising for semisupervised feature extraction.
INDEX TERMS
Linear discriminant analysis, semisupervised learning, semi-Riemannian manifolds, feature extraction.
CITATION
Wei Zhang, Zhouchen Lin, Xiaoou Tang, "Learning Semi-Riemannian Metrics for Semisupervised Feature Extraction", IEEE Transactions on Knowledge & Data Engineering, vol.23, no. 4, pp. 600-611, April 2011, doi:10.1109/TKDE.2010.143
REFERENCES
[1] P.N. Belhumeur, J.P. Hespanha, and D.J. Kriegman, "Eigenfaces versus Fisherfaces: Recognition Using Class Specific Linear Projection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[2] M. Belkin and P. Niyogi, "Laplacian Eigenmaps for Dimensionality Reduction and Data Representation," Neural Computation, vol. 15, pp. 1373-1396, 2003.
[3] M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples," J. Machine Learning Research, vol. 7, pp. 2399-2434, 2006.
[4] Y. Bengio, J. Paiement, P. Vincent, O. Delalleau, N. Le Roux, and M. Ouimet, "Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering," Advances in Neural Information Processing Systems, MIT Press, 2003.
[5] A. Blum and T. Mitchell, "Combining Labeled and Unlabeled Data with Co-Training," Proc. Workshop Computational Learning Theory (COLT), 1998.
[6] D. Cai, X. He, and J. Han, "Semi-Supervised Discriminant Analysis," Proc. Int'l Conf. Computer Vision, 2007.
[7] J. Friedman, "Regularized Discriminant Analysis," J. Am. Statistical Assoc., vol. 84, no. 405, pp. 165-175, 1989.
[8] J. Ham, D. Lee, S. Mika, and B. Schölkopf, "A Kernel View of the Dimensionality Reduction of Manifolds," Proc. Int'l Conf. Machine Learning, 2004.
[9] B. Hassibi, A. Sayed, and T. Kailath, "Linear Estimation in Krein Spaces-Part I: Theory," IEEE Trans. Automatic Control, vol. 41, no. 1, pp. 18-33, Jan. 1996.
[10] T. Hastie, A. Buja, and R. Tibshirani, "Penalized Discriminant Analysis," The Annals of Statistics, vol. 23, no. 1, pp. 73-102, 1995.
[11] X. He and P. Niyogi, "Locality Preserving Projections," Advances in Neural Information Processing Systems, MIT Press, 2003.
[12] S.C.H. Hoi, W. Liu, M.R. Lyu, and W.-Y. Ma, "Learning Distance Metrics with Contextual Constraints for Image Retrieval," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '06), 2006.
[13] P. Howland and H. Park, "Generalizing Discriminant Analysis Using the Generalized Singular Value Decomposition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 995-1006, Aug. 2004.
[14] I. Iokhvidov, M. Krein, and H. Langer, Introduction to the Spectral Theory of Operators in Spaces with an Indefinite Metric. Akademie-Verlag, 1982.
[15] M. Kowalski, M. Szafranski, and L. Ralaivola, "Multiple Indefinite Kernel Learning with Mixed Norm Regularization," Proc. Int'l Conf. Machine Learning, 2009.
[16] H. Li, T. Jiang, and K. Zhang, "Efficient and Robust Feature Extraction by Maximum Margin Criterion," IEEE Trans. Neural Networks, vol. 17, no. 1, pp. 157-165, Jan. 2006.
[17] G. Liu, Z. Lin, and Y. Yu, "Learning Semi-Riemannian Manifolds for Unsupervised Dimensionality Reduction," submitted to Pattern Recognition.
[18] Q. Liu, X. Tang, H. Lu, and S. Ma, "Face Recognition Using Kernel Scatter-Difference-Based Discriminant Analysis," IEEE Trans. Neural Networks, vol. 17, no. 4, pp. 1081-1085, July 2006.
[19] K. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf, "An Introduction to Kernel-Based Learning Algorithms," IEEE Trans. Neural Networks, vol. 12, no. 2, pp. 181-201, Mar. 2001.
[20] F. Nie, S. Xiang, Y. Jia, and C. Zhang, "Semi-Supervised Orthogonal Discriminant Analysis via Label Propagation," Pattern Recognition, vol. 42, no. 11, pp. 2615-2627, 2009.
[21] B. O'Neill, Semi-Riemannian Geometry with Application to Relativity. Academic Press, 1983.
[22] C. Ong, X. Mary, S. Canu, and A. Smola, "Learning with Non-Positive Kernels," Proc. Int'l Conf. Machine Learning, 2004.
[23] E. Pekalska and B. Haasdonk, "Kernel Discriminant Analysis for Positive Definite and Indefinite Kernels," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 6, pp. 1017-1032, June 2009.
[24] P.J. Phillips, P.J. Flynn, T. Scruggs, K.W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, "Overview of the Face Recognition Grand Challenge," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '05), 2005.
[25] X. Qiu and L. Wu, "Face Recognition by Stepwise Nonparametric Margin Maximum Criterion," Proc. Int'l Conf. Computer Vision, 2005.
[26] L.K. Saul, S.T. Roweis, and Y. Singer, "Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifolds," J. Machine Learning Research, vol. 4, pp. 119-155, 2003.
[27] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression Database," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[28] V. Sindhwani, P. Niyogi, and M. Belkin, "Beyond the Point Cloud: From Transductive to Semi-Supervised Learning," Proc. Int'l Conf. Machine Learning, 2005.
[29] Y. Song, F. Nie, C. Zhang, and S. Xiang, "A Unified Framework for Semi-Supervised Dimensionality Reduction," Pattern Recognition, vol. 41, no. 9, pp. 2789-2799, 2008.
[30] J.B. Tenenbaum, V. de Silva, and J.C. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, pp. 2319-2323, 2000.
[31] V. Vapnik, Statistical Learning Theory. John Wiley, 1998.
[32] F. Wang and C. Zhang, "Feature Extraction by Maximizing the Average Neighborhood Margin," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '07), 2007.
[33] H. Wang, W. Zheng, Z. Hu, and S. Chen, "Local and Weighted Maximum Margin Discriminant Analysis," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '07), 2007.
[34] H. Wang, S. Yan, D. Xu, X. Tang, and T. Huang, "Trace Ratio versus Ratio Trace for Dimensionality Reduction," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '07), 2007.
[35] X. Wang and X. Tang, "Unified Subspace Analysis for Face Recognition," Proc. Int'l Conf. Computer Vision, 2003.
[36] X. Wang and X. Tang, "Dual-Space Linear Discriminant Analysis for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '04), 2004.
[37] X. Wang and X. Tang, "Random Sampling LDA for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '04), 2004.
[38] X. Wang and X. Tang, "A Unified Framework for Subspace Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1222-1228, Sept. 2004.
[39] X. Wang and X. Tang, "Random Sampling for Subspace Face Recognition," Int'l J. Computer Vision, vol. 70, no. 1, pp. 91-104, 2006.
[40] S. Yan, D. Xu, Q. Yang, L. Zhang, X. Tang, and H.-J. Zhang, "Multilinear Discriminant Analysis for Face Recognition," IEEE Trans. Image Processing, vol. 16, no. 1, pp. 212-220, Jan. 2007.
[41] S. Yan, D. Xu, B. Zhang, H.-J. Zhang, Q. Yang, and S. Lin, "Graph Embedding and Extensions: A General Framework for Dimensionality Reduction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 40-51, Jan. 2007.
[42] J. Yang, A. Frangi, J. Yang, D. Zhang, and Z. Jin, "KPCA Plus LDA: A Complete Kernel Fisher Discriminant Framework for Feature Extraction and Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 2, pp. 230-244, Feb. 2005.
[43] W. Yang, S. Zhang, and W. Liang, "A Graph Based Subspace Semi-Supervised Learning Framework for Dimensionality Reduction," Proc. European Conf. Computer Vision, 2008.
[44] J. Ye, R. Janardan, and Q. Li, "Two-Dimensional Linear Discriminant Analysis," Advances in Neural Information Processing Systems, MIT Press, 2004.
[45] D. Zhang, Z.-H. Zhou, and S. Chen, "Semi-Supervised Dimensionality Reduction," Proc. SIAM Int'l Conf. Data Mining, 2007.
[46] T. Zhang, D. Tao, and J. Yang, "Discriminative Locality Alignment," Proc. European Conf. Computer Vision, 2008.
[47] W. Zhang, Z. Lin, and X. Tang, "Tensor Linear Laplacian Discrimination (TLLD) for Feature Extraction," Pattern Recognition, vol. 42, no. 9, pp. 1941-1948, 2009.
[48] Y. Zhang and D.-Y. Yeung, "Semi-Supervised Discriminant Analysis Using Robust Path-Based Similarity," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[49] Z. Zhang and H. Zha, "Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment," SIAM J. Scientific Computing, vol. 26, no. 1, pp. 313-338, 2005.
[50] D. Zhao, Z. Lin, and X. Tang, "Classification via Semi-Riemannian Spaces," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '08), 2008.
[51] D. Zhao, Z. Lin, R. Xiao, and X. Tang, "Linear Laplacian Discrimination for Feature Extraction," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '07), 2007.
[52] D. Zhou, O. Bousquet, T.N. Lal, J. Weston, and B. Schölkopf, "Learning with Local and Global Consistency," Advances in Neural Information Processing Systems, pp. 321-328, MIT Press, 2004.
[53] X. Zhu, "Semi-Supervised Learning Literature Survey," technical report, Computer Science Dept., Univ. of Wisconsin-Madison, 2005.
29 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool