The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.06 - June (2011 vol.33)
pp: 1147-1160
Tyng-Luh Liu , Academia Sinica, Taipei
Chiou-Shann Fuh , National Taiwan University, Taipei
ABSTRACT
In solving complex visual learning tasks, adopting multiple descriptors to more precisely characterize the data has been a feasible way for improving performance. The resulting data representations are typically high-dimensional and assume diverse forms. Hence, finding a way of transforming them into a unified space of lower dimension generally facilitates the underlying tasks such as object recognition or clustering. To this end, the proposed approach (termed MKL-DR) generalizes the framework of multiple kernel learning for dimensionality reduction, and distinguishes itself with the following three main contributions: First, our method provides the convenience of using diverse image descriptors to describe useful characteristics of various aspects about the underlying data. Second, it extends a broad set of existing dimensionality reduction techniques to consider multiple kernel learning, and consequently improves their effectiveness. Third, by focusing on the techniques pertaining to dimensionality reduction, the formulation introduces a new class of applications with the multiple kernel learning framework to address not only the supervised learning problems but also the unsupervised and semi-supervised ones.
INDEX TERMS
Dimensionality reduction, multiple kernel learning, object categorization, image clustering, face recognition.
CITATION
Tyng-Luh Liu, Chiou-Shann Fuh, "Multiple Kernel Learning for Dimensionality Reduction", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 6, pp. 1147-1160, June 2011, doi:10.1109/TPAMI.2010.183
REFERENCES
[1] F. Bach, G. Lanckriet, and M. Jordan, “Multiple Kernel Learning, Conic Duality, and the SMO Algorithm,” Proc. Int'l Conf. Machine Learning, 2004.
[2] A. Berg, T. Berg, and J. Malik, “Shape Matching and Object Recognition Using Low Distortion Correspondences,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 26-33, 2005.
[3] A. Berg and J. Malik, “Geometric Blur for Template Matching,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 607-614, 2001.
[4] O. Boiman, E. Shechtman, and M. Irani, “In Defense of Nearest-Neighbor Based Image Classification,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[5] A. Bosch, A. Zisserman, and X. Muñoz, “Image Classification Using Random Forests and Ferns,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[6] D. Cai, X. He, and J. Han, “Semi-Supervised Discriminant Analysis,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[7] J. Carreira and C. Sminchisescu, “Constrained Parametric Min-Cuts for Automatic Object Segmentation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[8] C.-P. Chen and C.-S. Chen, “Lighting Normalization with Generic Intrinsic Illumination Subspace for Face Recognition,” Proc. IEEE Int'l Conf. Computer Vision, pp. 1089-1096, 2005.
[9] H.-T. Chen, H.-W. Chang, and T.-L. Liu, “Local Discriminant Embedding and Its Variants,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 846-853, 2005.
[10] M. Christoudias, R. Urtasun, and T. Darrell, “Bayesian Localized Multiple Kernel Learning,” technical report, Electical Eng. and Computer Science Dept., Univ. of California, Berkeley, 2009.
[11] T. Cox and M. Cox, Multidimentional Scaling. Chapman & Hall, 1994.
[12] D. Dueck and B. Frey, “Non-Metric Affinity Propagation for Unsupervised Image Categorization,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[13] M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, and A. Zisserman, The PASCAL Visual Object Classes Challenge (VOC2007) Results, 2007.
[14] L. Fei-Fei, R. Fergus, and P. Perona, “Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories,” Proc. IEEE Computer Vision and Pattern Recognition Workshop Generative-Model Based Vision, 2004.
[15] B. Frey and D. Dueck, “Clustering by Passing Messages between Data Points,” Science, vol. 315, pp. 972-976, 2007.
[16] A. Frome, Y. Singer, and J. Malik, “Image Retrieval and Classification Using Local Distance Functions,” Advances in Neural Information Processing Systems, pp. 417-424, MIT Press, 2006.
[17] A. Frome, Y. Singer, F. Sha, and J. Malik, “Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[18] C. Galleguillos, B. McFee, S. Belongie, and G. Lanckriet, “Multi-Class Object Localization by Combining Local Contextual Interactions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[19] P. Gehler and S. Nowozin, “On Feature Combination for Multiclass Object Classification,” Proc. IEEE Int'l Conf. Computer Vision, 2009.
[20] M. Gönen and E. Alpaydin, “Localized Multiple Kernel Learning,” Proc. Int'l Conf. Machine Learning, pp. 352-359, 2008.
[21] K. Grauman and T. Darrell, “The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features,” Proc. IEEE Int'l Conf. Computer Vision, pp. 1458-1465, 2005.
[22] R. Gross and V. Brajovic, “An Image Preprocessing Algorithm for Illumination Invariant Face Recognition,” Proc. Int'l Conf. Audio-and Video-Based Biometric Person Authentication, pp. 10-18, 2003.
[23] X. He and P. Niyogi, “Locality Preserving Projections,” Advances in Neural Information Processing Systems, MIT Press, 2003.
[24] A. Holub, M. Welling, and P. Perona, “Combining Generative Models and Fisher Kernels for Object Recognition,” Proc. IEEE Int'l Conf. Computer Vision, pp. 136-143, 2005.
[25] I. Joliffe, Principal Component Analysis. Springer-Verlag, 1986.
[26] A. Kapoor, K. Grauman, R. Urtasun, and T. Darrell, “Gaussian Processes for Object Categorization,” Int'l J. Computer Vision, vol. 88, no. 2, pp. 169-188, 2010.
[27] S.-J. Kim, A. Magnani, and S. Boyd, “Optimal Kernel Selection in Kernel Fisher Discriminant Analysis,” Proc. Int'l Conf. Machine Learning, pp. 465-472, 2006.
[28] A. Kumar and C. Sminchisescu, “Support Kernel Machines for Object Recognition,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[29] G. Lanckriet, N. Cristianini, P. Bartlett, L. Ghaoui, and M. Jordan, “Learning the Kernel Matrix with Semidefinite Programming,” J. Machine Learning Research, vol. 5, pp. 27-72, 2004.
[30] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2169-2178, 2006.
[31] Y.-Y. Lin, T.-L. Liu, and C.-S. Fuh, “Local Ensemble Kernel Learning for Object Category Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[32] Y.-Y. Lin, T.-L. Liu, and C.-S. Fuh, “Dimensionality Reduction for Data in Multiple Feature Representations,” Advances in Neural Information Processing Systems, pp. 961-968, MIT Press, 2008.
[33] D. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[34] S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K.-R. Müller, “Fisher Discriminant Analysis with Kernels,” Proc. Workshop Neural Networks for Signal Processing, pp. 41-48, 1999.
[35] J. Mutch and D. Lowe, “Multiclass Object Recognition with Sparse, Localized Features,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 11-18, 2006.
[36] T. Ojala, M. Pietikäinen, and T. Mäenpää, “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971-987, July 2002.
[37] A. Oliva and A. Torralba, “Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope,” Int'l J. Computer Vision, vol. 42, no. 3, pp. 145-175, 2001.
[38] E. Pekalska, P. Paclik, and R. Duin, “A Generalized Kernel Approach to Dissimilarity-Based Classification,” J. Machine Learning Research, vol. 2, no. 2, pp. 175-211, 2002.
[39] A. Rakotomamonjy, F. Bach, S. Canu, and Y. Grandvalet, “More Efficiency in Multiple Kernel Learning,” Proc. Int'l Conf. Machine Learning, pp. 775-782, 2007.
[40] S. Roweis and L. Saul, “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” Science, vol. 290, pp. 2323-2326, 2000.
[41] B. Schölkopf, A. Smola, and K.-R. Müller, “Nonlinear Component Analysis as a Kernel Eigenvalue Problem,” Neural Computation, vol. 10, no. 5, pp. 1299-1319, 1998.
[42] T. Serre, L. Wolf, and T. Poggio, “Object Recognition with Features Inspired by Visual Cortex,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 994-1000, 2005.
[43] E. Shechtman and M. Irani, “Matching Local Self-Similarities across Images and Videos,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[44] T. Sim, S. Baker, and M. Bsat, “The CMU Pose, Illumination, and Expression Database,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[45] S. Sonnenburg, G. Rätsch, C. Schäfer, and B. Schölkopf, “Large Scale Multiple Kernel Learning,” J. Machine Learning Research, vol. 7, pp. 1531-1565, 2006.
[46] A. Strehl and J. Ghosh, “Cluster Ensembles—A Knowledge Reuse Framework for Combining Multiple Partitions,” J. Machine Learning Research, vol. 3, pp. 583-617, 2002.
[47] J. Tenenbaum, V. de Silva, and J. Langford, “A Global Geometric Framework for Nonlinear Dimensionality Reduction,” Science, vol. 290, pp. 2319-2323, 2000.
[48] S. Todorovic and N. Ahuja, “Learning Subcategory Relevances for Category Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[49] L. Vandenberghe and S. Boyd, “Semidefinite Programming,” SIAM Rev., vol. 38, pp. 49-95, 1996.
[50] M. Varma and D. Ray, “Learning the Discriminative Power-Invariance Trade-Off,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[51] A. Vedaldi, V. Gulshan, M. Varma, and A. Zisserman, “Multiple Kernels for Object Detection,” Proc. IEEE Int'l Conf. Computer Vision, 2009.
[52] H. Wang, S. Yan, D. Xu, X. Tang, and T. Huang, “Trace Ratio versus Ratio Trace for Dimensionality Reduction,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[53] M. Wu and B. Schölkopf, “A Local Learning Approach for Clustering,” Advances in Neural Information Processing Systems, pp. 1529-1536, MIT Press, 2006.
[54] S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin, “Graph Embedding and Extensions: A General Framework for Dimensionality Reduction,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 40-51, Jan. 2007.
[55] J. Yang, Y. Li, Y. Tian, L. Duan, and W. Gao, “Group-Sensitive Multiple Kernel Learning for Object Categorization,” Proc. IEEE Int'l Conf. Computer Vision, 2009.
[56] J. Ye, R. Janardan, and Q. Li, “GPCA: An Efficient Dimension Reduction Scheme for Image Compression and Retrieval,” Proc. ACM SIGKDD, pp. 354-363, 2004.
[57] J. Ye, R. Janardan, and Q. Li, “Two-Dimensional Linear Discriminant Analysis,” Advances in Neural Information Processing Systems, MIT Press, 2004.
[58] H. Zhang, A. Berg, M. Maire, and J. Malik, “SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2126-2136, 2006.
[59] J. Zhu, S. Rosset, H. Zou, and T. Hastie, “Multi-Class Adaboost,” technical report, Dept. of Statistics, Univ. of Michigan, 2005.
59 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool