The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2013 vol.25)
pp: 2381-2389
Mehmet Gonen , Aalto University School of Science, Espoo
ABSTRACT
For supervised learning problems, dimensionality reduction is generally applied as a preprocessing step. However, coupled training of dimensionality reduction and supervised learning steps may improve the prediction performance. In this paper, we propose a novel dimensionality reduction algorithm coupled with a supervised kernel-based learner, called supervised multiple kernel embedding, that integrates multiple kernel learning to dimensionality reduction and performs prediction on the projected subspace with a joint optimization framework. Combining multiple kernels allows us to combine different feature representations and/or similarity measures toward a unified subspace. We perform experiments on one digit recognition and two bioinformatics data sets. Our proposed method significantly outperforms multiple kernel Fisher discriminant analysis followed by a standard kernel-based learner, especially on low dimensions.
INDEX TERMS
Kernel, Training, Optimization, Vectors, Supervised learning, Support vector machines, Standards, subspace learning, Kernel, Training, Optimization, Vectors, Supervised learning, Support vector machines, Standards, supervised learning, Dimensionality reduction, kernel machines, multiple kernel learning
CITATION
Mehmet Gonen, "Supervised Multiple Kernel Embedding for Learning Predictive Subspaces", IEEE Transactions on Knowledge & Data Engineering, vol.25, no. 10, pp. 2381-2389, Oct. 2013, doi:10.1109/TKDE.2012.213
REFERENCES
[1] K. Pearson, "On Lines and Planes of Closest Fit to Systems of Points in Space," Philosophical Magazine, vol. 2, no. 6, pp. 559-572, 1901.
[2] B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem," Neural Computation, vol. 10, no. 5, pp. 1299-1319, 1998.
[3] R.A. Fisher, "The Use of Multiple Measurements in Taxonomic Problems," Annals of Eugenics, vol. 7, no. 2, pp. 179-188, 1936.
[4] S. Mika, G. Rätsch, J. Weston, B. Schölkopf, A. Smola, and K.-R. Müller, "Constructing Descriptive and Discriminative Nonlinear Features: Rayleigh Coefficients in Kernel Feature Space," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp. 623-628, May 2003.
[5] G.R.G. Lanckriet, N. Cristianini, P. Bartlett, L.E. Ghaoui, and M.I. Jordan, "Learning the Kernel Matrix with Semidefinite Programming," The J. Machine Learning Research, vol. 5, pp. 27-72, 2004.
[6] F.R. Bach, G.R.G. Lanckriet, and M.I. Jordan, "Multiple Kernel Learning, Conic Duality, and the SMO Algorithm," Proc. 21st Int'l Conf. Machine Learning, 2004.
[7] M. Gönen and E. Alpaydin, "Multiple Kernel Learning Algorithms," The J. Machine Learning Research, vol. 12, pp. 2211-2268, 2011.
[8] B. Schölkopf and A.J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2002.
[9] J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge Univ. Press, 2004.
[10] A. Edelman, T.A. Arias, and S.T. Smith, "The Geometry of Algorithms with Orthogonality Constraints," SIAM J. Matrix Analysis and Applications, vol. 20, no. 2, pp. 303-353, 1998.
[11] A. Rakotomamonjy, F.R. Bach, S. Canu, and Y. Grandvalet, "SimpleMKL," The J. Machine Learning Research, vol. 9, pp. 2491-2521, 2008.
[12] A. Globerson and S. Roweis, "Metric Learning by Collapsing Classes," Proc. Conf. Advances in Neural Information Processing Systems, vol. 18, 2006.
[13] K.Q. Weinberger and L.K. Saul, "Distance Metric Learning for Large Margin Nearest Neighbor Classification," The J. Machine Learning Research, vol. 10, pp. 207-244, 2009.
[14] F. Pereira and G. Gordon, "The Support Vector Decomposition Machine," Proc. 23rd Int'l Conf. Machine Learning, 2006.
[15] S. Ji and J. Ye, "Linear Dimensionality Reduction for Multi-Label Classification," Proc. 21st Int'l Joint Conf. Artifical Intelligence, 2009.
[16] Z. Liang and Y. Li, "Multiple Kernels for Generalised Discriminant Analysis," IET Computer Vision, vol. 4, pp. 117-128, 2010.
[17] Y.-Y. Lin, T.-Y. Liu, and C.-S. Fuh, "Multiple Kernel Learning for Dimensionality Reduction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 6, pp. 1147-1160, June 2011.
[18] S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin, "Graph Embedding and Extensions: A General Framework for Dimensionality Reduction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 40-51, Jan. 2007.
[19] X. He and P. Niyogi, "Locality Preserving Projections," Proc. Conf. Advances in Neural Information Processing Systems, vol. 16, 2004.
[20] D. Cai, X. He, and J. Han, "Semi-Supervised Discriminant Analysis," Proc. 11th Int'l Conf. Computer Vision, 2007.
[21] C. Hou, C. Zhang, Y. Wu, and Y. Jiao, "Stable Local Dimensionality Reduction Approaches," Pattern Recognition, vol. 42, no. 9, pp. 2054-2066, 2009.
[22] C.-C. Chang and C.-J. Lin, "LIBSVM: A Library for Support Vector Machine," http://www.csie.ntu.edu.tw/~cjlinlibsvm/, 2001.
[23] A. Zien and C.S. Ong, "Multiclass Multiple Kernel Learning," Proc. 24th Int'l Conf. Machine Learning, 2007.
26 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool