The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2012 vol.34)
pp: 465-479
Lixin Duan , Nanyang Technological University, Singapore
Ivor W. Tsang , Nanyang Technological University, Singapore
Dong Xu , Nanyang Technological University, Singapore
ABSTRACT
Cross-domain learning methods have shown promising results by leveraging labeled patterns from the auxiliary domain to learn a robust classifier for the target domain which has only a limited number of labeled samples. To cope with the considerable change between feature distributions of different domains, we propose a new cross-domain kernel learning framework into which many existing kernel methods can be readily incorporated. Our framework, referred to as Domain Transfer Multiple Kernel Learning (DTMKL), simultaneously learns a kernel function and a robust classifier by minimizing both the structural risk functional and the distribution mismatch between the labeled and unlabeled samples from the auxiliary and target domains. Under the DTMKL framework, we also propose two novel methods by using SVM and prelearned classifiers, respectively. Comprehensive experiments on three domain adaptation data sets (i.e., TRECVID, 20 Newsgroups, and email spam data sets) demonstrate that DTMKL-based methods outperform existing cross-domain learning and multiple kernel learning methods.
INDEX TERMS
Cross-domain learning, domain adaptation, transfer learning, support vector machine, multiple kernel learning.
CITATION
Lixin Duan, Ivor W. Tsang, Dong Xu, "Domain Transfer Multiple Kernel Learning", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 3, pp. 465-479, March 2012, doi:10.1109/TPAMI.2011.114
REFERENCES
[1] F.R. Bach, G.R.G. lanckriet, and M. Jordan, “Multiple Kernel Learning, Conic Duality, and the SMO Algorithm,” Proc. Int'l Conf. Machine Learning, 2004.
[2] J. Blitzer, M. Dredze, and F. Pereira, “Biographies, Bollywood, Boom-Boxes and Blenders: Domain Adaptation for Sentiment Classification,” Proc. Ann. Meeting Assoc. for Computational Linguistics, pp. 440-447, 2007.
[3] A. Blum and T. Mitchell, “Combining Labeled and Unlabeled Data with Co-Training,” Proc. Ann. Conf. Learning Theory, pp. 92-100, 1998.
[4] K.M. Borgwardt, A. Gretton, M.J. Rasch, H.-P. Kriegel, B. Schölkopf, and A.J. Smola, “Integrating Structured Biological Data by Kernel Maximum Mean Discrepancy,” Bioinformatics, vol. 22, no. 4, pp. 49-57, 2006.
[5] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
[6] L. Bruzzone and M. Marconcini, “Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 770-787, May 2010.
[7] C.-C. Chang and C.-J. Lin, “LIBSVM: A Library for Support Vector Machines,” http://www.csie.ntu.edu.tw/~cjlinlibsvm, 2001.
[8] S.-F. Chang, J. He, Y.-G. Jiang, E.E. Khoury, C.-W. Ngo, A. Yanagawa, and E. Zavesky, “Columbia University/VIREO-CityU/IRIT TRECVID2008 High-Level Feature Extraction and Interactive Video Search,” Proc. TREC Video Retrieval Evaluation Workshop, 2008.
[9] B. Chen, W. Lam, I.W. Tsang, and T.L. Wong, “Extracting Discriminative Concepts for Domain Adaptation in Text Mining,” Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 179-188, 2009.
[10] A. Corduneanu and T. Jaakkola, “Continuation Methods for Mixing Heterogeneous Sources,” Proc. Ann. Conf. Uncertainty in Artificial Intelligence, pp. 111-118, 2002.
[11] H. DauméIII, “Frustratingly Easy Domain Adaptation,” Proc. Ann. Meeting Assoc. for Computational Linguistics, pp. 256-263, 2007.
[12] L. Duan, I.W. Tsang, D. Xu, and T.-S. Chua, “Domain Adaptation from Multiple Sources via Auxiliary Classifiers,” Proc. Int'l Conf. Machine Learning, pp. 289-296, 2009.
[13] L. Duan, I.W. Tsang, D. Xu, and S.J. Maybank, “Domain Transfer SVM for Video Concept Detection,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1375-1381, 2009.
[14] L. Duan, D. Xu, I.W. Tsang, and J. Luo, “Visual Event Recognition in Videos by Learning from Web Data,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1959-1966, 2010.
[15] J. Huang, A.J. Smola, A. Gretton, K.M. Borgwardt, and B. Schölkopf, “Correcting Sample Selection Bias by Unlabeled Data,” Proc. Advances in Neural Information Processing Systems 19, pp. 601-608, 2007.
[16] W. Jiang, E. Zavesky, S.-F. Chang, and A. Loui, “Cross-Domain Learning Methods for High-Level Visual Concept Classification,” Proc. IEEE Int'l Conf. Image Processing, pp. 161-164, 2008.
[17] Y.-G. Jiang, J. Wang, S.-F. Chang, and C.-W. Ngo, “Domain Adaptive Semantic Diffusion for Large Scale Context-Based Video Annotation,” Proc. IEEE Int'l Conf. Computer Vision, pp. 1420-1427, 2009.
[18] G. Lanckriet, N. Cristianini, P. Bartlett, L.E. Ghaoui, and M.I. Jordan, “Learning the Kernel Matrix with Semidefinite Programming,” J. Machine Learning Research, vol. 5, pp. 27-72, 2004.
[19] Y. Liu, D. Xu, I.W. Tsang, and J. Luo, “Textual Query of Personal Photos Facilitated by Large-Scale Web Data,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 1022-1036, May 2011.
[20] M. Naphade, J.R. Smith, J. Tesic, S.-F. Chang, W. Hsu, L. Kennedy, A. Hauptmann, and J. Curtis, “Large-Scale Concept Ontology for Multimedia,” IEEE Multimedia, vol. 13, no. 3, pp. 86-91, July-Sept. 2006.
[21] S.J. Pan, J.T. Kwok, and Q. Yang, “Transfer Learning via Dimensionality Reduction,” Proc. Assoc. for the Advancement of Artificial Intelligence, pp. 677-682, 2008.
[22] S.J. Pan and Q. Yang, “A Survey on Transfer Learning,” IEEE Trans. Knowledge and Data Eng., vol. 22, no. 10, pp. 1345-1359, Oct. 2010.
[23] J.C. Platt, “Fast Training of Support Vector Machines Using Sequential Minimal Optimization,” Advances in Kernel Methods: Support Vector Learning, pp. 185-208, MIT Press, 1999.
[24] A. Rakotomamonjy, F.R. Bach, S. Canu, and Y. Grandvalet, “SimpleMKL,” J. Machine Learning Research, vol. 9, pp. 2491-2521, 2008.
[25] B. Schölkopf and A. Smola, Learning with Kernels. MIT Press, 2002.
[26] G. Schweikert, C. Widmer, B. Schölkopf, and G. Rätsch, “An Empirical Analysis of Domain Adaptation Algorithms for Genomic Sequence Analysis,” Proc. Advances in Neural Information Processing Systems, pp. 1433-1440, 2008.
[27] A.F. Smeaton, P. Over, and W. Kraaij, “Evaluation Campaigns and TRECVid,” Proc. ACM Int'l Workshop Multimedia Information Retrieval, 2006.
[28] S. Sonnenburg, G. Rätsch, C. Schäfer, and B. Schölkopf, “Large Scale Multiple Kernel Learning,” J. Machine Learning Research, vol. 7, pp. 1531-1565, 2006.
[29] A.J. Storkey and M. Sugiyama, “Mixture Regression for Covariate Shift,” Proc. Advances in Neural Information Processing Systems 19, pp. 1337-1344, 2007.
[30] J. Sun, X. Wu, S. Yan, L.-F. Cheong, T.-S. Chua, and J. Li, “Hierarchical Spatio-Temporal Context Modeling for Action Recognition,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 2004-2011, 2009.
[31] P. Wu and T.G. Dietterich, “Improving SVM Accuracy by Training on Auxiliary Data Sources,” Proc. Int'l Conf. Machine Learning, pp. 871-878, 2004.
[32] X. Wu, D. Xu, L. Duan, and J. Luo, “Action Recognition Using Context and Appearance Distribution Features,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2011.
[33] A. Vedaldi, V. Gulshan, M. Varma, and A. Zisserman, “Multiple Kernels for Object Detection,” Proc. IEEE Int'l Conf. Computer Vision, pp. 606-613, 2009.
[34] D. Xu and S.-F. Chang, “Video Event Recognition Using Kernel Methods with Multilevel Temporal Alignment,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 11, pp. 1985-1997, Nov. 2008.
[35] A. Yanagawa, W. Hsu, and S.-F. Chang, “Columbia University's Baseline Detectors for 374 LSCOM Semantic Visual Concepts,” Columbia Univ. ADVENT technical report, 2007.
[36] J. Yang, R. Yan, and A.G. Hauptmann, “Cross-Domain Video Concept Detection Using Adaptive SVMs,” Proc. ACM Int'l Conf. Multimedia, pp. 188-197, 2007.
[37] X. Zhu, “Semi-Supervised Learning Literature Survey,” technical report, Univ. of Wisconsin-Madison, 2008.
[38] A. Zien and C.S. Ong, “Multiclass Multiple Kernel Learning,” Proc. Int'l Conf. Machine Learning, pp. 1191-1198, 2007.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool