The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2013 vol.25)
pp: 704-719
Xiangnan Kong , Nanjing University, Nanjing
Michael K. Ng , Hong Kong Baptist University, Hong Kong
Zhi-Hua Zhou , Nanjing University, Nanjing
ABSTRACT
The problem of multilabel classification has attracted great interest in the last decade, where each instance can be assigned with a set of multiple class labels simultaneously. It has a wide variety of real-world applications, e.g., automatic image annotations and gene function analysis. Current research on multilabel classification focuses on supervised settings which assume existence of large amounts of labeled training data. However, in many applications, the labeling of multilabeled data is extremely expensive and time consuming, while there are often abundant unlabeled data available. In this paper, we study the problem of transductive multilabel learning and propose a novel solution, called Trasductive Multilabel Classification (TraM), to effectively assign a set of multiple labels to each instance. Different from supervised multilabel learning methods, we estimate the label sets of the unlabeled instances effectively by utilizing the information from both labeled and unlabeled data. We first formulate the transductive multilabel learning as an optimization problem of estimating label concept compositions. Then, we derive a closed-form solution to this optimization problem and propose an effective algorithm to assign label sets to the unlabeled instances. Empirical studies on several real-world multilabel learning tasks demonstrate that our TraM method can effectively boost the performance of multilabel classification by using both labeled and unlabeled data.
INDEX TERMS
Optimization, Learning systems, Training data, Data mining, Machine learning, Closed-form solution, Semisupervised learning, unlabeled data, Data mining, machine learning, multilabel learning, transductive learning, semi-supervised learning
CITATION
Xiangnan Kong, Michael K. Ng, Zhi-Hua Zhou, "Transductive Multilabel Learning via Label Set Propagation", IEEE Transactions on Knowledge & Data Engineering, vol.25, no. 3, pp. 704-719, March 2013, doi:10.1109/TKDE.2011.141
REFERENCES
[1] M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold Regularization: A Geometric Framework for Learning from Examples," J. Machine Learning Research, vol. 7, pp. 2399-2434, 2006.
[2] N. Biggs, Algebraic Graph Theory. Cambridge Univ. Press, 1974.
[3] A. Blum and S. Chawla, "Learning from Labeled and Unlabeled Data Using Graph Mincuts," Proc. 18th Int'l Conf. Machine Learning, pp. 19-26, 2001.
[4] M.R. Boutell, J. Luo, X. Shen, and C.M. Brown, "Learning Multi-Label Scene Classification," Pattern Recognition, vol. 37, no. 9, pp. 1757-1771, 2004.
[5] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
[6] G. Carneiro, A. Chan, P. Moreno, and N. Vasconcelos, "Supervised Learning of Semantic Classes for Image Annotation and Retrieval," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 3, pp. 394-410, Mar. 2007.
[7] Semi-Supervised Learning. O. Chapelle, B. Schölkopf, and A. Zien, eds., MIT Press, 2006.
[8] F.D. Comité, R. Gilleron, and M. Tommasi, "Learning Multi-Label Altenating Decision Tree From Texts and Data," Proc. Third Int'l Conf. Machine Learning and Data Mining in Pattern Recognition, pp. 35-49, 2003.
[9] P. Duygulu, K. Barnard, N. De Freitas, and D. Forsyth, "Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary," Proc. Seventh European Conf. Computer Vision, pp. 97-112, 2002.
[10] A. Elisseeff and J. Weston, "A Kernel Method for Multi-Labelled Classification," Advances in Neural Information Processing Systems 14, T.G. Dietterich, S. Becker and Z. Ghahramani, eds., pp. 681-687, MIT Press, 2002.
[11] Y. Freund and R.E. Schapire, "A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting," J. Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1997.
[12] I. Vlahavas and G. Tsoumakas, "Random K-Labelsets: An Ensemble Method for Multi-Label Classification," Proc. 18th European Conf. Machine Learning, pp. 406-417, 2007.
[13] S. Gao, W. Wu, C.H. Lee, and T.-S. Chua, "A MFoM Learning Approach to Robust Multiclass Multi-Label Text Categorization," Proc. 21th Int'l Conf. Machine Learning, pp. 329-336, 2004.
[14] N. Ghamrawi and A. McCallum, "Collective Multi-Label Classification," Proc. 14th Int'l Conf. Information and Knowledge Management, pp. 195-200, 2005.
[15] S. Godbole and S. Sarawagi, "Discriminative Methods for Multi-Labeled Classification," Proc. Eight Pacific-Asia Conf. Knowledge Discovery and Data Mining, pp. 22-30, 2004.
[16] W. Hackbusch, "Iterative Solution of Large Sparse Systems of Equations," Math. of Computation, vol. 64, no. 212, pp. 1759-1761, 1995.
[17] T. Joachims, "Transductive Inference for Text Classification Using Support Vector Machines," Proc. 16th Int'l Conf. Machine Learning, pp. 200-209, 1999.
[18] T. Joachims, "Transductive Learning via Spectral Graph Partitioning," Proc. 20th Int'l Conf. Machine Learning, pp. 290-297, 2003.
[19] F. Kang, R. Jin, and R. Sukthankar, "Correlated Label Propagation with Application to Multi-Label Learning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1719-1726, 2006.
[20] H. Kazawa, T. Izumitani, H. Taira, and E. Maeda, "Maximal Margin Labeling for Multi-Topic Text Categorization," Advances in Neural Information Processing Systems 17, L.K. Saul, Y. Weiss, and L. Bottou, eds., pp. 649-656, MIT Press, 2005.
[21] D.D. Lewis, Y. Yang, T. Rose, and F. Li, "RCV1: A New Benchmark Collection for Text Categorization Research," J. Machine Learning Research, vol. 5, pp. 361-397, 2004.
[22] Y. Liu, R. Jin, and L. Yang, "Semi-Supervised Multi-Label Learning by Constrained Non-Negative Matrix Factorization," Proc. 21st Nat'l Conf. Artificial Intelligence, pp. 421-426, 2006.
[23] P. Matstoms, "Sparse QR Factorization in MATLAB," ACM Trans. Math. Software, vol. 20, no. 1, pp. 136-159, 1994.
[24] A. McCallum, "Multi-Label Text Classification with a Mixture Model Trained by EM," Proc. Working Notes Am. Assoc. Artificial Intelligence Workshop Text Learning (AAAI '99), 1999.
[25] M. Ng, G. Qiu, and A. Yip, "A Study of Interactive Multiple Class Image Segmentation Problems," Technical Report 07-51, UCLA CAM, 2007.
[26] P. Pavlidis, J. Weston, J. Cai, and W.N. Grundy, "Combining Microarray Expression Data and Phylogenetic Profiles to Learn Functional Categories Using Support Vector Machines," Proc. Fifth Int'l Conf. Computational Biology, pp. 242-248, 2001.
[27] G.J. Qi, X.S. Hua, Y. Rui, J. Tang, T. Mei, and H.J. Zhang, "Correlative Multi-Label Video Annotation," Proc. 15th Int'l Conf. Multimedia, pp. 17-26, 2007.
[28] R.E. Schapire and Y. Singer, "BoosTexter: A Boosting-Based System for Text Categorization," Machine Learning, vol. 39, nos. 2/3, pp. 135-168, 2000.
[29] L. Sun, S.-W. Ji, and J.-P. Ye, "Hypergraph Spectral Learning for Multi-Label Classification," Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 668-676, 2008.
[30] G. Tsoumakas and I. Katakis, "Multi-Label Classification: An Overview," Int'l J. Data Warehousing and Mining, vol. 3, no. 3, pp. 1-13, 2007.
[31] N. Ueda and K. Saito, "Parametric Mixture Models for Multi-Labeled Text," Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, and K. Obermayer, eds., pp. 721-728, MIT Press, 2003.
[32] V.N. Vapnik, Statistical Learning Theory. Wiley, 1998.
[33] R. Weber, H.-J. Schek, and S. Blott, "A Quantitative Analysis and Performance Study for Similaritysearch Methods in High-Dimensional Space," Proc. 24th Int'l Conf. Very Large Data Bases, pp. 194-205, 1998.
[34] M.-L. Zhang and Z.-H. Zhou, "Multi-Label Neural Networks with Applications to Functional Genomics and Text Categorization," IEEE Trans. Knowledge and Data Eng., vol. 18, no. 10, pp. 1479-1493, Oct. 2006.
[35] M.-L. Zhang and Z.-H. Zhou, "ML-kNN: A Lazy Learning Approach to Multi-Label Learning," Pattern Recognition, vol. 40, no. 7, pp. 2038-2048, 2007.
[36] Y. Zhang and Z.-H. Zhou, "Multilabel Dimensionality Reduction via Dependence Maximization," ACM Trans. Knowledge Discovery from Data, vol. 4, no. 3,article 14, 2010.
[37] D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Scholkopf, "Learning with Local and Global Consistency," Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, and K. Obermayer, eds., pp. 321-328, MIT Press, 2003.
[38] Z.-H. Zhou and M. Li, "Semi-Supervised Learning by Disagreement," Knowledge and Information Systems, vol. 24, no. 3, pp. 415-439, 2010.
[39] Z.-H. Zhou and M.-L. Zhang, "Multi-Instance Multi-Label Learning with Application to Scene Classification," Advances in Neural Information Processing Systems 18, Y. Weiss, B. Schölkopf, and J. Platt, eds., pp. 1609-1616, MIT Press, 2006.
[40] Z.-H. Zhou and M.-L. Zhang, and S.-J. Huang, and Y.-F. Li, "Multi-Instance Multi-Label Learning," Artificial Intelligence, vol. 176, no. 1, pp. 2291-2320, 2012.
[41] S. Zhu, X. Ji, W. Xu, and Y. Gong, "Multi-Labelled Classification Using Maximum Entropy Method," Proc. 28th Int'l Conf. Research and Development in Information Retrieval, pp. 274-281, 2005.
[42] X. Zhu, "Semi-Supervised Learning Literature Survey," Technical Report 1530, Department of Computer Sciences, Univ. of Wisconsin at Madison, 2006.
22 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool