This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Two-Dimensional Multilabel Active Learning with an Efficient Online Adaptation Model for Image Classification
October 2009 (vol. 31 no. 10)
pp. 1880-1897
Guo-Jun Qi, University of Illinois at Urbana-Champaign, Urbana
Xian-Sheng Hua, Microsoft Research Asia, Beijing
Yong Rui, Microsoft China R&D Group, Beijing
Jinhui Tang, National University of Singapore, Singapore
Hong-Jiang Zhang, Microsoft Advanced Technology Center, Beijing
Conventional active learning dynamically constructs the training set only along the sample dimension. While this is the right strategy in binary classification, it is suboptimal for multilabel image classification. We argue that for each selected sample, only some effective labels need to be annotated while others can be inferred by exploring the label correlations. The reason is that the contributions of different labels to minimizing the classification error are different due to the inherent label correlations. To this end, we propose to select sample-label pairs, rather than only samples, to minimize a multilabel Bayesian classification error bound. We call it two-dimensional active learning because it considers both the sample dimension and the label dimension. Furthermore, as the number of training samples increases rapidly over time due to active learning, it becomes intractable for the offline learner to retrain a new model on the whole training set. So we develop an efficient online learner to adapt the existing model with the new one by minimizing their model distance under a set of multilabel constraints. The effectiveness and efficiency of the proposed method are evaluated on two benchmark data sets and a realistic image collection from a real-world image sharing Web site—Corbis.

[1] L. Fei-Fei, R. Fergus, and P. Perona, “One-Shot Learning of Object Categories,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 594-611, Apr. 2006.
[2] A. Kapoor, K. Grauman, R. Urtasun, and T. Darrel, “Active Learning with Gaussian Processes for Object Categorization,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[3] G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, T. Mei, and H.-J. Zhang, “Correlative Multi-Label Video Annotation,” Proc. ACM Conf. Multimedia, 2007.
[4] S. Zhu, X. Ji, W. Xu, and Y. Gong, “Multi-Labelled Classification Using Maximum Entropy Method,” Proc. Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, 2005.
[5] G.-J. Qi, Y. Song, X.-S. Hua, L.-R. Dai, and H.-J. Zhang, “Video Annotation by Active Learning and Cluster Tuning,” Proc. Int'l Workshop Semantic Learning Applications in Multimedia (in association with CVPR), 2006.
[6] S.C.H. Hoi and M.R. Lyu, “A Semi-Supervised Active Learning Framework for Image Retrieval,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2005.
[7] A. Dong and B. Bhanu, “Active Concept Learning for Image Retrieval in Dynamic Databases,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[8] R. Yan, J. Yang, and A. Hauptmann, “Automatically Labeling Data Using Multi-Class Active Learning,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[9] S. Tong and E.Y. Chang, “Support Vector Machine Active Learning for Image Retrieval,” Proc. ACM Conf. Multimedia, 2001.
[10] E.Y. Chang, S. Tong, K. Goh, and C. Chang, “Support Vector Machine Concept-Dependent Active Learning for Image Retrieval,” IEEE Trans. Multimedia, 2005.
[11] A. Krause, A. Singh, and C. Guestrin, “Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies,” J. Machine Learning Research, vol. 9, pp.235-284, 2008.
[12] X. Li, L. Wang, and E. Sung, “Multi-Label SVM Active Learning for Image Classification,” Proc. Int'l Conf. Image Processing, 2004.
[13] M.R. Boutell, J. Luo, X. Shen, and C.M. Brown, “Learning Multi-Label Scene Classification,” Pattern Recognition, vol. 37, no. 9, pp.1757-1771, 2004.
[14] K. Brinker, “On Active Learning in Multi-Label Classification,” From Data and Information Analysis to Knowledge Eng., Springer, 2006.
[15] M.E. Hellman and J. Raviv, “Probability of Error, Equivocation, and the Chernoff Bound,” IEEE Trans. Information Theory, vol. 16, no. 4, pp. 368-372, July 1970.
[16] A. Kapoor and E. Horvitz, “On Discarding, Caching, and Recalling Samples in Active Learning,” Proc. Conf. Uncertainty and Artificial Intelligence, 2007.
[17] F. Jing, M. Li, and H.-J. Zhang, “Entropy-Based Active Learning with Support Vector Machine for Content-Based Image Retrieval,” Proc. IEEE Int'l Conf. Multimedia and Expo, 2004.
[18] N. Roy and A. McCallum, “Toward Optimal Active Learning through Sampling Estimation of Error Reduction,” Proc. Int'l Conf. Machine Learning, 2001.
[19] T. Cover and J. Thomas, Elements of Information Theory, second ed. John Wiley and Sons, 2006.
[20] G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, and H.-J. Zhang, “Two Dimensional Active Learning for Image Classification,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2008.
[21] C. Bishop, “Approximate Inference,” Pattern Recognition and Machine Learning, pp. 461-473, Springer, 2006.
[22] S.F. Chen and R. Rosenfeld, “A Gaussian Prior for Smoothing Maximum Entropy Models,” Technical Report CMU-CS-99-108, School of Computer Science, Carnegie Mellon Univ., 1999.
[23] J. Wu, X.-S. Hua, and B. Zhang, “Tracking Concept Drifting with Gaussian Mixture Model,” Proc. Int'l Conf. Visual Comm. and Image Processing, 2005.
[24] D.C. Liu and J. Nocedal, “On the Limited Memory BFGS Method for Large Scale Optimization,” Math. Programming B, vol. 45, nos.1-3, pp. 503-528, 1989.
[25] N. Syed, H. Liu, and K. Sung, “Incremental Learning with Support Vector Machines,” Proc. Workshop Support Vector Machines, at the Int'l Joint Conf. Artificial Intelligence, 1999.
[26] G. Cauwenberghs and T. Poggio, “Incremental and Decremental Support Vector Machine,” Proc. Conf. Neural Information Processing Systems, 2000.
[27] J. Yang, R. Yan, and A. Hauptmann, “Cross-Domain Video Concept Detection Using Adaptive svms,” Proc. ACM Conf. Multimedia, 2007.
[28] A.P. Dempster, N.M. Laird, and D.B. Rubin, “Maximum-Likelihood from Incomplete Data via EM Algorithm,” J. Royal Statistical Soc., vol. 39, no. 1, pp. 1-38, 1977.
[29] R. Neal and G. Hinton, A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants, M. Jordan, ed. Kluwer Academic Press, 1998.
[30] R.M. Neal, “Probabilistic Inference Using Markov Chain Monte Carlo Methods,” Technical Report CRG-TR-93-1, Univ. of Toronto, 1993.
[31] B.J. Frey and D.J.C. MacKay, “A Revolution: Belief Propagation in Graphs with Cycles,” Advances in Neural Information Processing Systems, vol. 10, MIT Press, 1998.
[32] T. Minka, “Expectation Propagation for Approximate Bayesian Inference,” Proc. 17th Conf. Uncertainty in Artificial Intelligence, 2001.
[33] K.P. Murphy, Y. Weiss, and M.I. Jordan, “Loopy Belief Propagation for Approximate Inference: An Empirical Study,” Proc. Conf. Uncertainty in Artificial Intelligence, 1999.
[34] T. Volkmer, J.R. Smith, and A. Natsev, “A Web-Based System for Collaborative Annotation of Large Image and Video Collections,” Proc. ACM Int'l Conf. Multimedia, 2005.
[35] A. Elisseeff and J. Weston, “A Kernel Method for Multi-Labelled Classification,” Proc. Conf. Neural Information Processing Systems, 2002.
[36] L. Fei-Fei and P. Perona, “A Bayesian Hierarchical Model for Learning Natural Scene Categories,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2005.
[37] G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, Z.-J. Zha, and H.-J. Zhang, “A Joint Appearance-Spatial Distance for Kernel-Based Image Categorization,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2008.
[38] B. Merialdo, J. Jiten, E. Galmar, and B. Huet, “A New Approach to Probabilistic Image Modeling with Multidimensional Hidden Markov Models,” Proc. Fourth Int'l Workshop Adaptive Multimedia Retrieval, 2006.
[39] C.G.M. Snoek, M. Worring, J.C. Gemert, J.-M. Geusebroek, and A.W.M. Smeulders, “The Challenge Problem for Automated Detection of 101 Semantic Concepts in Multimedia,” Proc. ACM Int'l Conf. Multimedia, pp. 421-430, Oct. 2006.

Index Terms:
Active learning, online adaption, multilabel classification, image annotation.
Citation:
Guo-Jun Qi, Xian-Sheng Hua, Yong Rui, Jinhui Tang, Hong-Jiang Zhang, "Two-Dimensional Multilabel Active Learning with an Efficient Online Adaptation Model for Image Classification," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 10, pp. 1880-1897, Oct. 2009, doi:10.1109/TPAMI.2008.218
Usage of this product signifies your acceptance of the Terms of Use.