CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2014 vol.36 Issue No.03 - March

Subscribe

Issue No.03 - March (2014 vol.36)

pp: 536-549

Hao Xia , Sch. of Comput. Eng., Nanyang Technol. Univ., Singapore, Singapore

Steven C. H. Hoi , Sch. of Comput. Eng., Nanyang Technol. Univ., Singapore, Singapore

Rong Jin , Dept. of Comput. Sci. & Eng., Michigan State Univ., East Lansing, MI, USA

Peilin Zhao , Sch. of Comput. Eng., Nanyang Technol. Univ., Singapore, Singapore

ABSTRACT

Recent years have witnessed a number of studies on distance metric learning to improve visual similarity search in content-based image retrieval (CBIR). Despite their successes, most existing methods on distance metric learning are limited in two aspects. First, they usually assume the target proximity function follows the family of Mahalanobis distances, which limits their capacity of measuring similarity of complex patterns in real applications. Second, they often cannot effectively handle the similarity measure of multimodal data that may originate from multiple resources. To overcome these limitations, this paper investigates an online kernel similarity learning framework for learning kernel-based proximity functions which goes beyond the conventional linear distance metric learning approaches. Based on the framework, we propose a novel online multiple kernel similarity (OMKS) learning method which learns a flexible nonlinear proximity function with multiple kernels to improve visual similarity search in CBIR. We evaluate the proposed technique for CBIR on a variety of image data sets in which encouraging results show that OMKS outperforms the state-of-the-art techniques significantly.

INDEX TERMS

Kernel, Measurement, Optimization, Image retrieval, Visualization, Support vector machines, Algorithm design and analysis,content-based image retrieval, Similarity search, kernel methods, multiple kernel learning, online learning

CITATION

Hao Xia, Steven C. H. Hoi, Rong Jin, Peilin Zhao, "Online Multiple Kernel Similarity Learning for Visual Search",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.36, no. 3, pp. 536-549, March 2014, doi:10.1109/TPAMI.2013.149REFERENCES

- [1] M.S. Lew, N. Sebe, C. Djeraba, and R. Jain, "Content-Based Multimedia Information Retrieval: State of the Art and Challenges,"
ACM Trans. Multimedia Computer Comm. Applications, vol. 2, no. 1, pp. 1-19, 2006.- [2] Y. Jing and S. Baluja, "VisualRank: Applying PageRank to Large-Scale Image Search,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 11, pp. 1877-1890, Nov. 2008.- [3] D. Grangier and S. Bengio, "A Discriminative Kernel-Based Approach to Rank Images from Text Queries,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 8, pp. 1371-1384, Aug. 2008.- [4] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-Based Image Retrieval at the end of the Early Years,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.- [5] S.C.H. Hoi, W. Liu, M.R. Lyu, and W.-Y. Ma, "Learning Distance Metrics with Contextual Constraints for Image Retrieval,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 17-22, June 2006.- [6] R. Rahmani, S.A. Goldman, H. Zhang, S.R. Cholleti, and J.E. Fritts, "Localized Content-Based Image Retrieval,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 11, pp. 1902-1912, Nov. 2008.- [7] S. Aksoy and R.M. Haralick, "Probabilistic vs. Geometric Similarity Measures for Image Retrieval,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2357-2362. 2000.- [8] A. Qamra, Y. Meng, and E.Y. Chang, "Enhanced Perceptual Distance Functions and Indexing for Image Replica Recognition,"
IEEE Trans. Pattern Analysis and Machine Intelligence , vol. 27, no. 3, pp. 379-391, Mar. 2005.- [9] L. Si, R. Jin, S.C.H. Hoi, and M.R. Lyu, "Collaborative Image Retrieval via Regularized Metric Learning,"
ACM Multimedia Systems J., vol. 12, no. 1, pp. 34-44, 2006.- [10] J.-E. Lee, R. Jin, and A.K. Jain, "Rank-Based Distance Metric Learning: An Application to Image Retrieval,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.- [11] S.C.H. Hoi, W. Liu, and S.-F. Chang, "Semi-Supervised Distance Metric Learning for Collaborative Image Retrieval and Clustering,"
ACM Trans. Multimedia Computer Comm. Applications, vol. 6, no. 3, pp. 18:1-18:26, Aug. 2010.- [12] R. Chatpatanasiri, T. Korsrilabutr, P. Tangchanachaianan, and B. Kijsirikul, "A New Kernelization Framework for Mahalanobis Distance Learning Algorithms,"
Neurocomputing, vol. 73, nos. 10-12, pp. 1570-1579, 2010.- [13] P. Jain, B. Kulis, J.V. Davis, and I.S. Dhillon, "Metric and Kernel Learning Using a Linear Transformation,"
J. Machine Learning Research, vol. 13, pp. 519-547, 2012.- [14] S.C.H. Hoi, J. Wang, and P. Zhao,
LIBOL: A Library for Online Learning Algorithms, Nanyang Technological Univ., http:/ LIBOL.stevenhoi.org, 2012.- [15] K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer, "Online Passive-Aggressive Algorithms,"
J. Machine Learning Research, vol. 7, pp. 551-585, 2006.- [16] S.C. Hoi, M.R. Lyu, and R. Jin, "A Unified Log-Based Relevance Feedback Scheme for Image Retrieval,"
IEEE Trans. Knowledge and Data Eng., vol. 18, no. 4, pp. 509-204, Apr. 2006.- [17] P. Wu, S.C.H. Hoi, P. Zhao, and Y. He, "Mining Social Images with Distance Metric Learning for Automated Image Tagging,"
Proc. Fourth ACM Int'l Conf. Web Search and Data Mining (WSDM '11), pp. 197-206, 2011.- [18] A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall, "Learning a Mahalanobis Metric from Equivalence Constraints,"
J. Machine Learning Research, vol. 6, pp. 937-965, 2005.- [19] E.P. Xing, A.Y. Ng, M.I. Jordan, and S. Russell, "Distance Metric Learning with Application to Clustering with Side-Information,"
Proc. Conf. Advances in Neural Information Processing Systems, 2002.- [20] K. Weinberger, J. Blitzer, and L. Saul, "Distance Metric Learning for Large Margin Nearest Neighbor Classification,"
Proc. Conf. Advances in Neural Information Processing Systems, pp. 1473-1480, 2006.- [21] L. Wu, S.C.H. Hoi, R. Jin, J. Zhu, and N. Yu, "Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging,"
Proc. 17th ACM Int'l Conf. Multimedia, pp. 135-144, 2009.- [22] L. Yang, R. Jin, L.B. Mummert, R. Sukthankar, A. Goode, B. Zheng, S.C.H. Hoi, and M. Satyanarayanan, "A Boosting Framework for Visuality-Preserving Distance Metric Learning and Its Application to Medical Image Retrieval,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 30-44, 2010.- [23] A. Globerson and S. Roweis, "Metric Learning by Collapsing Classes,"
Proc. Ann. Conf. Advances in Neural Information Processing Systems, 2005.- [24] A. Frome, Y. Singer, F. Sha, and J. Malik, "Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification,"
Proc. 11th IEEE Int'l Conf. Computer Vision (ICCV '07), pp. 1-8, 2007.- [25] J.V. Davis, B. Kulis, P. Jain, S. Sra, and I.S. Dhillon, "Information-Theoretic Metric Learning,"
Proc. 24th Int'l Conf. Machine Learning (ICML '07), pp. 209-216, 2007.- [26] G. Chechik, V. Sharma, U. Shalit, and S. Bengio, "Large Scale Online Learning of Image Similarity through Ranking,"
J. Machine Learning Research, vol. 11, pp. 1109-1135, 2010.- [27] S. Tong and E. Chang, "Support Vector Machine Active Learning for Image Retrieval,"
Proc. Ninth ACM Int'l Conf. Multimedia, pp. 107-118, 2001.- [28] H. Chang and D.-Y. Yeung, "Kernel-Based Distance Metric Learning for Content-Based Image Retrieval,"
Image Vision Computing, vol. 25, no. 5, pp. 695-703, 2007.- [29] D.-Y. Yeung and H. Chang, "A Kernel Approach for Semisupervised Metric Learning,"
IEEE Trans. Neural Networks, vol. 18, no. 1, pp. 141-149, Jan. 2007.- [30] T. Hertz, A. Bar-Hillel, and D. Weinshall, "Learning a Kernel Function for Classification with Small Training Samples,"
Proc. 23rd Int'l Conf. Machine Learning (ICML '06), pp. 401-408. 2006.- [31] S.C.H. Hoi, R. Jin, and M.R. Lyu, "Learning Nonparametric Kernel Matrices from Pairwise Constraints,"
Proc. Int'l Conf. Machine Learning, pp. 361-368. 2007.- [32] J. Zhuang, I.W. Tsang, and S.C.H. Hoi, "A Family of Simple Non-Parametric Kernel Learning Algorithms,"
J. Machine Learning Research, vol. 12, pp. 1313-1347, July 2011.- [33] G.R.G. Lanckriet, N. Cristianini, P. Bartlett, L.E. Ghaoui, and M.I. Jordan, "Learning the Kernel Matrix with Semidefinite Programming,"
J. Machine Learning Research, vol. 5, pp. 27-72, 2004.- [34] S. Sonnenburg, G. Rätsch, C. Schäfer, and B. Schölkopf, "Large Scale Multiple Kernel Learning,"
J. Machine Learning Research, vol. 7, pp. 1531-1565, 2006.- [35] Z. Xu, R. Jin, I. King, and M.R. Lyu, "An Extended Level Method for Efficient Multiple Kernel Learning,"
Proc. Ann. Conf. Advances in Neural Information Processing Systems, 2008.- [36] A. Zien and C.S. Ong, "Multiclass Multiple Kernel Learning,"
Proc. Int'l Conf. Machine Learning, pp. 1191-1198, 2007.- [37] S. Ji, L. Sun, R. Jin, and J. Ye, "Multi-Label Multiple Kernel Learning,"
Proc. Ann. Conf. Advances in Neural Information Processing Systems, 2008.- [38] M. Varma and B.R. Babu, "More Generality in Efficient Multiple Kernel Learning,"
Proc. Int'l Conf. Machine Learning, pp. 1065-1072, 2009.- [39] S.V.N. Vishwanathan, Z. sun, N. Ampornpunt, and M. Varma, "Multiple Kernel Learning and the SMO Algorithm,"
Proc. Ann. Conf. Advances in Neural Information Processing Systems (NIPS '10), pp. 2361-2369, 2010.- [40] A. Jain, S.V.N. Vishwanathan, and M. Varma, "Spg-gmkl: Generalized Multiple Kernel Learning with a Million Kernels,"
Proc. ACM Conf. Knowledge Discovery and Data Mining (SIGKDD '12), Aug. 2012.- [41] R. Jin, S.C.H. Hoi, and T. Yang, "Online Multiple Kernel Learning: Algorithms and Mistake Bounds,"
Proc. 21st Int'l Conf. Algorithmic Learning Theory (ALT '10), pp. 390-404, 2010.- [42] S.C.H. Hoi, R. Jin, P. Zhao, and T. Yang, "Online Multiple Kernel Classification,"
Machine Learning, vol. 90, no. 2, pp. 289-316, 2013.- [43] M. Varma and D. Ray, "Learning the Discriminative Power-Invariance Trade-Off,"
Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.- [44] A. Vedaldi, V. Gulshan, M. Varma, and A. Zisserman, "Multiple Kernels for Object Detection,"
Proc. IEEE 12th Int'l Conf. Computer Vision (ICCV '09), pp. 606-613, 2009.- [45] A. Quattoni and A. Torralba, "Recognizing Indoor Scenes,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 413-420, 2009.- [46] G. Griffin, A. Holub, and P. Perona, "Caltech-256 Object Category Data Set," technical report, California Inst. of Tech nology, p. 7694, 2007.
- [47] J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object Retrieval with Large Vocabularies and Fast Spatial Matching,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.- [48] A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall, "Learning Distance Functions Using Equivalence Relations,"
Proc. Int'l Conf. Machine Learning, pp. 11-18, 2003.- [49] I.W. Tsang, P. Ming Cheung, and J.T. Kwok, "Kernel Relevant Component Analysis for Distance Metric Learning,"
Proc. IEEE Int'l Joint Conf. Neural Networks (IJCNN '05), pp. 954-959, 2005.- [50] S.C.H. Hoi and M.R. Lyu, "A Multimodal and Multilevel Ranking Scheme for Large-Scale Video Retrieval,"
IEEE Trans. Multimedia, vol. 10, no. 4, pp. 607-619, June 2008.- [51] P. Zhao, S.C.H. Hoi, and R. Jin, "Double Updating Online Learning."
J. Machine Learning Research, vol. 12, pp. 1587-1615, 2011.- [52] P. Zhao, J. Wang, P. Wu, R. Jin, and S.C.H. Hoi, "Fast Bounded Online Gradient Descent Algorithms for Scalable Kernel-Based Online Learning,"
Proc. 29th Int'l Conf. Machine Learning (ICML '12), 2012. |