The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - Feb. (2013 vol.25)
pp: 448-460
Xin Jin , University of Illinois at Urbana Champaign, Champaign
Jiebo Luo , Eastman Kodak Company, Rochester
Jie Yu , GE Global Research, Niskayuna
Gang Wang , Nanyang Technological University, and Advanced Digital Science Center, Singapore
Dhiraj Joshi , Eastman Kodak Company, Rochester
Jiawei Han , University of Illinois at Urbana Champaign, Champaign
ABSTRACT
Social multimedia sharing and hosting websites, such as Flickr and Facebook, contain billions of user-submitted images. Popular Internet commerce websites such as Amazon.com are also furnished with tremendous amounts of product-related images. In addition, images in such social networks are also accompanied by annotations, comments, and other information, thus forming heterogeneous image-rich information networks. In this paper, we introduce the concept of (heterogeneous) image-rich information network and the problem of how to perform information retrieval and recommendation in such networks. We propose a fast algorithm heterogeneous minimum order k-SimRank (HMok-SimRank) to compute link-based similarity in weighted heterogeneous information networks. Then, we propose an algorithm Integrated Weighted Similarity Learning (IWSL) to account for both link-based and content-based similarities by considering the network structure and mutually reinforcing link similarity and feature weight learning. Both local and global feature learning methods are designed. Experimental results on Flickr and Amazon data sets show that our approach is significantly better than traditional methods in terms of both relevance and speed. A new product search and recommendation system for e-commerce has been implemented based on our algorithm.
INDEX TERMS
Complexity theory, Semantics, Vectors, Mathematical model, Visualization, Equations, Image edge detection, ranking, Information retrieval, image mining, information network
CITATION
Xin Jin, Jiebo Luo, Jie Yu, Gang Wang, Dhiraj Joshi, Jiawei Han, "Reinforced Similarity Integration in Image-Rich Information Networks", IEEE Transactions on Knowledge & Data Engineering, vol.25, no. 2, pp. 448-460, Feb. 2013, doi:10.1109/TKDE.2011.228
REFERENCES
[1] X. Jin, J. Luo, J. Yu, G. Wang, D. Joshi, and J. Han, "iRIN: Image Retrieval in Image-Rich Information Networks," Proc. 19th Int'l Conf. World Wide Web (WWW '10), pp. 1261-1264, 2010.
[2] R.L. Cilibrasi and P.M.B. Vitanyi, "The Google Similarity Distance," IEEE Trans. Knowledge and Data Eng., vol. 19, no. 3, pp. 370-383, Mar. 2007.
[3] L. Wu, X.-S. Hua, N. Yu, W.-Y. Ma, and S. Li, "Flickr Distance," Proc. 16th ACM Int'L conf. Multimedia, pp. 31-40, 2008.
[4] Y. Jing and S. Baluja, "VisualRank: Applying Pagerank to Large-Scale Image Search," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 11, pp. 1877-1890, Nov. 2008.
[5] R.C. Veltkamp and M. Tanase, "Content-Based Image Retrieval Systems: A Survey," technical report, Dept. of Computing Science, Utrecht Univ., 2002.
[6] H. Tamura and N. Yokoya, "Image Database Systems: A Survey," Pattern Recognition, vol. 17, no. 1, pp. 29-43, 1984.
[7] W.I. Grosky, "Multimedia Information Systems," IEEE MultiMedia, vol. 1, no. 1, pp. 12-24, Spring, 1994.
[8] V.N. Gudivada and V.V. Raghavan, "Content-Based Image Retrieval Systems," Computer, vol. 28, no. 9, pp. 18-22, Sept. 1995.
[9] Y. Rui, T.S. Huang, and S.-F. Chang, "Image Retrieval: Current Techniques, Promising Directions, and Open Issues," J. Visual Comm. and Image Representation, vol. 10, no. 1, pp. 39-62, 1999.
[10] R. Datta, D. Joshi, J. Li, and J.Z. Wang, "Image Retrieval: Ideas, Influences, and Trends of the New Age," ACM Computing Surveys, vol. 40, no. 2, pp. 1-60, Apr. 2008.
[11] T. Deselaers and H. Mller, "Combining Textual- and Content-Based Image Retrieval, Tutorial," Proc. 19th Int'l Conf. Pattern Recognition (ICPR '08), http://thomas.deselaers.de/teaching/files/ tutorial_icpr0804-combinatio n.pdf, 2008.
[12] S. Sclaroff, M. La Cascia, and S. Sethi, "Unifying Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web," Computing Vision Image Understanding, vol. 75, pp. 86-98, July 1999.
[13] Z. Ye, X. Huang, Q. Hu, and H. Lin, "An Integrated Approach for Medical Image Retrieval through Combining Textual and Visual Features," Proc. 10th Int'l Conf. Cross-Language Evaluation Forum: Multimedia Experiments (CLEF '09), pp. 195-202, 2010.
[14] G. Jeh and J. Widom, "SimRank: A Measure of Structural-Context Similarity," Proc. Eighth Int'l Conf. Knowledge Discovery and Data Mining (KDD '02), 2002.
[15] L. Page, S. Brin, R. Motwani, and T. Winograd, "The Pagerank Citation Ranking: Bringing Order to the Web," technical report, Stanford InfoLab, 1999.
[16] D. Lizorkin, P. Velikhov, M. Grinev, and D. Turdakov, "Accuracy Estimate and Optimization Techniques for Simrank Computation," VLDB Endowment, vol. 1, no. 1, pp. 422-433, 2008.
[17] D. Fogaras and B. Racz, "Scaling Link-Based Similarity Search," Proc. 14th Int'l Conf. World Wide Web, pp. 641-650, 2005.
[18] J. Wang, H.-J. Zeng, Z. Chen, H. Lu, L. Tao, and W.-Y. Ma, "ReCoM: Reinforcement Clustering of Multi-Type Interrelated Data Objects," Proc. 26th ACM Ann. SIGIR Conf. Research and Development in Information Retrieval, pp. 274-281, 2003.
[19] X. Yin, J.H., and P.S. Yu, "LinkClus: Efficient Clustering via Heterogeneous Semantic Links," Proc. 32nd Int'l Conf. Very Large Data Bases, pp. 427-438, 2006.
[20] T. Deselaers, D. Keysers, and H. Ney, "Features for Image Retrieval: An Experimental Comparison," Information Retrieval, vol. 11, no. 2, pp. 77-107, 2008.
[21] Z. Yang and C.-C.J. Kuo, "Survey on Image Content Analysis, Indexing, and Retrieval Techniques and Status Report of Mpeg-7," Tamkang J. Science and Eng., vol. 3, no. 2, pp. 101-118, 1999.
[22] R.C. Ltd., "MINDS's Descriptors for Still Images - Spatial Edge Distribution Descriptor," ISO/IEC/JTC1/SC29/WG11, Lancaster, United Kingdom, p. 109, 1999.
[23] J. Huang, S.R. Kumar, M. Mitra, W.-J. Zhu, and R. Zabih, "Image Indexing Using Color Correlograms," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR '97), pp. 762-768, 1997.
[24] S. Chatzichristofis and Y. Boutalis, "CEDD: Color and Edge Directivity Descriptor: A Compact Descriptor for Image Indexing and Retrieval," Proc. Sixth Int'l Conf. Computer Vision Systems, pp. 312-322, 2008.
[25] S. Aksoy and R.M. Haralick, "Textural Features for Image Data base Retrieval," Proc. IEEE Workshop Content - Based Access of Image and Video Libraries (CBAIVL '98), p. 45, 1998.
[26] K. Muller and J.R. Ohm, "Wavelet-Based Contour Descriptor," ISO/IEC/JTC1/SC29/WG11, Lancaster, Uinted Kingdom, p. 567, 1999.
[27] M. Park, J.S. Jin, and L.S. Wilson, "Fast Content-Based Image Retrieval Using Quasi-Gabor Filter and Reduction of Image Feature Dimension," Proc. IEEE Fifth Southwest Symp. Image Analysis and Interpretation (SSIAI '02), p. 178, 2002.
[28] D.M. Squire, W. Muler, H. Muler, and T. Pun, "Content-Based Query of Image Databases: Inspirations from Text Retrieval," Pattern Recognition Letters - Selected Papers from 11th Scandinavian Conf. Image, vol. 21, nos. 13/14, pp. 1193-1198, 2000.
[29] T., Inc. "Normalized Contour as a Shape Descriptor for Visual Objects," ISO/IEC/JTC1/SC29/WG11, Lancaster, United Kingdom, p. 579, 1999.
[30] W.Y. Kim and Y.S. Kim, "A Rotation Invariant Geometric Shape Descriptor Using Zernike Moment," ISO/IEC/JTC1/SC29/WG11, Lancaster, United Kingdom, p. 687, 1999.
[31] D.G. Lowe, "Object Recognition from Local Scale-Invariant Features," Proc. IEEE Seventh Int'l Conf. Computer Vision, vol. 2, pp. 1150-1157, 1999.
[32] J. Tang, H. Li, G.-J. Qi, and T.-S. Chua, "Image Annotation by Graph-Based Inference with Integrated Multiple/Single Instance Representations," IEEE Trans. Multimedia, vol. 12, no. 2, pp. 131-141, Feb. 2010.
[33] L. Yang, R. Jin, L. Mummert, R. Sukthankar, A. Goode, B. Zheng, S.C. Hoi, and M. Satyanarayanan, "A Boosting Framework for Visuality-Preserving Distance Metric Learning and its Application to Medical Image Retrieval," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 30-44, Jan. 2010.
[34] E. Xing, A. Ng, M. Jordan, and S. Russell, "Distance Metric Learning with Applications to Clustering with Side Information," Proc. 16th Conf. Advances in Neural Information Processing Systems, 2002.
[35] L. Wu, S.C. Hoi, R. Jin, J. Zhu, and N. Yu, "Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging," Proc. 17th ACM Int'l Conf. Multimedia, pp. 135-144, 2009.
[36] B. Babenko, S. Branson, and S. Belongie, "Similarity Functions for Categorization: From Monolithic to Category Specific," Proc. IEEE 12th Int'l Conf. Computer Vision (ICCV '09), 2009.
[37] L. Yang and A.R. Jin, "Contents Distance Metric Learning: A Comprehensive Survey," technical report, Dept. of Computer Science and Eng., Michigan State Univ., 2006.
[38] S. Belongie, J. Malik, and J. Puzicha, "Shape Matching and Object Recognition Using Shape Contexts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002.
[39] G. Wang and D. Forsyth, "Object Image Retrieval by Exploiting Online Knowledge Resources," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1-8, 2008.
[40] J.R.R. Uijlings, A.W.M. Smeulders, and R.J.H. Scha, "Real-Time Bag of Words, Approximately," Proc. ACM Int'l Conf. Image and Video Retrieval (CIVR '09), pp. 6:1-6:8, 2009.
[41] Y.-G. Jiang, C.-W. Ngo, and J. Yang, "Towards Optimal Bag-of-Features for Object Categorization and Semantic Video Retrieval," Proc. Sixth ACM Int'l Conf. Image and Video Retrieval (CIVR '07), pp. 494-501, 2007.
[42] J. Zhang, M. Marszalek, S. Lazebnik, and C. Schmid, "Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study," Int'l J. Computer Vision, vol. 73, pp. 213-238, June 2007.
[43] S. Chakrabarti, B. Dom, P. Raghavan, and S. Rajagopalan, "Automatic Resource Compilation by Analyzing Hyperlink Structure and Associated Text," Proc. Seventh World Wide Web Conf. (WWW '97), pp. 65-74, 1997.
[44] J.L. Bentley, "Multidimensional Divide-and-Conquer," Comm. ACM, vol. 23, no. 4, pp. 214-229, Apr. 1980.
[45] M. Datar, N. Immorlica, P. Indyk, and V. Mirrokni, "Locality Sensitive Hashing Scheme Based on P-Stable Distribution," Proc. 20th Ann. Symp. Computational Geometry, 2004.
[46] A. Beygelzimer, S. Kakade, and J. Langford, "Cover Trees for Nearest Neighbor," Proc. 23rd Int'l Conf. Machine Learning, pp. 97-104, 2006.
[47] M.F. Porter, "An Algorithm for Suffix Stripping," Program, vol. 14, no. 3, pp. 130-137, 1980.
[48] J. Yu, X. Jin, J. Han, and J. Luo, "Collection-Based Sparse Label Propagation and Its Application on Social Group Suggestion from Photos," ACM Trans. Intelligent Systems Technology, vol. 2, pp. 12:1-12:21, Feb. 2011.
44 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool