The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - August (2010 vol.22)
pp: 1191-1199
Shikui Wei , Beijing Jiaotong University, Beijing
Yao Zhao , Beijing Jiaotong University, Beijing
Zhenfeng Zhu , Beijing Jiaotong University, Beijing
Nan Liu , Beijing Jiaotong University, Beijing
ABSTRACT
Analysis on click-through data from a very large search engine log shows that users are usually interested in the top-ranked portion of returned search results. Therefore, it is crucial for search engines to achieve high accuracy on the top-ranked documents. While many methods exist for boosting video search performance, they either pay less attention to the above factor or encounter difficulties in practical applications. In this paper, we present a flexible and effective reranking method, called CR-Reranking, to improve the retrieval effectiveness. To offer high accuracy on the top-ranked results, CR-Reranking employs a cross-reference (CR) strategy to fuse multimodal cues. Specifically, multimodal features are first utilized separately to rerank the initial returned results at the cluster level, and then all the ranked clusters from different modalities are cooperatively used to infer the shots with high relevance. Experimental results show that the search quality, especially on the top-ranked results, is improved significantly.
INDEX TERMS
Clustering, image/video retrieval, multimedia databases.
CITATION
Shikui Wei, Yao Zhao, Zhenfeng Zhu, Nan Liu, "Multimodal Fusion for Video Search Reranking", IEEE Transactions on Knowledge & Data Engineering, vol.22, no. 8, pp. 1191-1199, August 2010, doi:10.1109/TKDE.2009.145
REFERENCES
[1] M.S. Lew, N. Sebe, C. Djeraba, and R. Jain, "Content-Based Multimedia Information Retrieval: State of the Art and Challenges," ACM Trans. Multimedia Computing, Comm., and Applications, vol. 2, pp. 1-19, 2006.
[2] A. Smeaton and T. Ianeva, "TRECVID-2006 Search Task," TREC Video Retrieval Evaluation Online Proc., 2006.
[3] T. Joachims, "Optimizing Search Engines Using Clickthrough Data," Proc. ACM SIGKDD, pp. 133-142, 2002.
[4] C. Silverstein, H. Marais, M. Henzinger, and M. Moricz, "Analysis of a Very Large Web Search Engine Query Log," ACM SIGIR Forum, vol. 33, pp. 6-12, 1999.
[5] Y. Cao, J. Xu, T.-Y. Liu, H. Li, Y. Huang, and H.-W. Hon, "Adapting Ranking SVM to Document Retrieval," Proc. 29th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, 2006.
[6] C.G.M. Snoek, J.C. van Gemert, J.M. Geusebroek, B. Huurnink, D.C. Koelma, G.P. Nguyen, O. de Rooij, F.J. Seinstra, A.W.M. Smeulders, C.J. Veenman, and M. Worring, "The MediaMill TRECVID 2005 Semantic Video Search Engine," TREC Video Retrieval Evaluation Online Proc., 2005.
[7] A. Amir, J. Argillander, M. Campbell, A. Haubold, G. Iyengar, S. Ebadollahi, F. Kang, M.R. Naphade, A. Natsev, J.R. Smith, J. Tešić, and T. Volkmer, "IBM Research TRECVID-2005 Video Retrieval System," TREC Video Retrieval Evaluation Online Proc., 2005.
[8] S.F. Chang, W.H. Hsu, L. Kennedy, L. Xie, A. Yanagawa, E. Zavesky, and D.-Q. Zhang, "Columbia University TRECVID-2005Video Search and High-Level Feature Extraction," TREC Video Retrieval Evaluation Online Proc., 2005.
[9] A.G. Hauptmann, M. Christel, R. Concescu, J. Gao, Q. Jin, W.-H. Lin, J.-Y. Pan, S.M. Stevens, R. Yan, J. Yang, and Y. Zhang, "CMU Informedia's TRECVID 2005 Skirmishes," TREC Video Retrieval Evaluation Online Proc., 2005.
[10] J.H. Yuan, W.J. Zheng, L. Chen, D.Y. Ding, D. Wang, Z.J. Tong, H.Y. Wang, J. Wu, J.M. Lin, and B. Zhang, "Tsinghua University at TRECVID 2005," TREC Video Retrieval Evaluation Online Proc., 2005.
[11] S.K. Wei, Y. Zhao, Z.F. Zhu, N. Liu, Y.F. Zhao, L. Zhang, and F. Wang, "BJTU TRECVID 2006 Video Retrieval System," TREC Video Retrieval Evaluation Online Proc., 2006.
[12] W.H. Hsu, L.S. Kennedy, and S.-F. Chang, "Reranking Methods for Visual Search," IEEE Trans. Multimedia, vol. 14, no. 3, pp. 14-22, July-Sept. 2007.
[13] R. Yan and A.G. Hauptmann, "Co-Retrieval: A Boosted Reranking Approach for Video Retrieval," IEE Proc. Vision, Image and Signal Processing, vol. 152, pp. 888-895, 2005.
[14] J.H. Lee, "Analyses of Multiple Evidence Combination," ACM SIGIR Forum, vol. 31, pp. 267-276, 1997.
[15] J.A. Aslam and M. Montague, "Models for Metasearch," Proc. 24th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 276-284, 2001.
[16] J. Liu, W. Lai, X.S. Hua, Y. Huang, and S. Li, "Video Search Re-Ranking via Multi-Graph Propagation," Proc. 15th Int'l Conf. Multimedia, pp. 208-217, 2007.
[17] A. Haubold, A. Natsev, and M.R. Naphade, "Semantic Multimedia Retrieval Using Lexical Query Expansion and Model-Based Reranking," Proc. IEEE Int'l Conf. Multimedia and Expo, 2006.
[18] L. Page, S. Brin, R. Motwani, and T. Winograd, "The PageRank Citation Ranking: Bringing Order to the Web," Stanford Digital Libraries Working Paper, 1998.
[19] T.H. Haveliwala, "Topic-Sensitive PageRank: A Context-Sensitive Ranking Algorithm for Web Search," IEEE Trans. Knowledge and Data Eng., vol. 15, no. 4, pp. 784-796, July/Aug. 2003.
[20] J.M. Kleinberg, "Authoritative Sources in a Hyperlinked Environment," J. ACM, vol. 46, pp. 604-632, 1999.
[21] W.H. Hsu, L.S. Kennedy, and S.-F. Chang, "Video Search Reranking via Information Bottleneck Principle," Proc. 14th Ann. Int'l Conf. Multimedia, pp. 35-44, 2006.
[22] L. Kennedy, M. Naaman, S. Ahern, R. Nair, and T. Rattenbury, "How Flickr Helps Us Make Sense of the World: Context and Content in Community-Contributed Media Collections," Proc. 15th Int'l Conf. Multimedia, pp. 631-640, 2007.
[23] A. Blum and T. Mitchell, "Combining Labeled and Unlabeled Data with Co-Training," Proc. 11th Ann. Conf. Computational Learning Theory, pp. 92-100, 1998.
[24] K. Nigam and R. Ghani, "Understanding the Behavior of Co-Training," Proc. KDD-2000 Workshop Text Mining, 2000.
[25] Z.H. Zhou and M. Li, "Tri-Training: Exploiting Unlabeled Data Using Three Classifiers," IEEE Trans. Knowledge and Data Eng., vol. 17, no. 11, pp. 1529-1541, Nov. 2005.
[26] I. Muslea, S. Minton, and C. Knoblock, "Active+ Semi-Supervised Learning $=$ Robust Multi-View Learning," Proc. Int'l Conf. Machine Learning, pp. 435-442, 2002.
[27] R. Yan and M. Naphade, "Multi-Modal Video Concept Extraction Using Co-Training," Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 514-517, 2005.
[28] TRECVID, "TREC Video Retrieval Evaluation," http:// www-nlpir.nist.gov/ projectstrecvid/, 2009.
[29] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[30] D.P. Huttenlocher, G.A. Klanderman, and W.J. Rucklidge, "Comparing Images Using the Hausdorff Distance," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 9, pp. 850-863, Sept. 1993.
[31] A. Yanagawa, W.H. Hsu, and S.-F. Chang, "Brief Descriptions of Visual Features for Baseline TRECVID Concept Detectors," ADVENT Technical Report #219-2006-5, Columbia Univ., July 2006.
6 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool