The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2009 vol.21)
pp: 321-334
Heng Tao Shen , The University of Queensland, Brisbane
Jie Shao , The University of Queensland, Brisbane
Zi Huang , The University of Queensland, Brisbane
Xiaofang Zhou , The University of Queensland, Brisbane
ABSTRACT
Content-based video retrieval has been well investigated. However, despite the importance, few studies on video subsequence identification, which is to find the similar content to a short query clip from a long video sequence, have been published. This paper presents a graph transformation and matching approach to this problem, with extension to identify the occurrence of potentially different ordering, alignment or length due to content editing. With a batch query algorithm to retrieve similar frames, the mapping relationship between the query and the database video is first represented by a bipartite graph. The densely matched parts along the long sequence are then extracted, followed by a filter-and-refine search strategy to prune some irrelevant subsequences. During the filtering stage, Maximum Size Matching (MSM) is deployed for each subgraph constructed by the query and candidate subsequence to obtain a smaller set of candidates. During the refinement stage, Sub-Maximum Similarity Matching (SMSM) is devised to identify the subsequence, according to a robust video similarity model which incorporates visual content, temporal order, frame alignment and length information. The performance studies conducted on a long and diverse video recording validate our approach is promising in terms of both search accuracy and speed.
INDEX TERMS
Multimedia databases, Search process, Information filtering
CITATION
Heng Tao Shen, Jie Shao, Zi Huang, Xiaofang Zhou, "Effective and Efficient Query Processing for Video Subsequence Identification", IEEE Transactions on Knowledge & Data Engineering, vol.21, no. 3, pp. 321-334, March 2009, doi:10.1109/TKDE.2008.168
REFERENCES
[1] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-Based Image Retrieval at the End of the Early Years,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
[2] C. Faloutsos, M. Ranganathan, and Y. Manolopoulos, “Fast Subsequence Matching in Time-Series Databases,” Proc. ACM SIGMOD '94, pp. 419-429, 1994.
[3] H. Wang, A. Divakaran, A. Vetro, S.-F. Chang, and H. Sun, “Survey of Compressed-Domain Features Used in Audio-Visual Indexing and Analysis,” J. Visual Comm. and Image Representation, vol. 14, no. 2, pp. 150-183, 2003.
[4] R. Mohan, “Video Sequence Matching,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing (ICASSP '98), pp. 3697-3700, 1998.
[5] C. Kim and B. Vasudev, “Spatiotemporal Sequence Matching for Efficient Video Copy Detection,” IEEE Trans. Circuits and Systems for Video Technology, vol. 15, no. 1, pp. 127-132, 2005.
[6] X.-S. Hua, X. Chen, and H. Zhang, “Robust Video Signature Based on Ordinal Measure,” Proc. IEEE Int'l Conf. Image Processing (ICIP'04), pp. 685-688, 2004.
[7] C.-Y. Chiu, C.-H. Li, H.-A. Wang, C.-S. Chen, and L.-F. Chien, “A Time Warping Based Approach for Video Copy Detection,” Proc. 18th Int'l Conf. Pattern Recognition (ICPR '06), vol. 3, pp. 228-231, 2006.
[8] M.R. Naphade, M.M. Yeung, and B.-L. Yeo, “A Novel Scheme for Fast and Efficient Video Sequence Matching Using Compact Signatures,” Proc. Storage and Retrieval for Image and Video Databases (SPIE '00), pp. 564-572, 2000.
[9] K.M. Pua, J.M. Gauch, S. Gauch, and J.Z. Miadowicz, “Real Time Repeated Video Sequence Identification,” Computer Vision and Image Understanding, vol. 93, no. 3, pp. 310-327, 2004.
[10] K. Kashino, T. Kurozumi, and H. Murase, “A Quick Search Method for Audio and Video Signals Based on Histogram Pruning,” IEEE Trans. Multimedia, vol. 5, no. 3, pp. 348-357, 2003.
[11] S.-C.S. Cheung and A. Zakhor, “Efficient Video Similarity Measurement with Video Signature,” IEEE Trans. Circuits and Systems for Video Technology, vol. 13, no. 1, pp. 59-74, 2003.
[12] S.-C.S. Cheung and A. Zakhor, “Fast Similarity Search and Clustering of Video Sequences on the World-Wide-Web,” IEEE Trans. Multimedia, vol. 7, no. 3, pp. 524-537, 2005.
[13] H.T. Shen, B.C. Ooi, X. Zhou, and Z. Huang, “Towards Effective Indexing for Very Large Video Sequence Database,” Proc. ACM SIGMOD '05, pp. 730-741, 2005.
[14] H.T. Shen, X. Zhou, Z. Huang, J. Shao, and X. Zhou, “Uqlips: A Real-Time Near-Duplicate Video Clip Detection System,” Proc. 33rd Int'l Conf. Very Large Databases (VLDB '07), pp. 1374-1377, 2007.
[15] J. Yuan, L.-Y. Duan, Q. Tian, S. Ranganath, and C. Xu, “Fast and Robust Short Video Clip Search for Copy Detection,” Proc. Fifth IEEE Pacific-Rim Conf. Multimedia (PCM '04), vol. 2, pp. 479-488, 2004.
[16] A. Hampapur, K.-H. Hyun, and R.M. Bolle, “Comparison of Sequence Matching Techniques for Video Copy Detection,” Proc. Storage and Retrieval for Image and Video Databases (SPIE '02), pp.194-201, 2002.
[17] T.C. Hoad and J. Zobel, “Detection of Video Sequences Using Compact Signatures,” ACM Trans. Information Systems, vol. 24, no. 1, pp. 1-50, 2006.
[18] J. Shao, Z. Huang, H.T. Shen, X. Zhou, E.-P. Lim, and Y. Li, “Batch Nearest Neighbor Search for Video Retrieval,” IEEE Trans. Multimedia, vol. 10, no. 3, pp. 409-420, 2008.
[19] Y. Peng and C.-W. Ngo, “Clip-Based Similarity Measure for Query-Dependent Clip Retrieval and Video Summarization,” IEEE Trans. Circuits and Systems for Video Technology, vol. 16, no. 5, pp. 612-627, 2006.
[20] D.R. Shier, “Matchings and Assignments,” Handbook of Graph Theory, J.L. Gross and J. Yellen, eds., pp. 1103-1116, CRC Press, 2004.
[21] L. Chen and T.-S. Chua, “A Match and Tiling Approach to Content-Based Video Retrieval,” Proc. IEEE Int'l Conf. Multimedia and Expo (ICME '01), pp. 417-420, 2001.
[22] X. Liu, Y. Zhuang, and Y. Pan, “A New Approach to Retrieve Video by Example Video Clip,” Proc. Seventh ACM Int'l Conf. Multimedia (MULTIMEDIA '99), vol. 2, pp. 41-44, 1999.
[23] Y. Wu, Y. Zhuang, and Y. Pan, “Content-Based Video Similarity Model,” Proc. Eighth ACM Int'l Conf. Multimedia (MULTIMEDIA '00), pp. 465-467, 2000.
[24] S.-L. Lee, S.-J. Chun, D.-H. Kim, J.-H. Lee, and C.-W. Chung, “Similarity Search for Multidimensional Data Sequences,” Proc. 16th Int'l Conf. Data Eng. (ICDE '00), pp. 599-608, 2000.
[25] E.J. Keogh, “Exact Indexing of Dynamic Time Warping,” Proc. 28th Int'l Conf. Very Large Data Bases (VLDB '02), pp. 406-417, 2002.
[26] M. Vlachos, D. Gunopulos, and G. Kollios, “Discovering Similar Multidimensional Trajectories,” Proc. 18th Int'l Conf. Data Eng. (ICDE '02), pp. 673-684, 2002.
[27] D.A. Adjeroh, M.-C. Lee, and I. King, “A Distance Measure for Video Sequences,” Computer Vision and Image Understanding, vol. 75, nos. 1-2, pp. 25-45, 1999.
[28] L. Chen and R.T. Ng, “On the Marriage of LP-Norms and Edit Distance,” Proc. 30th Int'l Conf. Very Large Data Bases (VLDB '04), pp. 792-803, 2004.
[29] D. DeMenthon, V. Kobla, and D.S. Doermann, “Video Summarizationby Curve Simplification,” Proc. Sixth ACM Int'l Conf. Multimedia (MULTIMEDIA '98), pp. 211-218, 1998.
[30] M. Vlachos, M. Hadjieleftheriou, D. Gunopulos, and E.J. Keogh, “Indexing Multidimensional Time-Series,” The VLDB J., vol. 15, no. 1, pp. 1-20, 2006.
[31] J. Zhou and X.-P. Zhang, “Automatic Identification of Digital Video Based on Shot-Level Sequence Matching,” Proc. 13th ACM Int'l Conf. Multimedia (MULTIMEDIA '05), pp. 515-518, 2005.
[32] S. Santini and R. Jain, “Similarity Measures,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 9, pp. 871-883, Sept. 1999.
[33] Y. Rubner, C. Tomasi, and L.J. Guibas, “A Metric for Distributions with Applications to Image Databases,” Proc. Int'l Conf. Computer Vision (ICCV '98), pp. 59-66, 1998.
[34] H. Ehrig and G. Taentzer, “Computing by Graph Transformation: A Survey and Annotated Bibliography,” Technical Report 96-21, TU Berlin, 1996.
[35] A. Mehta, A. Saberi, U.V. Vazirani, and V.V. Vazirani, “Adwords and Generalized On-Line Matching,” Proc. 46th IEEE Symp. Foundations of Computer Science (FOCS '05), pp. 264-273, 2005.
[36] A. Kini, S. Shankar, J.F. Naughton, and D.J. DeWitt, “Database Support for Matching: Limitations and Opportunities,” Proc. ACM SIGMOD '06, pp. 85-96, 2006.
7 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool