This Article 
 Bibliographic References 
 Add to: 
Hierarchical Indexing Structure for Efficient Similarity Search in Video Retrieval
November 2006 (vol. 18 no. 11)
pp. 1544-1559
With the rapid increase in both centralized video archives and distributed WWW video resources, content-based video retrieval is gaining its importance. To support such applications efficiently, content-based video indexing must be addressed. Typically, each video is represented by a sequence of frames. Due to the high dimensionality of frame representation and the large number of frames, video indexing introduces an additional degree of complexity. In this paper, we address the problem of content-based video indexing and propose an efficient solution, called the Ordered VA-File (OVA-File) based on the VA-file. OVA-File is a hierarchical structure and has two novel features: 1) partitioning the whole file into slices such that only a small number of slices are accessed and checked during k Nearest Neighbor (kNN) search and 2) efficient handling of insertions of new vectors into the OVA-File, such that the average distance between the new vectors and those approximations near that position is minimized. To facilitate a search, we present an efficient approximate kNN algorithm named Ordered VA-LOW (OVA-LOW) based on the proposed OVA-File. OVA-LOW first chooses possible OVA-Slices by ranking the distances between their corresponding centers and the query vector, and then visits all approximations in the selected OVA-Slices to work out approximate kNN. The number of possible OVA-Slices is controlled by a user-defined parameter \delta. By adjusting \delta, OVA-LOW provides a trade-off between the query cost and the result quality. Query by video clip consisting of multiple frames is also discussed. Extensive experimental studies using real video data sets were conducted and the results showed that our methods can yield a significant speed-up over an existing VA-file-based method and iDistance with high query result quality. Furthermore, by incorporating temporal correlation of video content, our methods achieved much more efficient performance.

[1] I. Koprinska and S. Carrato, “Temporal Video Segmentation: A Survey,” Signal Processing: Image Comm., vol. 16, pp. 477-500, 2001.
[2] A. Hanjalic, “Shot-Boundary Detection: Unraveled and Resolved?” IEEE Trans. Circuits and Systems for Video Technology, vol. 12, no. 2, pp. 90-105, 2002.
[3] A. Girgensohn and J.S. Boreczky, “Time-Constrained Keyframe Selection Technique,” Multimedia Tools and Applications, vol. 11, no. 3, pp. 347-358, 2000.
[4] T. Liu and J.R. Kender, “Optimization Algorithms for the Selection of Key Frame Sequences of Variable Length,” Proc. European Conf. Computer Vision, pp. 403-417, 2002.
[5] N. Dimitrova, H.-J. Zhang, B. Shahraray, M. Sezan, T. Huang, and A. Zakoh, “Applications of Video-Content Analysis and Retrieval,” IEEE Trans. Multimedia, vol. 9, no. 3, pp. 42-55, 2002.
[6] Y.A. Aslandagan and C.T. Yu, “Techniques and Systems for Image and Video Retrieval,” IEEE Trans. Knowledge and Data Eng., vol. 11, no. 1, pp. 56-63, Jan./Feb. 2002.
[7] G. Lu, “Techniques and Data Structures for Efficient Multimedia Retrieval Based on Similarity,” IEEE Trans. Multimedia, vol. 4, no. 3, pp. 372-384, 2002.
[8] J.S. Boreczky and L.A. Rowe, “Comparison of Video Shot Boundary Detection Techniques,” SPIE Proc. Storage and Retrieval for Still Image and Video Databases IV, vol. 2670, pp. 170-179, Mar. 1996.
[9] U. Gargi, R. Kasturi, and S.H. Strayer, “Performance Characterization of Video-Shot-Change Detection Methods,” IEEE Trans. Circuits and Systems for Video Technology, vol. 10, no. 1, pp. 1-13, 2000.
[10] C. Böhm, S. Berchtold, and D. Keim, “Searching in High-Dimensional Space: Index Structures for Improving the Performance of Multimedia Databases,“ vol. 33, no. 3, pp. 322-373, 2001.
[11] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” Computer, vol. 28, no. 9, pp. 23-32, 1995.
[12] H.J. Zhang, C.Y. Low, S.W. Smoliar, and J.H. Wu, “Video Parsing, Retrieval, and Browsing: An Integrated and Content Based Solution,” ACM Multimedia, pp. 15-24, 1995.
[13] S. Cheung and A. Zakhor, “Efficient Video Similarity Measurement with Video Signature,” vol. 13, no. 1, pp. 59-74, 2003.
[14] H.T. Shen, B.C. Ooi, X. Zhou, and Z. Huang, “Towards Effective Indexing for Very Large Video Sequence Database,” Proc. ACM SIGMOD, pp. 730-741, 2005.
[15] M.R. Naphade, R. Wang, and T.S. Huang, “Multimodal Pattern Matching for Audio-Visual Query and Retrieval,” Proc. Storage and Retrieval for Media Datbases (SPIE), pp. 188-195, 2001.
[16] H. Chang, S. Sull, and S. Lee, “Efficient Video Indexing Scheme for Content-Based Retrieval,” vol. 9, no. 8, pp. 1269-1279, 1999.
[17] P. Indyk, G. Iyengar, and N. Shivakumar, “Finding Pirated Video Sequences on the Iinternet,” technical report, Stanford Infolab, 1999.
[18] J. Oostveen, T. Keller, and J. Haitsma, “Feature Extraction and a Database Strategy for Video Fingerprinting,” Proc. Fifth Int'l Conf. Recent Advances in Visual Information Systems (VISUAL), vol. 2314, pp. 117-128, 2002.
[19] N. Koudas, B.C. Ooi, H.T. Shen, and A. Tung, “LDC: Enabling Search by Partial Distance in a Hyper-Dimensional Space,” Proc. Int'l Conf. Data Eng., pp. 6-17, 2004.
[20] R. Weber, H. Schek, and S. Blott, “A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces,” Proc. 24th Int'l Conf. Very Large Data Bases, pp. 194-205, 1998.
[21] G.H. Cha, X.M. Zhu, D. Petkovic, and C.W. Chung, “An Efficient Indexing Method for Nearest Neighbor Searches in High-Dimensional Image Databases,” IEEE Trans. Multimedia, vol. 4, no. 1, pp.76-87, 2002.
[22] E. Tuncle, H. Ferhatosmanoglu, and K. Rose, “VQ-Index: An Index Structure for Similarity Searching in Multimedia Databases,” ACM Multimedia, pp. 543-553, 2002.
[23] S. Berchtold, C. Böhm, H.V. Jagadish, H.P. Kriegel, and J. Sander, “Independent Quantization: An Index Compression Technique for High-Dimensional Data Spaces,” Proc. 16th Int'l Conf. Data Eng., pp. 577-588, 2000.
[24] G.H. Cha and C.W. Chuang, “The GC-Tree: A High-Dimensional Index Structure for Similarity Search in Image Databases,” IEEE Trans. Multimedia, vol. 4, no. 2, pp. 235-247, 2002.
[25] S. Arya, D.M. Mount, N. Netanyahu, R. Silverman, and A.Y. Wu, “An Optimal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions,” J. ACM, vol. 45, no. 6, pp. 891-923, 1998.
[26] P. Indyk and R. Motwani, “Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality,” Proc. Symp. Theory of Computing, pp. 604-613, 1998.
[27] E. Kushilevitz, R. Ostrovsky, and Y. Rabani, “Efficient Ssearch for Approximate Nearest Neighbor in High Dimensional Spaces,” Proc. Symp. Theory of Computing, pp. 614-623, 1998.
[28] A. Gionis, P. Indyk, and R. Motwani, “Similarity Search in High Dimensions via Hashing,” Proc. 28th Int'l Conf. Very Large Data Bases, pp. 518-529, 1999.
[29] Z. Yand, T. Ooi, and Q. Sun, “Hierarchical, Non-Uniform Locality Sensitive Hashing and Its Application to Video Identification,” Proc. Int'l Conf. Multimedia and Expo, pp. 743-746, 2004.
[30] R. Weber and K. Bohm, “Trading Quality for Time with Nearest-Neighbor Search,” Proc. Seventh Int'l Conf. Extending Database Technology, pp. 21-35, 2000.
[31] C. Yu, B.C. Ooi, K.L. Tan, and H. Jagadish, “Indexing the Distance: An Efficient Method to KNN Processing,” Proc. Int'l Conf. Very Large Data Bases, pp. 421-430, 2001.
[32] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. New York: Wiley-Interscience, 2000.
[33] Text of ISO/IEC 15938-3/FCD Information Technology—Multimedia Content Description Interface—Part 3 Visual, L. Cieplinski, M. Kim, J. Ohm, M. Pickering, and A. Yamada, eds. Boston: MPEG-7, 2001.
[34] Description of MPEG-7 Content Set, ISO/IEC JTC1/SC29/WG11/N2467, Atlantic City, N.J., Oct. 1998.

Index Terms:
Video retrieval, index structure, high-dimensional data, ordered VA-File, similarity query, kNN.
Hong Lu, Beng Chin Ooi, Heng Tao Shen, Xiangyang Xue, "Hierarchical Indexing Structure for Efficient Similarity Search in Video Retrieval," IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 11, pp. 1544-1559, Nov. 2006, doi:10.1109/TKDE.2006.174
Usage of this product signifies your acceptance of the Terms of Use.