The Community for Technology Leaders
RSS Icon
Issue No.11 - November (2011 vol.10)
pp: 1646-1660
Tse-Wei Chen , National Taiwan University, Taipei
Yi-Ling Chen , National Taiwan University, Taipei
Shao-Yi Chien , National Taiwan University, Taipei
A new photo retrieval system for mobile devices is proposed. The system can be used to search for photos with similar spatial layouts efficiently, and it adopts an image segmentation algorithm that extracts features of image regions based on K-Means clustering. Since K-Means is computationally intensive for real-time applications and prone to generate clustering results with local optima, parallel hardware architectures are designed to meet the real-time requirement of the retrieval process. Experiments show that the proposed algorithm in the photo retrieval system obtains better mean average precision than other methods, and it is tested with image recognition problems. The robustness of the algorithm is also evaluated with noise and image blurring. Besides, the proposed K-Means hardware can provide a trade-off between the execution time and the retrieval performance on the software and hardware cosimulation platform. The contribution of this work is twofold. The first is the development of a photo retrieval framework for mobile devices, where a new texture feature is employed in the algorithm to enhance the retrieval performance. The other is the integration of the K-Means hardware accelerator and the photo retrieval system. The hardware architecture is analyzed, and the specifications are compared with previous works.
Photo retrieval, image segmentation, K-Means clustering, parallel processing, hardware acceleration.
Tse-Wei Chen, Yi-Ling Chen, Shao-Yi Chien, "Photo Retrieval Based on Spatial Layout with Hardware Acceleration for Mobile Devices", IEEE Transactions on Mobile Computing, vol.10, no. 11, pp. 1646-1660, November 2011, doi:10.1109/TMC.2011.23
[1] P. Dubey, "Recognition, Mining and Synthesis Moves Computers to the Era of Tera," Technology@Intel Magazine, Feb. 2005.
[2] J. Meng, S. Chakradhar, and A. Raghunathan, "Best-Effort Parallel Execution Framework for Recognition and Mining Applications," Proc. IEEE Int'l Symp. Parallel and Distributed Processing, pp. 1-12, 2009.
[3] S. Hamilton, "Semiconductor Research Corporation: Taking Moore's Law into the Next Century," Computer, vol. 32, no. 1, pp. 43-48, Jan. 1999.
[4] R. Bez, E. Camerlenghi, A. Modelli, and A. Visconti, "Introduction to Flash Memory," Proc. IEEE, vol. 91, no. 4, pp. 489-502, Apr. 2003.
[5] Y. Rui, T.S. Huang, and S.-F. Chang, "Image Retrieval: Current Techniques, Promising Directions, and Open Issues," J. Visual Comm. and Image Representation, vol. 10, no. 1, pp. 39-62, Mar. 1999.
[6] J.R. Smith and S.-F. Chang, "VisualSEEk: A Fully Automated Content-Based Image Query System," Proc. ACM Int'l Conf. Multimedia, pp. 87-98, 1996.
[7] C. Carson, M. Thomas, S. Belongie, J.M. Hellerstein, and J. Malik, "Blobworld: A System for Region-Based Image Indexing and Retrieval," Proc. Third Int'l Conf. Visual Information Systems, pp. 509-516, 1999.
[8] J.Z. Wang, G. Wiederhold, O. Firschein, and S.X. Wei, "Content-Based Image Indexing and Searching Using Daubechies' Wavelets," Int'l J. Digital Libraries, vol. 1, no. 4, pp. 311-328, 1997.
[9] W.-Y. Ma and B.S. Manjunath, "NeTra: A Toolbox for Navigating Large Image Databases," Multimedia Systems, vol. 7, no. 3, pp. 184-198, 1999.
[10] J.Z. Wang, J. Li, and G. Wiederhold, "SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 9, pp. 947-963, Sept. 2001.
[11] J. MacQueen, "Some Methods for Classification and Analysis of Multivariate Observations," Proc. Fifth Berkeley Symp. Math. Statistics and Probability, pp. 281-297, 1967.
[12] J.-W. Hsieh, W.E.L. Grimson, C.-C. Chiang, and Y.-S. Huang, "Region-Based Image Retrieval," Proc. IEEE Int'l Conf. Image Processing, vol. 1, pp. 77-80, 2000.
[13] B. Ko, H.-S. Lee, and H. Byun, "Region-Based Image Retrieval System Using Efficient Feature Description," Proc. Int'l Conf. Pattern Recognition, vol. 4, pp. 283-286, 2000.
[14] J. Fauqueur and N. Boujemaa, "Region-Based Image Retrieval: Fast Coarse Segmentation and Fine Color Description," J. Visual Languages and Computing, vol. 15, no. 1, pp. 69-95, Feb. 2004.
[15] J.-W. Hsieh and W.E.L. Grimson, "Spatial Template Extraction for Image Retrieval by Region Matching," IEEE Trans. Image Processing, vol. 12, no. 11, pp. 1404-1415, Nov. 2003.
[16] M.R. Boutell, J. Luo, and C.M. Brown, "Scene Parsing Using Region-Based Generative Models," IEEE Trans. Multimedia, vol. 9, no. 1, pp. 136-146, Jan. 2007.
[17] L. Lucchese and S.K. Mitra, "Color Image Segmentation: A State-of-the-Art Survey," Proc. Indian Nat'l Science Academy, vol. 67-A, no. 2, pp. 207-221, Mar. 2001.
[18] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 416-423, 2001.
[19] X. Ren and J. Malik, "Learning a Classification Model for Segmentation," Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 10-17, 2003.
[20] G. Mori, "Guiding Model Search Using Segmentation," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1417-1423, 2005.
[21] J. Adams, K. Parulski, and K. Spaulding, "Color Processing in Digital Cameras," IEEE Micro, vol. 18, no. 6, pp. 20-30, Nov./Dec. 1998.
[22] A.K. Jain, M.N. Murty, and P.J. Flynn, "Data Clustering: A Review," ACM Computing Surveys, vol. 31, no. 3, pp. 264-323, 1999.
[23] M. Luo, Y.-F. Ma, and H.-J. Zhang, "A Spatial Constrained K-Means Approach to Image Segmentation," Proc. Joint Conf. Int'l Conf. Information, Comm. and Signal Processing, and Pacific Rim Conf. Multimedia, vol. 2, pp. 738-742, 2003.
[24] V. Mezaris, I. Kompatsiaris, and M.G. Strintzis, "Still Image Segmentation Tools for Object-Based Multimedia Applications," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 18, no. 4, pp. 701-725, June 2004.
[25] T. Yokoyama, S. Furukawa, and T. Watanabe, "Moving Region Detection by Transportation Problem Solving," Proc. Ninth IEEE Int'l Symp. Multimedia, pp. 86-91, 2007.
[26] T.-W. Chen, Y.-L. Chen, and S.-Y. Chien, "Fast Image Segmentation Based on K-Means Clustering with Histograms in HSV Color Space," Proc. IEEE Int'l Workshop Multimedia Signal Processing, pp. 322-325, 2008.
[27] J.M. Peña, J.A. Lozano, and P. Larrañaga, "An Empirical Comparison of Four Initialization Methods for the K-Means Algorithm," Pattern Recognition Letters, vol. 20, no. 10, pp. 1027-1040, 1999.
[28] R.M. Haralick, "Statistical and Structural Approaches to Texture," Proc. IEEE, vol. 67, no. 5, pp. 786-804, May 1979.
[29] H. Tamura, S. Mori, and T. Yamawaki, "Textural Features Corresponding to Visual Perception," IEEE Trans. Systems, Man and Cybernetics, vol. 8, no. 6, pp. 460-473, June 1978.
[30] B.S. Manjunath and W.Y. Ma, "Texture Features for Browsing and Retrieval of Image Data," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 837-842, Aug. 1996.
[31] S. Livens, P. Scheunders, G. van de Wouwer, and D. Van Dyck, "Wavelets for Texture Analysis, an Overview," Proc. Sixth Int'l Conf. Image Processing and Its Applications, vol. 2, pp. 581-585, 1997.
[32] G. van de Wouwer, P. Scheunders, and D. Van Dyck, "Statistical Texture Characterization from Discrete Wavelet Representations," IEEE Trans. Image Processing, vol. 8, no. 4, pp. 592-598, Apr. 1999.
[33] M. Kokare, P.K. Biswas, and B.N. Chatterji, "Texture Image Retrieval Using New Rotated Complex Wavelet Filters," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 35, no. 6, pp. 1168-1178, Dec. 2005.
[34] Y. Rubner, C. Tomasi, and L.J. Guibas, "The Earth Mover's Distance as a Metric for Image Retrieval," Int'l J. Computer Vision, vol. 40, no. 2, pp. 99-121, 2000.
[35] B. Ko and H. Byun, "Integrated Region-Based Image Retrieval Using Region's Spatial Relationships," Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 196-199, 2002.
[36] D.P. Huttenlocher, G.A. Klanderman, and W.J. Rucklidge, "Comparing Images Using the Hausdorff Distance," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 9, pp. 850-863, Sept. 1993.
[37] K. Fukuda and T. Matsui, "Finding All Minimum Cost Perfect Matchings in Bipartite Graphs," Networks, vol. 22, no. 5, pp. 461-468, 1992.
[38] K.R. Varadarajan and P.K. Agarwal, "Approximation Algorithms for Bipartite and Nonbipartite Matching in the Plane," Proc. 10th Ann. ACM-SIAM Symp. Discrete Algorithms, pp. 805-814, 1999.
[39] D. Nistér and H. Stewénius, "Scalable Recognition with a Vocabulary Tree," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2161-2168, 2006.
[40] K. Krishna and M. Narasimha Murty, "Genetic K-Means Algorithm," IEEE Trans. Systems, Man, and Cybernetics—Part B: Cybernetics, vol. 29, no. 3, pp. 433-439, June 1999.
[41] S. Phillips, "Reducing the Computation Time of the Isodata and K-Means Unsupervised Classification Algorithms," Proc. IEEE Int'l Geoscience and Remote Sensing Symp., vol. 3, pp. 1627-1629, 2002.
[42] M. Estlick, M. Leeser, J. Theiler, and J.J. Szymanski, "Algorithmic Transformations in the Implementation of K-Means Clustering on Reconfigurable Hardware," Proc. ACM/SIGDA Int'l Symp. Field Programmable Gate Arrays, pp. 103-110, 2001.
[43] W.-C. Liu, J.-L. Huang, and M.-S. Chen, "KACU: K-Means with Hardware Centroid-Updating," Proc. Fifth Emerging Information Technology Conf., 2005.
[44] T. Saegusa and T. Maruyama, "An FPGA Implementation of Real-Time K-Means Clustering for Color Images," J. Real-Time Image Processing, vol. 2, no. 4, pp. 309-318, Nov. 2007.
[45] A.G. da S. Filho, A.C. Frery, C.C. de Araújo, H. Alice, J. Cerqueira, J.A. Loureiro, M.E. de Lima, M. das G.S. Oliveira, and M.M. Horta, "Hyperspectral Images Clustering on Reconfigurable Hardware Using the K-Means Algorithm," Proc. Symp. Integrated Circuits and Systems Design, pp. 99-104, 2003.
[46] T.-W. Chen, C.-H. Sun, J.-Y. Bai, H.-R. Chen, and S.-Y. Chien, "Architectural Analyses of K-Means Silicon Intellectual Property for Image Segmentation," Proc. IEEE Int'l Symp. Circuits and Systems, pp. 2578-2581, 2008.
[47] T.-W. Chen and S.-Y. Chien, "Bandwidth Adaptive Hardware Architecture of K-Means Clustering for Video Analysis," IEEE Trans. Very Large Scale Integration Systems, vol. 18, no. 6, pp. 957-966, June 2010.
[48] K. Krishna, K.R. Ramakrishnan, and M.A.L. Thathachar, "Vector Quantization Using Genetic K-Means Algorithm for Image Compression," Proc. Int'l Conf. Information, Comm. and Signal Processing, vol. 3, pp. 1585-1587, 1997.
[49] B. Kövesi, J.-M. Boucher, and S. Saoudi, "Stochastic K-Means Algorithm for Vector Quantization," Pattern Recognition Letters, vol. 22, nos. 6-7, pp. 603-610, 2001.
[50] H.-S. Chiu, G.-Y. Chen, C.-J. Lee, and B. Chen, "Position Information for Language Modeling in Speech Recognition," Proc. Int'l Symp. Chinese Spoken Language Processing, pp. 1-4, 2008.
[51] T.-W. Chen, S.-C. Hsu, and S.-Y. Chien, "Robust Video Object Segmentation Based on K-Means Background Clustering and Watershed in Ill-Conditioned Surveillance Systems," Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 787-790, 2007.
[52] L. Fei-Fei, R. Fergus, and P. Perona, "Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, Workshop Generative-Model Based Vision, 2004.
[53] Y.-Q. Cheng, V. Wu, R.T. Collins, A.R. Hanson, and E.M. Riseman, "Maximum-Weight Bipartite Matching Technique and Its Application in Image Feature Matching," Proc. SPIE Visual Comm. and Image Processing, 1996.
[54] H.A.B. Saip and C.L. Lucchesi, "Matching Algorithms for Bipartite Graphs," Technical Report DCC -03/93, Departamento de Ciencia da Computação, Universidade Estadual de Campinas, 1993.
[55] A.K. Jain, Y. Zhou, T. Mustufa, E.C. Burdette, G.S. Chirikjian, and G. Fichtinger, "Matching and Reconstruction of Brachytherapy Seeds Using the Hungarian Algorithm (MARSHAL)," Medical Physics, vol. 32, no. 11, pp. 3475-3492, 2005.
[56] S. Lazebnik, C. Schmid, and J. Ponce, "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categoires," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2169-2178, 2006.
[57] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[58] S.-Y. Chien and T.-W. Chen, "Motion Adaptive Spatio-Temporal Gaussian Noise Reduction Filter for Double-Shot Images," Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 1659-1662, 2007.
[59] J. Yang, Y.-G. Jiang, A.G. Hauptmann, and C.-W. Ngo, "Evaluating Bag-of-Visual-Words Representations in Scene Classification," Proc. Int'l Workshop Multimedia Information Retrieval, pp. 197-206, 2007.
[60] J.D. Hall and J.C. Hart, "GPU Acceleration of Iterative Clustering," Proc. ACM Workshop General Purpose Computing on Graphics Processors, pp. C-6, 2004.
[61] B. Catanzaro, B.-Y. Su, N. Sundaram, Y. Lee, M. Murphy, and K. Keutzer, "Efficient, High-Quality Image Contour Detection," Proc. IEEE Int'l Conf. Computer Vision, pp. 2381-2388, 2009.
5 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool