The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - April (2013 vol.62)
pp: 690-704
C. Ttofis , Dept. of Electr. & Comput. Eng., Univ. of Cyprus, Nicosia, Cyprus
S. Hadjitheophanous , Dept. of Electr. & Comput. Eng., Univ. of Cyprus, Nicosia, Cyprus
A. S. Georghiades , Dept. of Electr. & Comput. Eng., Univ. of Cyprus, Nicosia, Cyprus
T. Theocharides , Dept. of Electr. & Comput. Eng., Univ. of Cyprus, Nicosia, Cyprus
ABSTRACT
Stereo Vision, a technique aimed at inferring depth information from stereo images, has been used in a wide range of computer vision applications, with real-time requirements in emerging embedded vision systems. Computation of the disparity map, a vital step in extracting depth information from stereo images, requires a significant amount of computational resources. As such, existing software implementations require high-end hardware platforms to achieve real-time frame rates, suggesting that dedicated hardware mechanisms might be more suitable for embedded applications. In this paper, we present a disparity map computation architecture targeting embedded stereo vision applications with hard real-time requirements. The architecture integrates a hardware edge detection mechanism that reduces the search space, improving the overall performance, and is configurable in terms of various application parameters, making it suitable for a number of application environments. The paper also presents a study on the impact of the various parameters in terms of the performance and hardware/power overheads. An experimental prototype of the architecture was implemented on the Xilinx ML505 FPGA Evaluation Platform, achieving 50 Frames Per Second (fps) for 1,280 × 1,024 image sizes. Moreover, the quality of the disparity maps generated by the proposed system is comparable to other existing hardware implementations featuring local stereo correspondence methods.
INDEX TERMS
stereo image processing, computer vision, edge detection, feature extraction, field programmable gate arrays, local stereo correspondence method, edge-directed hardware architecture, realtime disparity map computation, stereo image, depth information extraction, embedded vision system, embedded stereo vision application, hardware edge detection mechanism, application parameter, hardware-power overhead, performance overhead, ML505 FPGA evaluation platform, field programmable gate array, image size, Image edge detection, Hardware, Computer architecture, Stereo vision, Real time systems, Correlation, Detectors, image processing, Stereo vision systems, real-time disparity map computation, FPGA design
CITATION
C. Ttofis, S. Hadjitheophanous, A. S. Georghiades, T. Theocharides, "Edge-Directed Hardware Architecture for Real-Time Disparity Map Computation", IEEE Transactions on Computers, vol.62, no. 4, pp. 690-704, April 2013, doi:10.1109/TC.2012.32
REFERENCES
[1] E. Trucco and A. Verri, Introductory Techniques For 3-D Computer Vision. Prentice Hall PTR, 1998.
[2] B. Cyganek and J.P. Siebert, Introduction to 3D Computer Vision Techniques and Algorithms. Wiley, John & Sons, 2009.
[3] K. Mühlmann, D. Maier, J. Hesser, and R. Männer, "Calculating Dense Disparity Maps from Color Stereo Images, an Efficient Implementation," Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 79-88, Apr.-June 2002.
[4] S. Forstmann, Y. Kanou, J. Ohya, S. Thuering, and A. Schmitt, "Real-Time Stereo by Using Dynamic Programming," Proc. Computer Vision and Pattern Recognition Workshop (CVPRW `04), pp. 29-29, June 2004.
[5] L. Di Stefano, M. Marchionni, and S. Mattoccia, "A Fast Area-Based Stereo Matching Algorithm," Proc. 15th Int'l Conf. Vision Interface, vol. 22, no. 12, pp. 983-1005, Oct. 2004.
[6] H. Hirschmüller, P.R. Innocent, and J. Garibaldi, "Real-Time Correlation-Based Stereo Vision with Reduced Border Errors," Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 229-246, Apr.-June 2002.
[7] R. Yang and M. Pollefeys, "A Versatile Stereo Implementation on Commodity Graphics Hardware," Real-Time Imaging, vol. 11, no. 1, pp. 7-18, Feb. 2005.
[8] H. Hile and C. Zheng, Stereo Video Process, for Depth Map, technical report, Univ. of Washington, 2004.
[9] Y. Miyajima and T. Maruyama, "A Real-Time Stereo Vision System with FPGA," Proc. 13th Int'l Conf. Field-Programmable Logic and Applications (FPL '03), pp. 448-457, 2003.
[10] L. Nalpantidis, G. Sirakoulis, and A. Gasteratos, "Review of Stereo Matching Algorithms for 3D Vision," Proc. 16th Int'l Symp. Measurement and Control in Robotics (ISMCR '07), pp. 116-124, June 2007.
[11] M. Arias-Estrada and J.M. Xicotencatl, "Multiple Stereo Matching Using an Extended Architecture," Field-Programable Logic and Applications, vol. 2778, pp. 203-212, Springer, 2003.
[12] S.H. Lee, J. Yi, and J. Kim, "Real-Time Stereo Vision on a Reconfigurable System," Embedded Computer Systems: Architectures, Modelling and Simulation, vol. 3553, pp. 299-307, Springer, 2005.
[13] M. Hariyama, Y. Kobayashi, H. Sasaki, and M. Kameyama, "FPGA Implementation of a Stereo Matching Processor Based on Window-Parallel-and-Pixel-Parallel Architecture," Proc. 48th Midwest Symp. Circuits and Systems, vol. 2, pp. 1219-1222, 2005.
[14] H. Kim and K. Sohn, "3D Reconstruction of Stereo Images for Interaction between Real and Virtual Worlds," Proc. IEEE/ACM Second Int'l Symp. Mixed and Augmented Reality, pp. 169-176, Oct. 2003.
[15] O. Faugeras, Three Dimensional Computer Vision. MIT Press, 1993.
[16] T.A. Abbasi and M.U. Abbasi, "A Proposed FPGA Based Architecture for Sobel Edge Detection Operator," J. Active and Passive Electronic Devices, vol. 2, pp. 271-277, 2007.
[17] D.V. Papadimitriou and T.J. Dennis, "Epipolar Line Estimation and Rectification for Stereo Image Pairs," IEEE Trans. Image Processing, vol. 5, no. 4, pp. 672-676, Apr. 1996.
[18] E. Staudinger, M. Humenberger, and W. Kubinger, "FPGA-Based Rectification and Lens Undistortion for a Real-Time Embedded Stereo Vision Sensor," Proc. FH Science Day, pp. 18-25, 2008.
[19] R.-P M. Berretty, A.K. Riemens, and P.F. Machado, "Real-Time Embedded System for Stereo Video Processing for Multiview Displays," Proc. SPIE, vol. 6490, p. 649014, Jan. 2007.
[20] Middlebury College: Stereo Vision Research Page, http://vision. middlebury.edustereo/, Oct. 2010.
[21] J. Ralli, F. Pelayo, and J. Diaz, "Increasing Efficiency in Disparity Calculation," Proc. Second Int'l Conf. Advances in Brain, Vision, and Artificial Intelligence, vol. 4729, pp. 298-307, 2007.
[22] S. Cavanag and M. Manzke, "Real Time Disparity Map Estimation on the Cell Processor," Proc. Eurographics Ireland Workshop, pp 67-74, Dec. 2009.
[23] C. Georgoulas and I. Andreadis, "A Real-Time Occlusion Aware Hardware Structure for Disparity Map Computation," Proc. 15th Int'l Conf. Image Analysis and Processing (ICIAP '09), vol. 5716/2009, pp. 721-730, 2009.
[24] Z. Vasicek and L. Sekanina, "An Evolvable Hardware System in Xilinx Virtex II Pro FPGA," Int'l J. Innovative Computing and Applications, vol. 1, no. 1, pp. 63-73, 2007.
[25] M. Raman and H. Aggarwal, "Study and Comparison of Various Image Edge Detection Techniques," Int'l J. Image Processing, vol. 3, no. 1, pp. 1-12, 2009.
[26] J. Liu, Y. Xu, R. Klette, H. Chen, and T. Vaudrey, "Disparity Map Computation on a Cell Processor," Proc. Int'l Conf. Modelling, Simulation and Identification (MSI '09), Oct. 2009.
[27] Virtex-5 LXT FPGA ML505 Evaluation Platform, http://www. xilinx.com/products/devkitsHW-V5-ML505-UNI-G.htm , May 2010.
[28] MicroBlaze Soft Processor Core, http://www.xilinx.com/toolsmicroblaze.htm , May 2010.
[29] S. Hadjitheophanous, C. Ttofis, A.S. Georghiades, and T. Theocharides, "Towards Hardware Stereoscopic 3D Reconstruction: A Real-Time FPGA Computation of the Disparity Map," Proc. Design, Automation and Test in Europe Conf. and Exhibition (DATE '10), 2010.
[30] T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, "Differences between Stereo and Motion Behaviour on Synthetic and Real-World Stereo Sequences," Proc. 23rd Int'l Conf. Image and Vision Computing (IVCNZ '08), pp.1-6, Nov. 2008.
[31] K. Ambrosch, M. Humenberger, W. Kubinger, and A. Steininger, "A SAD-Based Stereo Matching Using FPGAs," Embedded Computer Vision Part II, pp. 121-138, 2009.
[32] D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol. 47, pp. 7-42, 2002.
[33] Q. Yang, L. Wang, R. Yang, S. Wang, M. Liao, and D. Nister, "Real-Time Global Stereo Matching Using Hierarchical Belief Propagation," Proc. The British Machine Vision Conf., 2006.
[34] M.Z. Brown, D. Burschka, G.D. Hager, and S. Member, "Advances in Computational Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 993-1008, Aug. 2003.
[35] J. Diaz, E. Ros, R. Carrillo, and A. Prieto, "Real-Time System for High-Image Resolution Disparity Estimation," IEEE Trans. Image Processing, vol. 16, no. 1, pp. 280-285, Jan. 2007.
[36] A. Darabiha, J. MacLean, and J. Rose, "Reconfigurable Hardware Implementation of a Phase-Correlation Stereo Algorithm," Machine Vision Applications, vol. 17, no. 2, pp. 116-132, 2006.
[37] S. Jin, J. Cho, X.D. Pham, K.M. Lee, S-K. Park, M. Kim, and J.W. Jeon, "FPGA Design and Implementation of a Real-Time Stereo Vision System," IEEE Trans. Circuits and Systems for Video Technology (CSVT), vol. 20, no. 1, pp. 15-26, Jan. 2010.
[38] N.Y.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang, "Algorithm and Architecture of Disparity Estimation with Mini-Census Adaptive Support Weight," IEEE Trans. Circuits and Systems for Video Technology (CSVT), vol. 20, no.6, pp.792-805, June 2010.
[39] enpeda., Image Sequence Analysis Test Site, http://www. mi.auckland.ac.nzEISATS/, 2012.
[40] B. Khaleghi, S. Ahuja, and Q. Wu, "An Improved Real-Time Miniaturized Embedded Stereo Vision System (MESVS-II)," IEEE CS Conf. Computer Vision and Pattern Recognition Workshops (CVPRW '08), pp. 1-8, June 2008.
[41] C. Zinner and M. Humenberger, "Distributed Real-Time Stereo Matching on Smart Cameras," Proc. ACM/IEEE Fourth Int'l Conf. Distributed Smart Cameras (ICDSC '10), 2010.
[42] K. Ambrosch and W. Kubinger, "Accurate Hardware-Based Stereo Vision," Computer Vision and Image Understanding, vol. 114, pp. 1303-1316, 2010.
[43] C. Kyrkou, C. Ttofis, and T. Theocharides, "Depth-Directed Hardware Object Detection," Proc. Design, Automation and Test in Europe Conf. and Exhibition (DATE '11), 2011.
22 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool