This Article 
 Bibliographic References 
 Add to: 
Robust Active Stereo Vision Using Kullback-Leibler Divergence
March 2012 (vol. 34 no. 3)
pp. 548-563
Yongchang Wang, Univ. of Kentucky, Lexington, KY, USA
Kai Liu, Univ. of Kentucky, Lexington, KY, USA
Qi Hao, Univ. of Alabama, Tuscaloosa, AL, USA
Xianwang Wang, Hewlett-Packard (HP) Labs., Cupertino, CA, USA
D. L. Lau, Univ. of Kentucky, Lexington, KY, USA
L. G. Hassebrook, Univ. of Kentucky, Lexington, KY, USA
Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.

[1] J. Salvi, J. Pages, and J. Batlle, “Pattern Codification Strategies in Structured Light System,” Pattern Recognition, vol. 37, no. 4, pp. 827-849, 2004.
[2] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithm,” Int'l J. Computer Vision, vol. 47, pp. 7-12, Apr.-June 2002.
[3] A. Dipanda and S. Woo, “Efficient Correspondence Problem-Solving in 3D Shape Reconstruction Using a Structured Light System,” Optical Eng., vol. 44, p. 093602, 2005.
[4] A. Dipanda, S. Woo, F. Marzani, and J.M. Bilbault, “3D Shape Reconstruction in an Active Stereo Vision System Using Genetic Algorithms,” Pattern Recognition, vol. 36, pp. 2143-2159, 2003.
[5] D. Scharstein and R. Szeliski, “High-Accuracy Stereo Depth Maps Using Structured Light,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 195-202, July 2003.
[6] J. Pages, J. Salvi, R. Garcia, and C. Matabosch, “Overview of Coded Light Projection Techniques for Automatic 3D Profiling,” Proc. IEEE Conf. Robotics and Automation, pp. 133-138, 2003.
[7] S. Zhang, D. Royer, and S.T. Yau, “High-Resolution, Real-Time-Geometry Video Acquisition System,” Proc. ACM Siggraph, no. 14, 2006.
[8] C. Guan, L.G. Hassebrook, and D.L. Lau, “Composite Structured Light Pattern for Three-Dimensional Video,” Optics Express, vol. 11, no. 5, pp. 406-417, 2003.
[9] X. Zhou and X. Su, “Effect of the Modulation Transfer Function of a Digital Image-Acquisition Device on Phase-Measuring Profilometry,” Applied Optics, vol. 33, pp. 8210-8215, 1994.
[10] T.P. Koninckx and L. van Gool, “Real-Time Range Acquisition by Adaptive Structured Light,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 432-445, Mar. 2006.
[11] J. Guehring, “Dense 3D Surface Acquisition by Structured Light Using Off-the-Shelf Components,” Proc. SPIE, vol. 4309, pp. 220-231, 2001.
[12] K. Liu, Y. Wang, D.L. Lau, Q. Hao, and L.G. Hassebrook, “Gamma Model and Its Analysis for Phase Measuring Profilometry,” J. Optical Soc. Am. A, vol. 27, pp. 553-562, 2010.
[13] S.G. Narasimhan, S.J. Koppal, and S. Yamazaki, “Temporal Dithering of Illumination for Fast Active Vision,” Proc. 10th European Conf. Computer Vision, pp. 830-844, 2008.
[14] J. Li, L.G. Hassebrook, and C. Guan, “Optimized Two-Frequency Phase-Measuring-Profilometry Light-Senso Temporal-Noise Sensitivity,” J. Optical Soc. Am. A, vol. 20, no. 1, pp. 106-115, 2003.
[15] H.K. George, L.T. Watkins, and R.J. Benjamin, “Method and Device for Reduction in Noise in Images from Shiny Parts,” US Patent 6678057, Jan. 2004.
[16] T. Chen, H.P. Seidel, and H. Lensch, “Modulated Phase-Shifting for 3D Scanning,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[17] L. Zhang, B. Curless, and S.M. Seitz, “Rapid Shape Acquisition Using Color Structured Light and Multi-Pass Dynamic Programming,” Proc. First Int'l Symp. 3D Data Processing Visualization and Transmission, pp. 24-36, 2002.
[18] S.Y. Chen, Y.F. Li, and J. Zhang, “Vision Processing for Realtime 3D Data Acquisition Based on Coded Structured Light,” IEEE Trans. Image Processing, vol. 17, no. 2, pp. 167-176, Feb. 2008.
[19] R. Bergevin, M. Soucy, H. Gagnon, and D. Laurendeau, “Towards a General Multi-View Registration Technique,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 5, pp. 540-547, May 1996.
[20] K. Pulli, “Multiview Registration for Large Data Sets,” Proc. Int'l Conf. 3D Digital Imaging and Modeling, pp. 160-168, 1999.
[21] J. Davis, D. Nehab, R. Ramamoorthi, and S. Rusinkiewicz, “Space Stereo: A Unifying Framework for Depth from Triangulation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 2, pp. 296-302, Feb. 2005.
[22] S. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 519-526, 2006.
[23] S. Zhang and S.T. Yau, “Three-Dimensional Data Merging Using Holoimage,” Optical Eng., vol. 47, no. 3, pp. 033608-03380, 2008.
[24] R. Furukawa and H. Kawasaki, “Dense 3D Reconstruction with an Uncalibrated Stereo System Using Coded Structured Light,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, p. 107, June 2005.
[25] L. Zhang, B. Curless, and S. Seitz, “Spacetime Stereo: Shape Recovery for Dynamic Scenes,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 367-374, June 2003.
[26] Y. Wang, K. Liu, D.L. Lau, and L.G. Hassebrook, “Multicamera Phase Measuring Profilometry for Accurate Depth Measurement,” Proc. SPIE, vol. 6555, p. 655509, Apr. 2007.
[27] L. Zhang, B. Curless, and S. Seitz, “Spacetime Faces: High Resolution Capture for Modeling and Animation,” ACM Trans. Graphics, vol. 23, no. 3, pp. 548-558, 2004.
[28] S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-Time 3D Model Acquisition,” ACM Trans. Graphics, vol. 21, no. 3, pp. 438-436, 2002.
[29] T. Weise, B. Leibe, and L.V. Gool, “Fast 3D Scanning with Automatic Motion Compensation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[30] R. Ishiyama, S. Sakamoto, J. Tajima, T. Okatani, and K. Deguchi, “Absolute Phase Measurements Using Geometric Constraints between Multiple Cameras and Projectors,” Applied Optics, vol. 46, no. 17, pp. 3528-3538, 2007.
[31] H. Lee and H. Cho, “Stereo Moire Technique: A Novel 3D Measurement Method Using a Stereo Camera and a Digital Pattern Projector,” Int'l J. Optomechatronics, vol. 1, pp. 209-230, 2007.
[32] D. Hong, H. Lee, M.Y. Kim, H. Cho, and J.I. Moon, “Sensor Fusion of Phase Measuring Profilometry and Stereo Vision for Three-Dimensional Inspection of Electronic Components Assembled on Printed Circuit Boards,” Applied Optics, vol. 48, no. 21, pp. 4158-4169, 2009.
[33] M. Young, E. Beeson, and J. Davis, “Viewpoint-Coded Structured Light,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2007.
[34] B. Kamgar-Parsi and B. Kamgar-Parsi, “Evaluation of Quantization Error in Computer Vision,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 9, pp. 929-939, Sept. 1989.
[35] S. Savarese, J. Bouguet, and P. Perona, “3D Depth Recovery with Grayscale Structured Lighting,” technical report, California Inst. of Tech nology, 1999.
[36] Y.Y. Schechner, S.K. Nayar, and P.N. Belhumeur, “A Theory of Multiplexed Illumination,” Proc. Ninth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 808-815, Oct. 2003.
[37] F. Blais, “Review of 20 Years of Range Senso Development,” J. Electronic Imaging, vol. 13, no. 1, pp. 231-240, 2004.
[38] V. Kolmogorov, R. Zabih, and S. Gortler, “Multi-Camera Scene Reconstruction via Graph Cuts,” Proc. European Conf. Computer Vision, pp. 82-96, 2002.
[39] S. Kullback, Information Theory and Statistics. John Wiley and Sons, 1959.
[40] J. Sun, N.N. Zheng, and H.Y. Shum, “Stereo Matching Using Belief Propagation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 787-800, July 2003.
[41] V. Srinivasan, H. Liu, and M. Halioua, “Automated Phase-Measuring Profilometry of 3D Diffuse Objects,” Applied Optics, vol. 23, no. 18, pp. 3105-3108, 1984.
[42] H. Liu, W. Su, K. Reichard, and S. Yin, “Calibration-Based Phase-Shifting Projected Fringe Profilometry for Accurate Absolute 3D Surface Profile Measurement,” Optics Comm., vol. 216, nos. 1-3, pp. 65-80, 2003.
[43] P.J. Tavares and M.A. Vaz, “Linear Calibration Procedure for the Phase-to-Height Relationship in Phase Measurement Profilometry,” Optics Comm., vol. 274, no. 2, pp. 307-314, 2007.
[44] R. Anchini, G.D. Leo, C. Liguori, and A. Paolillo, “A New Calibration Procedure for 3D Shape Measurement System Based on Phase-Shifting Projected Fringe Profilometry,” IEEE Trans. Instrumentation and Measurement, vol. 58, no. 5, pp. 1291-1298, May 2009.
[45] K. Liu, Y. Wang, D.L. Lau, Q. Hao, and L.G. Hassebrook, “Dual-Frequency Pattern Scheme for High-Speed 3D Shape Measurement,” Optics Express, vol. 18, pp. 5229-5244, 2010.
[46] M.J. Baker, J. Xi, and J.F. Chicharo, “Elimination of Non-Linear Luminance Effects for Digital Video Projection Phase Measuring Profilometers,” Proc. IEEE Fourth Int'l Symp. Electronic Design, Test and Applications, pp. 496-501, 2008.
[47] H. Guo, H. He, and M. Chen, “Gamma Correction for Digital Fringe Projection Profilometry,” Applied Optics, vol. 43, pp. 2906-2914, 2004.
[48] Y.F. Li and S.Y. Chen, “Automatic Recalibration of an Active Structured Light Vision System,” IEEE Trans. Robotics and Automation, vol. 19, no. 2, pp. 259-268, Apr. 2003.
[49] B. Pan, L.H.Q. Kemao, and A. Asundi, “Phase Error Analysis and Compensation for Nonsinusoidal Waveforms in Phase-Shifting Digital Fringe Projection Profilometry,” Optics Letters, vol. 34, pp. 416-418, 2009.
[50] D. Dudley, W. Duncan, and J. Slaughter, “Emerging Digital Micromirror Device Applications,” Proc. SPIE, vol. 4985, pp. 14-25, 2003.
[51] N.M.G.M. Cotting, D., and H. Fuchs, “Embedding Imperceptible Patterns into Projected Images for Simultaneous Acquisition and Display,” Proc. IEEE/ACM Int'l Symp. Mixed and Augmented Reality, pp. 100-109, 2004.
[52] H. Gudbjartsson and S. Patz, “The Rician Distribution of Noisy Mri Data,” Magnetic Resonance in Medicine, vol. 34, pp. 910-914, 1995.
[53] Y. Baek, J. Kim, D. Cho, J. Lee, and W. Kim, “Integrated Noise Modeling for Image Sensor Using Bayer Domain Images,” Proc. Fourth Int'l Conf. Computer Vision/Computer Graphics Collaboration Techniques, pp. 413-424, 2009.
[54] Y. Tsin, V. Ramesh, and T. Kanade, “Statistical Calibration of Ccd Imaging Process,” Proc. Eighth IEEE Int'l Conf. Computer Vision, pp. 480-487, 2001.
[55] P. Leclercq and J. Morris, “Robustness to Noise of Stereo Matching,” Proc. 12th Int'l Conf. Image Analysis and Processing, pp. 606-611, 2003.
[56] S. Lehmann, A.P. Bradley, I.V.L. Clarkson, J. Williams, and P.J. Kootsookos, “Correspondence-Free Determination of the Affine Fundamental Matrix,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 82-97, Jan. 2007.
[57] L.G. Hassebrook and C. Guan, “Distortion,” Encyclopedia of Optics Eng., Marcel Dekker, 2003, doi 10.1081/E-EOE 120009793.

Index Terms:
Cameras,Stereo vision,Three dimensional displays,Noise,Reflection,Calibration,Frequency conversion,KL divergence.,Active stereo vision,phase matching,range data,data fusion
Yongchang Wang, Kai Liu, Qi Hao, Xianwang Wang, D. L. Lau, L. G. Hassebrook, "Robust Active Stereo Vision Using Kullback-Leibler Divergence," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 3, pp. 548-563, March 2012, doi:10.1109/TPAMI.2011.162
Usage of this product signifies your acceptance of the Terms of Use.