The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - January (2008 vol.30)
pp: 147-159
ABSTRACT
Traditional stereo matching algorithms are limited in their ability to produce accurate results near depth discontinuities, due to partial occlusions and violation of smoothness constraints. In this paper, we use small baseline multi-flash illumination to produce a rich set of feature maps that enable acquisition of discontinuity preserving point correspondences. First, from a single multi-flash camera, we formulate a qualitative depth map using a gradient domain method that encodes object relative distances. Then, in a multiview setup, we exploit shadows created by light sources to compute an occlusion map. Finally, we demonstrate the usefulness of these feature maps by incorporating them into two different dense stereo correspondence algorithms, the first based on local search and the second based on belief propagation. Experimental results show that our enhanced stereo algorithms are able to extract high quality, discontinuity preserving correspondence maps from scenes that are extremely challenging for conventional stereo methods. We also demonstrate that small baseline illumination can be useful to handle specular reflections in stereo imagery. Different from most existing active illumination techniques, our method is simple, inexpensive, compact, and requires no calibration of light sources.
INDEX TERMS
stereo matching, multi-flash imaging, depth discontinuities
CITATION
Rogerio Feris, Ramesh Raskar, Longbin Chen, Karhan Tan, Matthew Turk, "Multiflash Stereopsis: Depth-Edge-Preserving Stereo with Small Baseline Illumination", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 1, pp. 147-159, January 2008, doi:10.1109/TPAMI.2007.1136
REFERENCES
[1] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int'l J. Computer Vision, vol. 47, no. 1, pp. 7-42, 2002.
[2] M. Bell and W. Freeman, “Learning Local Evidence for Shading and Reflectance,” Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 1, pp. 670-677, 2001.
[3] L. Zhang, B. Curless, and S. Seitz, “Rapid Shape Acquisition Using Color Structured Light and Multi-Pass Dynamic Programming,” Proc. Int'l Symp. 3D Data Processing Visualization and Transmission, pp. 24-26, 2002.
[4] J. Davis, D. Nehab, R. Ramamoothi, and S. Rusinkiewicz, “Spacetime Stereo: A Unifying Framework for Depth from Triangulation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 2, Feb. 2005.
[5] T. Zickler, P. Belhumeur, and D. Kriegman, “Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction,” Proc. Seventh European Conf. Computer Vision, 2002.
[6] A. Hertzmann and S. Seitz, “Shape and Materials by Example: A Photometric Stereo Approach,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 533-540, 2003.
[7] M. Daum and G. Dudek, “On 3-D Surface Reconstruction Using Shape from Shadows,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 461-468, June 1998.
[8] R. Raskar, K. Tan, R. Feris, J. Yu, and M. Turk, “A Non-Photorealistic Camera: Depth Edge Detection and Stylized Rendering Using Multi-Flash Imaging,” Proc. 31st Int'l Conf. Computer Graphics and Interactive Techniques (SIGGRAPH '04)/ACM Trans. Graphics, 2004.
[9] K. Tan, J. Kobler, P. Dietz, R. Feris, and R. Raskar, “Shape-Enhanced Surgical Visualizations and Medical Illustrations with Multi-Flash Imaging,” Proc. Seventh Int'l Conf. Medical Imaging Computing and Computer Assisted Intervention, 2004.
[10] R. Feris, R. Raskar, K. Tan, and M. Turk, “Specular Reflection Reduction with Multi-Flash Imaging,” Proc. IEEE Brazilian Symp. Computer Graphics and Image Processing, 2004.
[11] R. Feris, M. Turk, R. Raskar, K. Tan, and G. Ohashi, “Exploiting Depth Discontinuities for Vision-Based Fingerspelling Recognition,” Proc. IEEE Workshop Real-Time Vision for Human-Computer Interaction (in conjunction with the IEEE CS Conf. Computer Vision and Pattern Recognition), 2004.
[12] T. Kanade and M. Okutomi, “A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 9, pp. 920-932, Sept. 1994.
[13] S. Kang, R. Szeliski, and J. Chai, “Handling Occlusions in Dense Multi-View Stereo,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 102-110, 2001.
[14] P. Belhumeur and D. Mumford, “A Bayesian Treatment of the Stereo Correspondence Problem Using Half-Occluded Regions,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp.506-512, 1992.
[15] S. Intille and A. Bobick, “Disparity-Space Images and Large Occlusion Stereo,” Proc. Third European Conf. Computer Vision, pp.179-186, 1994.
[16] S. Birchfield and C. Tomasi, “Depth Discontinuities by Pixel-to-Pixel Stereo,” Int'l J. Computer Vision, vol. 35, no. 3, pp. 269-293, 1999.
[17] M. Tappen and W. Freeman, “Comparison of Graph Cuts with Belief Propagation for Stereo, Using Identical MRF Parameters,” Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[18] J. Sun, N. Zheng, and H. Shum, “Stereo Matching Using Belief Propagation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 787-800, July 2003.
[19] V. Kolmogorov and R. Zabih, “Computing Visual Correspondence with Occlusions Using Graph Cuts,” Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[20] J. Sun, S. Kang, and H. Shum, “Symmetric Stereo Matching for Occlusion Handling,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2005.
[21] J. Salvi, J. Pages, and J. Batlle, “Pattern Codification Strategies in Structured Light Systems,” Pattern Recognition, vol. 37, no. 4, pp.827-849, 2004.
[22] D. Scharstein and R. Szeliski, “High-Accuracy Stereo Depth Maps Using Structured Light,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 195-202, 2003.
[23] J. Posdamer and M. Altschuler, “Surface Measurement by Space-Encoded Projected Beam Systems,” Computer Graphics and Image Processing, vol. 18, no. 1, pp. 1-17, 1982.
[24] E. Horn and N. Kiryati, “Toward Optimal Structured Light Patterns,” Image and Vision Computing, vol. 17, no. 2, pp. 87-97, 1999.
[25] J. Tajima and M. Iwakawa, “3D Data Acquisition by Rainbow Range Finder,” Proc. Int'l Conf. Pattern Recognition, pp. 309-313, 1990.
[26] L. Zhang, N. Snavely, B. Curless, and S. Seitz, “Spacetime Faces: High-Resolution Capture for Modeling and Animation,” Proc. 31st Int'l Conf. Computer Graphics and Interactive Techniques (SIGGRAPH '04)/ACM Trans. Graphics, 2004.
[27] R. Woodham, “Photometric Method for Determining Surface Orientation from Multiple Images,” Optical Eng., vol. 19, no. 1, pp.139-144, 1980.
[28] S. Savarese, H. Rushmeier, F. Bernardini, and P. Perona, “Shadow Carving,” Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[29] J. Bouguet and P. Perona, “3D Photography on Your Desk,” Proc. Sixth IEEE Int'l Conf. Computer Vision, 1998.
[30] D. Yang, “Shape from Darkness under Error,” PhD dissertation, Columbia Univ., 1996.
[31] D. Kriegman and P. Belhumeur, “What Shadows Reveal about Object Structure,” J. Optical Soc. Am., pp. 1804-1813, 2001.
[32] C. Christoudias, B. Georgescu, and P. Meer, “Synergism in Low Level Vision,” Proc. Int'l Conf. Pattern Recognition, 2002.
[33] R. Fattal, D. Lischinski, and M. Werman, “Gradient Domain High Dynamic Range Compression,” Proc. 29th Int'l Conf. Computer Graphics and Interactive Techniques, 2002.
[34] G. Egnal and R. Wildes, “Detecting Binocular Half-Occlusions: Empirical Comparisons of Five Approaches,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1127-1133, Aug. 2002.
[35] M. Agrawal and L. Davis, “Window-Based, Discontinuity Preserving Stereo,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2004.
[36] H. Ishikawa and D. Geiger, “Occlusions, Discontinuities, and Epipolar Lines in Stereo,” Proc. Fifth European Conf. Computer Vision, June 1998.
[37] A. Agrawal, R. Raskar, S. Nayar, and Y. Li, “Removing Photography Artifacts Using Gradient Projection and Flash-Exposure Sampling,” Proc. 32nd Int'l Conf. Computer Graphics and Interactive Techniques (SIGGRAPH '05)/ACM Trans. Graphics, 2005.
[38] P. Felzenszwalb and D. Huttenlocher, “Efficient Belief Propagation for Early Vision,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2004.
[39] W. Matusik, C. Buehler, R. Raskar, S. Gortler, and L. McMillan, “Image-Based Visual Hulls,” Proc. 27th Int'l Conf. Computer Graphics and Interactive Techniques (SIGGRAPH '00), pp. 369-374, 2000.
[40] S. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[41] R. Cipolla and P. Giblin, Visual Motion of Curves and Surfaces. Cambridge Univ. Press, 2000.
[42] D. Crispell, D. Lanman, P. Sibley, Y. Zhao, and G. Taubin, “Beyond Silhouettes: Surface Reconstruction Using Multi-Flash Photography,” Proc. Int'l Symp. 3D Data Processing, Visualization and Transmission, 2006.
[43] I. Sato, Y. Sato, and K. Ikeuchi, “Stability Issues in Recovering Illumination Distribution from Brightness in Shadows,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 400-407, 2001.
[44] R. Feris, M. Turk, and R. Raskar, “Dealing with Multi-Scale Depth Changes and Motion in Depth Edge Detection,” Proc. IEEE Brazilian Symp. Computer Graphics and Image Processing (SIBGRAPI '06), 2006.
5 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool