The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - August (2011 vol.17)
pp: 1082-1095
Yebin Liu , Tsinghua University, Beijing
Qionghai Dai , Tsinghua University, Beijing
Bennett Wilburn , Microsoft Research Asia, Beijing
ABSTRACT
We propose a method to obtain a complete and accurate 3D model from multiview images captured under a variety of unknown illuminations. Based on recent results showing that for Lambertian objects, general illumination can be approximated well using low-order spherical harmonics, we develop a robust alternating approach to recover surface normals. Surface normals are initialized using a multi-illumination multiview stereo algorithm, then refined using a robust alternating optimization method based on the \ell_{1} metric. Erroneous normal estimates are detected using a shape prior. Finally, the computed normals are used to improve the preliminary 3D model. The reconstruction system achieves watertight and robust 3D reconstruction while neither requiring manual interactions nor imposing any constraints on the illumination. Experimental results on both real world and synthetic data show that the technique can acquire accurate 3D models for Lambertian surfaces, and even tolerates small violations of the Lambertian assumption.
INDEX TERMS
Multiview stereo, photometric stereo, Lambertian reflectance, \ell_{1} minimization.
CITATION
Yebin Liu, Qionghai Dai, Bennett Wilburn, "Fusing Multiview and Photometric Stereo for 3D Reconstruction under Uncalibrated Illumination", IEEE Transactions on Visualization & Computer Graphics, vol.17, no. 8, pp. 1082-1095, August 2011, doi:10.1109/TVCG.2010.224
REFERENCES
[1] E. de Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H.-P. Seidel, and S. Thrun, “Performance Capture from Sparse Multi-View Video,” ACM Trans. Graphics, vol. 27, no. 3, pp. 98:1-98:10, 2008.
[2] D. Vlasic, I. Baran, W. Matusik, and J. Popovic, “Articulated Mesh Animation from Multi-View Silhouettes,” ACM Trans. Graphics, vol. 27, no. 1, pp. 97(1)-97(9), 2008.
[3] D. Vlasic, P. Peers, I. Baran, P. Debevec, J. Popovic, S. Rusinkiewicz, and W. Matusik, “Dynamic Shape Capture Using Multi-View Photometric Stereo,” ACM Trans. Graphics, vol. 28, no. 5, Dec. 2009.
[4] D. Bradley, T. Popa, A. Sheffer, W. Heidrich, and T. Boubekeur, “Markerless Garment Capture,” ACM Trans. Graphics, vol. 27, no. 3, pp. 99-106, 2008.
[5] T. Popa, Q. Zhou, D. Bradley, V. Kraevoy, H. Fu, A. Sheffer, and W. Heidrich, “Wrinkling Captured Garments Using Space-Time Data-Driven Deformation,” Computer Graphics Forum, vol. 28, no. 2, pp. 427-435, 2009.
[6] S.M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '06), vol. 1, pp. 519-528, 2006.
[7] J. Starck, G. Miller, and A. Hilton, “Volumetric Stereo with Silhouette and Feature Constraints,” Proc. British Machine Vision Conf. (BMVC), Sept. 2006.
[8] J. Starck and A. Hilton, “Surface Capture for Performance Based Animation,” IEEE Computer Graphics and Applications, vol. 27, no. 3, pp. 21-31, May 2007.
[9] G. Vogiatzis, C.H. Esteban, P.H.S. Torr, and R. Cipolla, “Multi-View Stereo via Volumetric Graph-Cuts and Occlusion Robust Photoconsistency,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2241-2246, Dec. 2007.
[10] J.-P. Pons, R. Keriven, and O. Faugeras, “Modelling Dynamic Scenes by Registering Multi-View Image Sequences,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '05), vol. 2, pp. 822-827, 2005.
[11] D. Bradley, T. Boubekeur, and W. Heidrich, “Accurate Multi-View Reconstruction Using Robust Binocular Stereo and Surface Meshing,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '08), 2008.
[12] Y. Liu, Q. Dai, and W. Xu, “A Point Cloud Based Multi-View Stereo Algorithm for Free-Viewpoint Video,” IEEE Trans. Visualization and Computer Graphics, vol. 16, no. 3, pp. 407-418, May/June 2010.
[13] Y. Liu, X. Cao, Q. Dai, and W. Xu, “Continuous Depth Estimation for Multi-View Stereo,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '09), 2009.
[14] Y. Furukawa and J. Ponce, “Accurate, Dense, and Robust Multiview Stereopsis,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '07), 2007.
[15] mview, http://vision.middlebury.edumview/, 2010.
[16] T. Yu, N. Xu, and N. Ahuja, “Shape and View Independent Reflectance Map from Multiple Views,” Proc. European Conf. Computer Vision (ECCV), 2004.
[17] H. Jin, D. Cremers, D. Wang, E. Prados, A. Yezzi, and S. Soatto, “3-D Reconstruction of Shaded Objects from Multiple Images under Unknown Illumination,” Int'l J. Computer Vision, vol. 76, no. 3, pp. 245-256, 2008.
[18] R. Zhang, P.S. Tsai, J.E. Cryer, and M. Shah, “Shape from Shading: A Survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 690-706, Aug. 1999.
[19] R. Basri and D.W. Jacobs, “Lambertian Reflectance and Linear Subspaces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[20] R. Ramamoorthi and P. Hanrahan, “On the Relationship between Radiance and Irradiance: Determining the Illumination from Images of Convex Lambertian Object,” J. Optical Soc. of America A, vol. 18, pp. 2448-2459, 2001.
[21] R. Basri, D. Jacobs, and I. Kemelmacher, “Photometric Stereo with General, Unknown Lighting,” Int'l J. Computer Vision, vol. 72, no. 3, pp. 239-257, 2006.
[22] C.P. Chen and C.S. Chen, “The 4-Source Photometric Stereo under General Unknown Lighting,” Proc. European Conf. Computer Vision (ECCV '06), 2006.
[23] S.K. Zhou, G. Aggarwal, R. Chellappa, and D.W. Jacobs, “Appearance Characterization of Linear Lambertian Objects, Generalized Photometric Stereo, and Illumination-Invariant Face Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 230-245, Feb. 2007.
[24] P. Belhumeur, D. Kriegman, and A. Yuille, “The Bas-Relief Ambiquity,” Int'l J. Computer Vision, vol. 35, no. 1, pp. 33-44, 1999.
[25] D. Simakov, D. Frolova, and R. Basri, “Dense Shape Reconstruction of a Moving Object under Arbitrary, Unknown Lighting,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '03), pp. 1202-1209, 2003.
[26] D. Frolova, D. Simakov, and R. Basri, “Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects under Far and Near Lighting,” Proc. European Conf. Computer Vision (ECCV '04), 2004.
[27] A. Georghiades, “Recovering 3-D Shape and Reflectance from a Small Number of Photographs,” Proc. Eurographics Symp. Rendering (EGSR '03), pp. 230-240, 2003.
[28] T. Migita, S. Ogino, and T. Shakunaga, “Direct Bundle Estimation for Recovery of Shape, Reflectance Property and Light Position,” Proc. European Conf. Computer Vision (ECCV '08), 2008.
[29] L. Zhang, B. Curless, and S.M. Seitz, “Spacetime Stereo: Shape Recovery for Dynamic Scenes,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR '03), pp. 367-374, 2003.
[30] J. Davis, R. Ramamoothi, and S. Rusinkiewicz, “Spacetime Stereo : A Unifying Framework for Depth from Triangulation,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR '03), 2003.
[31] L. Zhang, B. Curless, A. Hertzmann, and S.M. Seitz, “Shape and Motion under Varying Illumination: Unifying Stucture from Motion, Photometric Stereo, and Multi-View Stereo,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '03), vol. 1, p. 618, 2003.
[32] J. Lim, J. Ho, M.H. Yang, and D. Kriegman, “Passive Photometric Stereo from Motion,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '05), vol. 2, pp. 1635-1642, 2005.
[33] N. Joshi and D. Kriegman, “Shape from Varying Illumination and Viewpoint,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '07), vol. 2, pp. 1-7, 2007.
[34] M. Weber, A. Blake, and R. Cipolla, “Towards a Complete Dense Geometric and Photometric Reconstruction under Varying Pose and Illumination,” Proc. British Machine Vision Conf. (BMVC '02), 2002.
[35] D. Nehab, S. Rusinkiewicz, J. Davis, and R. Ramamoorthi, “Efficiently Combining Positions and Normals for Precise 3D Geometry,” Proc. ACM SIGGRAPH '05, pp. 536-543, 2005.
[36] G. Vogiatzis, C. Hernandez, and R. Cipolla, “Reconstruction in the Round Using Photometric Normals,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '06), 2006.
[37] C. Hernandez, G. Vogiatzis, and R. Cipolla, “Muti-View Photometric Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 548-554, Mar. 2008.
[38] D. Samaras and D. Metaxas, “Incorporating Illumination Constraints in Deformable Models for Shape from Shading and Light Direction Estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 247-264, Feb. 2003.
[39] N. Birkbeck, D. Cobzas, P. Sturm, and M. Jagersand, “Variational Shape and Reflectance Estimation under Changing Light and Viewpoints,” Proc. European Conf. Computer Vision (ECCV '06), 2006.
[40] C. Theobalt, N. Ahmed, H. Lensch, M. Magnor, and H.P. Seidel, “Seeing People in Different Light-Joint Shape, Motion, and Reflectance Capture,” IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 4, pp. 663-674, July/Aug. 2007.
[41] N. Ahmed, C. Theobalt, P. Dobrev, H.-P. Seidel, and S. Thrun, “Robust Fusion of Dynamic Shape and Normal Capture for High-Quality Reconstruction of Time-Varying Geometry,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '08), 2008.
[42] W.C. Ma, T. Hawkins, P. Peers, C.F. Chabert, M. Weiss, and P. Debevec, “Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination,” Proc. Eurographics Symp. Rendering (EGSR '07), 2007.
[43] H.P.A. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H.-P. Seidel, “Image-Based Reconstruction of Spatial Appearance and Geometric Detail,” ACM Trans. Graphics, vol. 22, pp. 234-257, 2003.
[44] M. Kazhdan, M. Bolitho, and H. Hoppe, “Poisson Surface Reconstruction,” Proc. Eurographics Symp. Geometry Processing (EGSG '06), pp. 61-70, 2006.
[45] P. Rousseeuw and A. Leroy, Robust Regression and Outlier Detection. John Wiley and Sons, 1987.
[46] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
[47] J. Carranza, C. Theobalt, M. Magnor, and H.-P. Seidel, “Free-Viewpoint Video of Human Actors,” ACM Trans. Graphics, vol. 22, no. 3, pp. 569-577, 2003.
[48] J. Lawrence, A. Ben-Artzi, C. DeCoro, W. Matusik, H. Pfister, R. Ramamoorthi, and S. Rusinkiewicz, “Inverse Shade Trees for Non-Parametric Material Representation and Editing,” ACM Trans. Graphics, vol. 25, no. 3, pp. 735-745, 2006.
[49] E. Candes and T. Tao, “Decoding by Linear Programming,” IEEE Trans. Information Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005.
[50] R. Ramamoorthi and P. Hanrahan, “An Efficient Representation for Irradiance Environment Maps,” Proc. ACM SIGGRAPH '01, 2001.
[51] P. Debevec, “Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography,” Proc. ACM SIGGRAPH '98, pp. 189-198, July 1998.
[52] G. Turk and M. Levoy, “Zippered Polygon Meshes from Range Images,” Proc. ACM SIGGRAPH '96, pp. 311-318, 1996.
[53] R.H. Woodham, “Photometric Method for Determining Surface Orientation from Multiple Images,” Optical Eng., vol. 19, no. 1, pp. 139-144, 1980.
[54] Y.Y. Schechner, S.K. Nayar, and P.N. Belhumeur, “A Theory of Multiplexed Illumination,” Proc. IEEE Int'l Conf. Computer Vision (ICCV), vol. 2, pp. 808-815, 2003.
25 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool