Subscribe
Issue No.07 - July (2009 vol.31)
pp: 1310-1324
Yael Moses , The Interdisciplinary Center, Herzeliya
ABSTRACT
We present a new method for recovering the 3D shape of a featureless smooth surface from three or more calibrated images illuminated by different light sources (three of them are independent). This method is unique in its ability to handle images taken from unconstrained perspective viewpoints and unconstrained illumination directions. The correspondence between such images is hard to compute and no other known method can handle this problem locally from a small number of images. Our method combines geometric and photometric information in order to recover dense correspondence between the images and accurately computes the 3D shape. Only a single pass starting at one point and local computation are used. This is in contrast to methods that use the occluding contours recovered from many images to initialize and constrain an optimization process. The output of our method can be used to initialize such processes. In the special case of fixed viewpoint, the proposed method becomes a new perspective photometric stereo algorithm. Nevertheless, the introduction of the multiview setup, self-occlusions, and regions close to the occluding boundaries are better handled, and the method is more robust to noise than photometric stereo. Experimental results are presented for simulated and real images.
INDEX TERMS
3D shape reconstruction, featureless objects.
CITATION
Yael Moses, "3D Shape Recovery of Smooth Surfaces: Dropping the Fixed-Viewpoint Assumption", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 7, pp. 1310-1324, July 2009, doi:10.1109/TPAMI.2008.147
REFERENCES
[1] R. Basri and D. Jacobs, “Lambertian Reflectance and Linear Subspaces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[2] N. Birkbeck, D. Cobzas, P. Sturm, and M. Jägersand, “Variational Shape and Reflectance Estimation under Changing Light and Viewpoints,” Proc. Ninth European Conf. Computer Vision, vol. 1, pp. 536-549, May 2006.
[3] J.Y. Bouguet, Camera Calibration Toolbox for Matlab, http://www.vision.caltech.edu/bouguetjcalib_doc , 2008.
[4] A.M. Bruckstein, “On Shape from Shading,” Computer Vision, Graphics, and Image Processing, vol. 44, no. 2, pp. 139-154, 1988.
[5] M.S. Drew, “Direct Solution of Orientation-from-Color Problem Using a Modification of Pentland's Light-Source Direction Estimator,” Computer Vision and Image Understanding, vol. 64, no. 2, pp. 286-299, Sept. 1996.
[6] P. Dupuis and J. Oliensis, “Direct Method for Reconstructing Shape from Shading,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 453-458, 1992.
[7] J.D. Durou and F. Courteille, “Integration of a Normal Field without Boundary Condition,” Proc. Int'l Conf. Computer Vision Workshop Photometric Analysis for Computer Vision, 2007.
[8] J.D. Durou, M. Falcone, and M. Sagona, “Numerical Methods for Shape-from-Shading: A New Survey with Benchmarks,” Computer Vision and Image Understanding, vol. 109, no. 1, pp. 22-43, 2008.
[9] J. Fan and L.B. Wolff, “Surface Curvature and Shape Reconstruction from Unknown Multiple Illumination and Integrability,” Computer Vision and Image Understanding, vol. 65, no. 2, pp. 347-359, Feb. 1997.
[10] O. Faugeras and R. Keriven, “Complete Dense Stereovision Using Level Set Methods,” Proc. Fifth European Conf. Computer Vision, pp.379-393, 1998.
[11] W.T. Freeman and J.B. Tenenbaum, “Learning Bilinear Models for Two Factor Problems in Vision,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 554-560, 1997.
[12] P. Fua and Y. Leclerc, “Object-Centered Surface Reconstruction: Combining Multi-Image Stereo and Shading,” Proc. DARPA Image Understanding Workshop, pp. 1097-1120, 1993.
[13] R.I. Hartley and P. Sturm, “Triangulation,” Computer Vision and Image Understanding, vol. 68, no. 2, pp. 146-157, Nov. 1997.
[14] H. Hayakawa, “Photometric Stereo under a Light-Source with Arbitrary Motion,” J. Optical Soc. Am. A, vol. 11, no. 11, pp. 3079-3089, Nov. 1994.
[15] C. Hernández, G. Vogiatzis, and R. Cipolla, “Multiview Photometric Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 548-554, Mar. 2008.
[16] B.K.P. Horn and M. Brooks, Shape from Shading. MIT Press, 1989.
[17] I. Horowitz and N. Kiryati, “Depth from Gradient Fields and Control Points: Bias Correction in Photometric Stereo, Image and Vision Computing,” Image and Vision Computing, vol. 22, pp. 681-694, 2004.
[18] D.R. Hougen and N. Ahuja, “Estimation of the Light Source Distribution and Its Use in Integrated Shape Recovery from Stereo and Shading,” Proc. Fourth Int'l Conf. Computer Vision, pp. 148-155, 1993.
[19] K. Ikeuchi and B.K.P. Horn, “Numerical Shape from Shading and Occluding Boundaries,” Artificial Intelligence, vol. 17, pp. 141-184, 1981.
[20] H. Jin, D. Cremers, D. Wang, E. Prados, A. Yezzi, and S. Soatto, “3-D Reconstruction of Shaded Objects from Multiple Images under Unknown Illumination,” Int'l J. Computer Vision, vol. 76, no. 3, pp. 245-256, 2008.
[21] R. Kimmel and A.M. Bruckstein, “Global Shape from Shading,” Computer Vision and Image Understanding, vol. 62, no. 3, pp. 360-369, 1995.
[22] R. Kimmel and A.M. Bruckstein, “Tracking Level Sets by Level Sets: A Method for Solving the Shape from Shading Problem,” Computer Vision and Image Understanding, vol. 62, no. 2, pp. 47-58, 1995.
[23] R. Kimmel and J. Sethian, “An Optimal Time Algorithm for Shape from Shading,” Proc. Asian Conf. Computer Vision, 2000.
[24] R. Kimmel and I. Yavneh, “An Algebraic Multigrid Approach for Image Analysis,” SIAM J. Scientific Computing, vol. 24, no. 4, pp.1218-1231, 2003.
[25] R. Klette, R. Kozera, and K. Schluns, “Shape from Shading and Photometric Stereo Methods,” Technical Report CITR-TR-20, Univ. of Auckland, New Zealand, 1998.
[26] K. Kolev, M. Klodt, T. Brox, S. Esedoglu, and D. Cremers, “Continuous Global Optimization in Multiview 3D Reconstruction,” Proc. Sixth Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2007.
[27] L.L. Kontsevich, A.P. Petrov, and I.S. Vergelskaya, “Reconstruction of Shape from Shading in Color Images,” J. Optical Soc. Am., vol. 11, no. 3, pp. 1047-1052, Mar. 1994.
[28] J. Lim, J. Ho, M. Yang, and D. Kriegman, “Passive Photometric Stereo from Motion,” Proc. 10th IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1635-1642, 2005.
[29] A. Maki, M. Watanabe, and C. Wiles, “Geotensity: Computing Motion and Lighting for 3D Surface Reconstruction,” Int'l J. Computer Vision, vol. 48, no. 2, pp. 75-90, 2002.
[30] Y. Moses and I. Shimshoni, “3D Shape Recovery of Smooth Surfaces: Dropping the Fixed Viewpoint Assumption,” Proc. Seventh Asian Conf. Computer Vision, pp. 429-438, 2006.
[31] R. Onn and A.M. Bruckstein, “Integrability Disambiguates Surface Recovery in Two-Image Photometric Stereo,” Int'l J. Computer Vision, vol. 5, no. 1, pp. 105-113, Aug. 1990.
[32] E. Prados and O.D. Faugeras, “Perspective Shape from Shading and Viscosity Solutions,” Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 826-831, 2003.
[33] E. Prados and S. Soatto, “Fast Marching Method for Generic Shape from Shading,” Proc. Third Int'l Workshop Variational, Geometric, and Level Set Methods in Computer Vision, pp. 320-331, 2005.
[34] D. Samaras and D. Metaxas, “Incorporating Illumination Constraints in Deformable Models for Shape from Shading and Light Direction Estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 247-264, Feb. 2003.
[35] I. Shimshoni, Y. Moses, and M. Lindenbaum, “Shape Reconstruction of 3D Bilaterally Symmetric Surfaces,” Int'l J. Computer Vision, vol. 39, no. 2, pp. 97-110, Sept. 2000.
[36] D. Simakov, D. Frolova, and R. Basri, “Dense Shape Reconstruction of a Moving Object under Arbitrary, Unknown Lighting,” Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 1202-1207, 2003.
[37] A. Tankus and N. Kiryati, “Photometric Stereo under Perspective Projection,” Proc. 10th IEEE Int'l Conf. Computer Vision, pp. 611-616, 2005.
[38] A. Tankus, N.A. Sochen, and Y. Yeshurun, “A New Perspective [on] Shape-from-Shading,” Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 862-869, 2003.
[39] M. Weber, A. Blake, and R. Cipolla, “Towards a Complete Dense Geometric and Photometric Reconstruction under Varying Pose and Illumination,” Proc. 13th British Machine Vision Conf., pp. 83-92, 2002.
[40] R.J. Woodham, “Photometric Stereo: A Reflectance Map Technique for Determining Surface Orientation from Image Intensity,” Proc. SPIE, vol. 155, pp. 136-143, 1978.
[41] T. Yu, N. Xu, and N. Ahuja, “Shape and View Independent Reflectance Map from Multiple Views,” Int'l J. Computer Vision, vol. 73, no. 2, pp. 123-138, 2007.
[42] A.L. Yuille, D. Snow, R. Epstein, and P.N. Belhumeur, “Determining Generative Models of Objects under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability,” Int'l J. Computer Vision, vol. 35, no. 3, pp. 203-222, 1999.
[43] L. Zhang, B. Curless, A. Hertzmann, and S.M. Seitz, “Shape and Motion under Varying Illumination: Unifying Structure from Motion, Photometric Stereo, and Multi-View Stereo,” Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 618-625, 2003.
[44] R. Zhang, P.S. Tsai, J.E. Cryer, and M. Shah, “Shape from Shading: A Survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 690-706, Aug. 1999.
[45] Q. Zheng and R. Chellappa, “Estimation of Illuminant Direction, Albedo, and Shape from Shading,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 680-702, July 1991.