The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2010 vol.32)
pp: 546-560
Tai-Pang Wu , HKUST, Hong Kong
ABSTRACT
This paper presents a robust and automatic approach to photometric stereo, where the two main components, namely surface normals and visible surfaces, are respectively optimized by Expectation Maximization (EM). A dense set of input images is conveniently captured using a digital video camera while a handheld spotlight is being moved around the target object and a small mirror sphere. In our approach, the inherently complex optimization problem is simplified into a two-step optimization, where EM is employed in each step: 1) Using the dense input, the weight or importance of each observation is alternately optimized with the normal and albedo at each pixel and 2) using the optimized normals and employing the Markov Random Fields (MRFs), surface integrabilities and discontinuities are alternately optimized in visible surface reconstruction. Our mathematical derivation gives simple updating rules for the EM algorithms, leading to a stable, practical, and parameter-free implementation that is very robust even in the presence of complex geometry, shadows, highlight, and transparency. We present high-quality results on normal and visible surface reconstruction, where fine geometric details are automatically recovered by our method.
INDEX TERMS
Photometric stereo, expectation maximization, normal, albedo and visible surface reconstruction.
CITATION
Tai-Pang Wu, "Photometric Stereo via Expectation Maximization", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 3, pp. 546-560, March 2010, doi:10.1109/TPAMI.2009.15
REFERENCES
[1] A. Agrawal, R. Chellappa, and R. Raskar, “An Algebraic Approach to Surface Reconstruction from Gradient Fields,” Proc. Int'l Conf. Computer Vision, vol. 1, pp. 174-181, 2005.
[2] A. Agrawal, R. Raskar, and R. Chellappa, “What Is the Range of Surface Reconstructions from a Gradient Field?” Proc. European Conf. Computer Vision, pp. 578-591, 2006.
[3] S. Barsky and M. Petrou, “The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1239-1252, Oct. 2003.
[4] R. Basri and D.W. Jacobs, “Photometric Stereo with General, Unknown Lighting,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 374-381, 2001.
[5] J. Bilmes, “A Gentle Tutorial on the EM Algorithm and Its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models,” Technical Report ICSI-TR-97-021, ICSI, 1997.
[6] H.S. Chung and J. Jia, “Efficient Photometric Stereo on Glossy Surfaces with Wide Specular Lobes,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[7] E.N. Coleman,Jr. and R. Jain, “Obtaining 3-Dimensional Shape of Textured and Specular Surfaces Using Four-Source Photometry,” Computer Graphics and Image Processing, vol. 18, no. 4, pp. 309-328, Apr. 1982.
[8] D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Prentice-Hall, 2003.
[9] R.T. Frankot and R. Chellappa, “A Method for Enforcing Integrability in Shape from Shading Algorithms,” Proc. Int'l Conf. Computer Vision, pp. 118-128, 1987.
[10] A.S. Georghiades, “Incorporating the Torrance and Sparrow Model of Reflectance in Uncalibrated Photometric Stereo,” Proc. Int'l Conf. Computer Vision, pp. 816-823, 2003.
[11] D.B. Goldman, B. Curless, A. Hertzmann, and S.M. Seitz, “Shape and Spatially-Varying Brdfs from Photometric Stereo,” Proc. Int'l Conf. Computer Vision, Oct. 2005.
[12] A. Hertzmann and S.M. Seitz, “Shape Materials by Example: A Photometric Stereo Approach,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 533-540, 2003.
[13] B.K.P. Horn, Robot Vision. McGraw-Hill, 1986.
[14] B.K.P. Horn, R.J. Woodham, and W.M. Silver, “Determining Shape and Reflectance Using Multiple Images,” MIT AI Memo, 1978.
[15] J.W. Hsieh, H.Y.M. Liao, M.T. Ko, and K.C. Fan, “Wavelet-Based Shape from Shading,” Graphical Models and Image Processing, vol. 57, no. 4, pp. 343-362, July 1995.
[16] B. Karacali and W.E. Snyder, “Partial Integrability in Surface Reconstruction from a Given Gradient Field,” Proc. Int'l Conf. Image Processing, vol. II, pp. 525-528, 2002.
[17] B. Karacali and W.E. Snyder, “Reconstructing Discontinuous Surfaces from a Given Gradient Field Using Partial Integrability,” Computer Vision and Image Understanding, vol. 92, no. 1, pp.78-111, Oct. 2003.
[18] G. Kay and T. Caelly, “Estimating the Parameters of an Illumination Model Using Photometric Stereo,” Graphical Models and Image Processing, vol. 57, no. 5, pp. 365-388, 1995.
[19] P. Kovesi, “Shapelets Correlated with Surface Normals Produce Surfaces,” Proc. Int'l Conf. Computer Vision, pp. 994-1001, 2005.
[20] S.K. Nayar, K. Ikeuchi, and T. Kanade, “Determining Shape and Reflectance of Hybrid Surfaces by Photometric Sampling,” IEEE Trans. Robotics and Automation, vol. 6, no. 4, pp. 418-431, Aug. 1990.
[21] N. Petrovic, I. Cohen, B.J. Frey, R. Koetter, and T.S. Huang, “Enforcing Integrability for Surface Reconstruction Algorithms Using Belief Propagation in Graphical Models,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp.743-748, 2001.
[22] T. Simchony, R. Chellappa, and M. Shao, “Direct Analytical Methods for Solving Poisson Equations in Computer Vision Problems,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 5, pp. 435-446, May 1990.
[23] F. Solomon and K. Ikeuchi, “Extracting the Shape and Roughness of Specular Lobe Objects Using Four Light Photometric Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 4, pp. 449-454, Apr. 1996.
[24] H.D. Tagare and R.J.P. deFigueiredo, “A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 2, pp.133-152, Feb. 1991.
[25] K.L. Tang, C.K. Tang, and T.T. Wong, “Dense Photometric Stereo Using Tensorial Belief Propagation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 132-139, June 2005.
[26] D. Terzopoulos, “The Computation of Visible-Surface Representations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp. 417-438, July 1988.
[27] F. Verbiest and L. Van Gool, “Photometric Stereo with Coherent Outlier Handling and Confidence Estimation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[28] R.J. Woodham, “Photometric Method for Determining Surface Orientation from Multiple Images,” Optical Eng., vol. 19, no. 1, pp.139-144, Jan. 1980.
[29] T.P. Wu and C.K. Tang, “Dense Photometric Stereo Using a Mirror Sphere and Graph Cut,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 140-147, June 2005.
[30] T.P. Wu and C.K. Tang, “Dense Photometric Stereo by Expectation Maximization,” Proc. European Conf. Computer Vision, vol. IV, pp.159-172, 2006.
[31] T.P. Wu and C.K. Tang, “Visible Surface Reconstruction from Normals with Discontinuity Consideration,,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp.1793-1800, 2006.
[32] T.P. Wu, K.L. Tang, C.K. Tang, and T.T. Wong, “Dense Photometric Stereo: A Markov Random Field Approach,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp.1830-1846, Nov. 2006.
21 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool