The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - Feb. (2013 vol.35)
pp: 463-475
F. Moreno-Noguer , Inst. de Robot. i Inf. Ind., UPC, Barcelona, Spain
P. Fua , Comput. Vision Lab., EPFL, Lausanne, Switzerland
ABSTRACT
Recovering the 3D shape of deformable surfaces from single images is known to be a highly ambiguous problem because many different shapes may have very similar projections. This is commonly addressed by restricting the set of possible shapes to linear combinations of deformation modes and by imposing additional geometric constraints. Unfortunately, because image measurements are noisy, such constraints do not always guarantee that the correct shape will be recovered. To overcome this limitation, we introduce a stochastic sampling approach to efficiently explore the set of solutions of an objective function based on point correspondences. This allows us to propose a small set of ambiguous candidate 3D shapes and then use additional image information to choose the best one. As a proof of concept, we use either motion or shading cues to this end and show that we can handle a complex objective function without having to solve a difficult nonlinear minimization problem. The advantages of our method are demonstrated on a variety of problems including both real and synthetic data.
INDEX TERMS
Shape, Three dimensional displays, Image reconstruction, Space exploration, Covariance matrix, Light sources, Deformable models,monocular shape estimation, Deformable surfaces
CITATION
F. Moreno-Noguer, P. Fua, "Stochastic Exploration of Ambiguities for Nonrigid Shape Recovery", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 2, pp. 463-475, Feb. 2013, doi:10.1109/TPAMI.2012.102
REFERENCES
[1] A. Balan, M. Black, H. Haussecker, and L. Sigal, "Shining a Light on Human Pose: On Shadows, Shading and the Estimation of Pose and Shape," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.
[2] A. Blake and M. Isard, Active Contours. Springer, 1998.
[3] V. Blanz and T. Vetter, "A Morphable Model for the Synthesis of 3D Faces," Proc. ACM Siggraph, pp. 187-194, 1999.
[4] M. Brand, "Morphable 3D Models from Video," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
[5] C. Bregler, A. Hertzmann, and H. Biermann, "Recovering Non-Rigid 3D Shape from Image Streams," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000.
[6] A. Chiuso, R. Brockett, and S. Soatto, "Optimal Structure from Motion: Local Ambiguities and Global Estimates," Int'l J. Computer Vision, vol. 39, no. 3, pp. 195-228, 2000.
[7] L. Cohen and I. Cohen, "Finite-Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 11, pp. 1131-1147, Nov. 1993.
[8] T. Cootes, G. Edwards, and C. Taylor, "Active Appearance Models," Proc. European Conf. Computer Vision, pp. 484-498, 1998.
[9] A. Criminisi, Accurate Visual Metrology from Single and Multiple Uncalibrated Images. Springer, 2001.
[10] M. de La Gorce, N. Paragios, and D. Fleet, "Model-Based Hand Tracking with Texture, Shading and Self-Occlusions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[11] A. Ecker, A.D. Jepson, and K.N. Kutulakos, "Semidefinite Programming Heuristics for Surface Reconstruction Ambiguities," Proc. European Conf. Computer Vision, pp. 127-14, 2008.
[12] N. Gumerov, A. Zandifar, R. Duraiswami, and L. Davis, "Structure of Applicable Surfaces from Single Views," Proc. European Conf. Computer Vision, pp. 482-496, 2004.
[13] G. Hamerly and C. Elkan, "Learning the K in K-Means," Neural Information Processing Systems, vol. 17, 2003.
[14] N. Hansen, "The CMA Evolution Strategy: A Comparing Review," Towards a New Evolutionary Computation. Advances on Estimation of Distribution Algorithms, pp. 75-102, Springer, 2006.
[15] N. Hansen, "The CMA Evolution Strategy: A Tutorial," http://www.bionik.tu-berlin.de/user/niko cmatutorial.pdf, 2007.
[16] K. Hara, K. Nishino, and K. Ikeuchi, "Light Source Position and Reflectance Estimation from a Single View without the Distant Illumination Assumption," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 493-505, Apr. 2005.
[17] J. Liang, D. DeMenthon, and D. Doermann, "Flattening Curved Documents in Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 338-345, 2005.
[18] D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[19] J. MacCormick and M. Isard, "Partitioned Sampling, Articulated Objects, and Interface-Quality Hand Tracking," Proc. European Conf. Computer Vision, pp. 3-19, 2000.
[20] T. McInerney and D. Terzopoulos, "A Finite Element Model for 3D Shape Reconstruction and Nonrigid Motion Tracking," Proc. IEEE Int'l Conf. Computer Vision, pp. 518-523, 1993.
[21] D. Metaxas and D. Terzopoulos, "Constrained Deformable Superquadrics and Nonrigid Motion Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 580-591, 1993.
[22] F. Moreno-Noguer, J.M. Porta, and P. Fua, "Exploring Ambiguities for Monocular Non-Rigid Shape Estimation," Proc. European Conf. Computer Vision, pp. 361-374, 2010.
[23] F. Moreno-Noguer, M. Salzmann, V. Lepetit, and P. Fua, "Capturing 3D Stretchable Surfaces from Single Images in Closed Form," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1842-1849, 2009.
[24] M. Paladini, A. Del Bue, M. Stosic, M. Dodig, J. Xavier, and L. Agapito, "Optimal Metric Projections for Deformable and Articulated Structure-from-Motion," Int'l J. Computer Vision, vol. 96, pp. 1-25, 2011.
[25] M. Perriollat, R. Hartley, and A. Bartoli, "Monocular Template-Based Reconstruction of Inextensible Surfaces," Proc. British Machine Vision Conf., 2008.
[26] G. Qian and R. Chellappa, "Structure from Motion Using Sequential Montecarlo Methods," Int'l J. Computer Vision, vol. 59, no. 1, pp. 5-31, 2004.
[27] M. Salzmann and P. Fua, "Linear Local Models for Monocular Reconstruction of Deformable Surfaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 931-944, May 2011.
[28] M. Salzmann, F. Moreno-Noguer, V. Lepetit, and P. Fua, "Closed-Form Solution to Non-Rigid 3D Surface Registration," Proc. European Conf. Computer Vision, vol. 4, pp. 581-594, 2008.
[29] J. Sanchez, J. Ostlund, P. Fua, and F. Moreno-Noguer, "Simultaneous Pose, Correspondence and Non-Rigid Shape," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1189-1196, 2010.
[30] R. Szeliski and S. Kang, "Shape Ambiguities in Structure-from-Motion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 5, pp. 506-512, May 1997.
[31] L. Torresani, A. Hertzmann, and C. Bregler, "Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 878-892, May 2008.
[32] L. Tsap, D. Goldgof, and S. Sarkar, "Nonrigid Motion Analysis Based on Dynamic Refinement of Finite Element Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 5, pp. 526-543, May 2000.
[33] Y. Wang, Z.C. Liu, G. Hua, Z. Wen, Z.Y. Zhang, and D. Samaras, "Face Re-Lighting from a Single Image under Harsh Lighting Conditions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[34] R. White and D.A Forsyth, "Combining Cues: Shape from Shading and Texture," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 1809-1816, 2006.
[35] J. Xiao and T. Kanade, "Uncalibrated Perspective Reconstruction of Deformable Structures," Proc. IEEE Int'l Conf. Computer Vision, pp. 1075-1082, 2005.
[36] Y. Yu, P. Debevec, J. Malik, and T. Hawkins, "Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs," Proc. ACM Siggraph, pp. 215-224, 1999.
[37] J. Zhu, S. Hoi, Z. Xu, and M. Lyu, "An Effective Approach to 3D Deformable Surface Tracking," Proc. European Conf. Computer Vision, pp. 766-779, 2008.
56 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool