This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Monocular 3D Reconstruction of Locally Textured Surfaces
June 2012 (vol. 34 no. 6)
pp. 1118-1130
ASCII Text
x 
 
M. Salzmann, A. Shaji, A. Varol, P. Fua, "Monocular 3D Reconstruction of Locally Textured Surfaces," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 6, pp. 1118-1130, June, 2012.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2011.196,
author = {M. Salzmann and A. Shaji and A. Varol and P. Fua},
title = {Monocular 3D Reconstruction of Locally Textured Surfaces},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {34},
number = {6},
issn = {0162-8828},
year = {2012},
pages = {1118-1130},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.196},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Monocular 3D Reconstruction of Locally Textured Surfaces
IS - 6
SN - 0162-8828
SP1118
EP1130
EPD - 1118-1130
A1 - M. Salzmann,
A1 - A. Shaji,
A1 - A. Varol,
A1 - P. Fua,
PY - 2012
KW - surface texture
KW - image texture
KW - solid modelling
KW - surface reconstruction
KW - complex lighting
KW - monocular 3D reconstruction
KW - locally textured surfaces
KW - monocular nonrigid 3D shape recovery
KW - shading information
KW - monocular deformable shape recovery
KW - local surface patches
KW - Shape
KW - Three dimensional displays
KW - Surface reconstruction
KW - Surface texture
KW - Lighting
KW - Training
KW - Image reconstruction
KW - shape from shading.
KW - Deformable surfaces
KW - shape recovery
VL - 34
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
M. Salzmann, NICTA, Canberra, ACT, Australia
A. Shaji, EPFL-IC-Comput. Vision Lab., Lausanne, Switzerland
A. Varol, EPFL-IC-Comput. Vision Lab., Lausanne, Switzerland
P. Fua, EPFL-IC-Comput. Vision Lab., Lausanne, Switzerland
Most recent approaches to monocular nonrigid 3D shape recovery rely on exploiting point correspondences and work best when the whole surface is well textured. The alternative is to rely on either contours or shading information, which has only been demonstrated in very restrictive settings. Here, we propose a novel approach to monocular deformable shape recovery that can operate under complex lighting and handle partially textured surfaces. At the heart of our algorithm are a learned mapping from intensity patterns to the shape of local surface patches and a principled approach to piecing together the resulting local shape estimates. We validate our approach quantitatively and qualitatively using both synthetic and real data.

[1] H. Aanaes and F. Kahl, "Estimation of Deformable Structure and Motion," Proc. Vision and Modelling of Dynamic Scenes Workshop, 2002.
[2] A. Ahmed and A. Farag, "A New Formulation for Shape from Shading for Non-Lambertian Surfaces," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2006.
[3] I. Akhter, Y. Sheikh, and S. Khan, "In Defense of Orthonormality Constraints for Nonrigid Structure from Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009.
[4] P. Belhumeur, D. Kriegman, and A. Yuille, "The Bas-Relief Ambiguity," Int'l J. Computer Vision, vol. 35, no. 1, pp. 33-44, 1999.
[5] D.P. Bertsekas, Nonlinear Programming. Athena Scientific, 1999.
[6] P. Besl and N. Mckay, "A Method for Registration of 3D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[7] C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
[8] F. Bornemann and C. Rasch, "Finite-Element Discretization of Static Hamilton-Jacobi Equations Based on a Local Variational Principle," Computing and Visualization in Science, vol. 9, no. 2, pp. 57-69, 2006.
[9] M. Brand, "A Direct Method of 3D Factorization of Nonrigid Motion Observed in 2D," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 122-128, 2005.
[10] C. Bregler, A. Hertzmann, and H. Biermann, "Recovering Non-Rigid 3D Shape from Image Streams," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000.
[11] T. Collins and A. Bartoli, "Locally Affine and Planar Deformable Surface Reconstruction from Video," Proc. Int'l Workshop Vision, Modeling and Visualization, pp. 339-346, 2010.
[12] J.-D. Durou, M. Falcone, and M. Sagona, "Numerical Methods for Shape from Shading: A New Survey with Benchmarks," Computer Vision and Image Understanding, vol. 109, pp. 22-43, 2008.
[13] A. Ecker, A.D. Jepson, and K.N. Kutulakos, "Semidefinite Programming Heuristics for Surface Reconstruction Ambiguities," Proc. European Conf. Computer Vision, Oct. 2008.
[14] M. Falcone and M. Sagona, "An Algorithm for Global Solution of the Shape-from-Shading Model," Proc. Int'l Conf. Image Analysis and Processing, 1997.
[15] J. Fayad, L. Agapito, and A. Del Bue, "Piecewise Quadratic Reconstruction of Non-Rigid Surfaces from Monocular Sequences," Proc. 11th European Conf. Computer Vision, 2010.
[16] J. Fayad, A. Del Bue, L. Agapito, and P.M.Q. Aguiar, "Non-Rigid Structure from Motion Using Quadratic Deformation Models," Proc. British Machine Vision Conf., 2009.
[17] D.A. Forsyth and A. Zisserman, "Reflections on Shading," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 671-679, July 1991.
[18] N.A. Gumerov, A. Zandifar, R. Duraiswami, and L.S. Davis, "Structure of Applicable Surfaces from Single Views," Proc. European Conf. Computer Vision, May 2004.
[19] M. Han, W. Xu, H. Tao, and Y. Gong, "An Algorithm for Multiple Object Trajectory Tracking," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 864-871, June 2004.
[20] B.K.P. Horn and M.J. Brooks, Shape from Shading. MIT Press, 1989.
[21] V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1568-1583, Oct. 2006.
[22] R. Kozera, "Uniqueness in Shape from Shading Revisited," J. Math. Imaging and Vision, vol. 7, no. 2, pp. 123-138, 1997.
[23] D.J. Kriegman and P.N. Belhumeur, "What Shadows Reveal about Object Structure," Proc. European Conf. Computer Vision, pp. 399-414, 1998.
[24] J. Liang, D. Dementhon, and D. Doermann, "Flattening Curved Documents in Images," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 338-345, 2005.
[25] F. Moreno-Noguer, J. Porta, and P. Fua, "Exploring Ambiguities for Monocular Non-Rigid Shape Estimation," Proc. European Conf. Computer Vision, Sept. 2010.
[26] F. Moreno-Noguer, M. Salzmann, V. Lepetit, and P. Fu, "Capturing 3D Stretchable Surfaces from Single Images in Closed Form," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009.
[27] S.K. Nayar, K. Ikeuchi, and T. Kanade, "Shape from Interreflections," Int'l J. Computer Vision, vol. 6, no. 3, pp. 173-195, 1991.
[28] S.I. Olsen and A. Bartoli, "Implicit Non-Rigid Structure-From-Motion with Priors," J. Math. Imaging and Vision, vol. 31, pp. 233-244, 2008.
[29] M. Oren and S.K. Nayar, "A Theory of Specular Surface Geometry," Int'l J. Computer Vision, vol. 24, no. 2, pp. 105-124, 1996.
[30] M. Perriollat and A. Bartoli, "A Quasi-Minimal Model for Paper-Like Surfaces," Proc. BenCos: Workshop Towards Benchmarking Automated Calibration, Orientation and Surface Reconstruction from Images, 2007.
[31] M. Perriollat, R. Hartley, and A. Bartoli, "Monocular Template-Based Reconstruction of Inextensible Surfaces," Proc. British Machine Vision Conf., 2008.
[32] V. Rabaud and S. Belongie, "Re-Thinking Non-Rigid Structure from Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[33] V. Rabaud and S. Belongie, "Linear Embeddings in Non-Rigid Structure from Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009.
[34] R. Ramamoorthi and P. Hanrahan, "An Efficient Representation for Irradiance Environment Maps," Proc. ACM Siggraph, 2001.
[35] M. Salzmann and P. Fua, "Linear Local Models for Monocular Reconstruction of Deformable Surfaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 931-944, May 2011.
[36] M. Salzmann, V. Lepetit, and P. Fua, "Deformable Surface Tracking Ambiguities," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[37] M. Salzmann, R. Urtasun, and P. Fua, "Local Deformation Models for Monocular 3D Shape Recovery," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[38] D. Samaras and D. Metaxas, "Incorporating Illumination Constraints in Deformable Models," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 322-329, June 1998.
[39] D. Samaras, D. Metaxas, P. Fua, and Y. Leclerc, "Variable Albedo Surface Reconstruction from Stereo and Shape from Shading," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2000.
[40] A. Shaji and S. Chandran, "Riemannian Manifold Optimisation for Non-Rigid Structure from Motion," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2008.
[41] S. Shen, W. Shi, and Y. Liu, "Monocular 3D Tracking of Inextensible Deformable Surfaces under L2-Norm," Proc. Asian Conf. Computer Vision, 2009.
[42] J. Taylor, A.D. Jepson, and K.N. Kutulakos, "Non-Rigid Structure from Locally-Rigid Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2010.
[43] C. Tomasi and T. Kanade, "Shape and Motion from Image Streams under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[44] L. Torresani, A. Hertzmann, and C. Bregler, "Nonrigid Structure-From-Motion: Estimating Shape and Motion with Hierarchical Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 878-892, May 2008.
[45] P.S. Tsai and M. Shah, "Shape from Shading Using Linear Approximation," J. Image and Vision Computing, vol. 12, pp. 487-498, 1994.
[46] R. Urtasun and T. Darrell, "Sparse Probabilistic Regression for Activity-Independent Human Pose Inference," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[47] A. Varol, M. Salzmann, E. Tola, and P. Fua, "Template-Free Monocular Reconstruction of Deformable Surfaces," Proc. Int'l Conf. Computer Vision, Sept. 2009.
[48] T. Weise, B. Leibe, and L. Van Gool, "Fast 3D Scanning with Automatic Motion Compensation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[49] R. White and D.A. Forsyth, "Combining Cues: Shape from Shading and Texture," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[50] J. Xiao, J.-X. Chai, and T. Kanade, "A Closed-Form Solution to Non-Rigid Shape and Motion Recovery," Proc. European Conf. Computer Vision, pp. 573-587, 2004.
[51] Z. Zhang, C. Tan, and L. Fan, "Restoration of Curved Document Images through 3D Shape Modeling," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2004.
[52] J. Zhu, S. Hoi, C. Steven, Z. Xu, and M.R. Lyu, "An Effective Approach to 3D Deformable Surface Tracking," Proc. European Conf. Computer Vision, pp. 766-779, 2008.

Index Terms:
surface texture,image texture,solid modelling,surface reconstruction,complex lighting,monocular 3D reconstruction,locally textured surfaces,monocular nonrigid 3D shape recovery,shading information,monocular deformable shape recovery,local surface patches,Shape,Three dimensional displays,Surface reconstruction,Surface texture,Lighting,Training,Image reconstruction,shape from shading.,Deformable surfaces,shape recovery
Citation:
M. Salzmann, A. Shaji, A. Varol, P. Fua, "Monocular 3D Reconstruction of Locally Textured Surfaces," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 6, pp. 1118-1130, June 2012, doi:10.1109/TPAMI.2011.196
Usage of this product signifies your acceptance of the Terms of Use.