The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan. (2013 vol.35)
pp: 232-244
T. J. Cashman , Fac. of Inf., Univ. of Lugano, Lugano, Switzerland
A. W. Fitzgibbon , Microsoft Res. Ltd., Cambridge, UK
ABSTRACT
3D morphable models are low-dimensional parameterizations of 3D object classes which provide a powerful means of associating 3D geometry to 2D images. However, morphable models are currently generated from 3D scans, so for general object classes such as animals they are economically and practically infeasible. We show that, given a small amount of user interaction (little more than that required to build a conventional morphable model), there is enough information in a collection of 2D pictures of certain object classes to generate a full 3D morphable model, even in the absence of surface texture. The key restriction is that the object class should not be strongly articulated, and that a very rough rigid model should be provided as an initial estimate of the “mean shape.” The model representation is a linear combination of subdivision surfaces, which we fit to image silhouettes and any identifiable key points using a novel combined continuous-discrete optimization strategy. Results are demonstrated on several natural object classes, and show that models of rather high quality can be obtained from this limited information.
INDEX TERMS
Three dimensional displays, Solid modeling, Shape, Cameras, Image reconstruction, Dolphins, Biological system modeling,single-view reconstruction, Morphable model, shape from silhouette, subdivision surfaces, image-based modeling
CITATION
T. J. Cashman, A. W. Fitzgibbon, "What Shape Are Dolphins? Building 3D Morphable Models from 2D Images", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 1, pp. 232-244, Jan. 2013, doi:10.1109/TPAMI.2012.68
REFERENCES
[1] V. Blanz and T. Vetter, "A Morphable Model for the Synthesis of 3D Faces," Proc. ACM Siggraph, A. Rockwood, ed., pp. 187-194, 1999.
[2] B. Allen, B. Curless, and Z. Popović, "The Space of Human Body Shapes: Reconstruction and Parameterization from Range Scans," ACM Trans. Graphics, vol. 22, no. 3, pp. 587-594, 2003.
[3] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis, "SCAPE: Shape Completion and Animation of People," ACM Trans. Graphics, vol. 24, no. 3, pp. 408-416, 2005.
[4] A. Balan and M.J. Black, "The Naked Truth: Estimating Body Shape under Clothing," Proc. European Conf. Computer Vision, Part 2, D. Forsyth, P. Torr, and A. Zisserman, eds., pp. 15-29, 2008.
[5] S. Zhou, H. Fu, L. Liu, D. Cohen-Or, and X. Han, "Parametric Reshaping of Human Bodies in Images," ACM Trans. Graphics, vol. 29, no. 3, pp. 1-10, 2010.
[6] N. Snavely, S.M. Seitz, and R. Szeliski, "Photo Tourism: Exploring Photo Collections in 3D," ACM Trans. Graphics, vol. 25, no. 3, pp. 835-846, 2006.
[7] L. Torresani, A. Hertzmann, and C. Bregler, "Non-Rigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 878-892, May 2008.
[8] Y. Furukawa, A. Sethi, J. Ponce, and D.J. Kriegman, "Robust Structure and Motion from Outlines of Smooth Curved Surfaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 2, pp. 302-315, Feb. 2006.
[9] P. McIlroy and T. Drummond, "Reconstruction from Uncalibrated Affine Silhouettes," Proc. British Machine Vision Conf., pp. 1-11, 2009.
[10] M. Prasad, A. Zisserman, and A. Fitzgibbon, "Single View Reconstruction of Curved Surfaces," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1345-1354, 2006.
[11] T.F. Cootes, C.J. Taylor, D.H. Cooper, and J. Graham, "Active Shape Models—Their Training and Application," Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
[12] T. Vetter and T. Poggio, "Linear Object Classes and Image synthesis from a Single Example Image," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 733-742, July 1997.
[13] Y. Chen, T. Kim, and R. Cipolla, "Inferring 3D Shapes and Deformations from Single Views," Proc. European Conf. Computer Vision, Part 3, K. Daniilidis, P. Maragos, and N. Paragios, eds., pp. 300-313, 2010.
[14] D. Terzopoulos, A. Witkin, and M. Kass, "Symmetry-Seeking Models and 3D Object Reconstruction," Int'l J. Computer Vision, vol. 1, no. 3, pp. 211-221, 1988.
[15] Y. Gingold, T. Igarashi, and D. Zorin, "Structured Annotations for 2D-to-3D Modeling," ACM Trans. Graphics, vol. 28, no. 5, pp. 1-9, 2009.
[16] T. Igarashi, S. Matsuoka, and H. Tanaka, "Teddy: A Sketching Interface for 3D Freeform Design," Proc. ACM Siggraph, A. Rockwood, ed., pp. 409-416, 1999.
[17] O.A. Karpenko and J.F. Hughes, "SmoothSketch: 3D Free-Form Shapes from Complex Sketches," ACM Trans. Graphics, vol. 25, no. 3, pp. 589-598, 2006.
[18] X. Gu, S.J. Gortler, and H. Hoppe, "Geometry Images," ACM Trans. Graphics, vol. 21, no. 3, pp. 355-361, 2002.
[19] A. Nealen, T. Igarashi, O. Sorkine, and M. Alexa, "FiberMesh: Designing Freeform Surfaces with 3D Curves," ACM Trans. Graphics, vol. 26, no. 3, pp. 1-10, 2007.
[20] V. Kraevoy, A. Sheffer, and M. van de Panne, "Modeling from Contour Drawings," Proc. Eurographics Symp. Sketch-Based Interfaces and Modeling, pp. 37-44, 2009.
[21] J. Porrill and S. Pollard, "Curve Matching and Stereo Calibration," Image and Vision Computing, vol. 9, no. 1, pp. 45-50, 1991.
[22] R. Cipolla, K.E. Åström, and P.J. Giblin, "Motion from the Frontier of Curved Surfaces," Proc. Fifth Int'l Conf. Computer Vision, pp. 269-275, 1995.
[23] C. Bregler, A. Hertzmann, and H. Biermann, "Recovering Non-Rigid 3D Shape from Image Streams," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 690-696, 2000.
[24] M. Prasad, A.W. Fitzgibbon, A. Zisserman, and L. Van Gool, "Finding Nemo: Deformable Object Class Modelling Using Curve Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[25] S. Ilić, M. Salzmann, and P. Fua, "Implicit Meshes for Effective Silhouette Handling," Int'l J. Computer Vision, vol. 72, no. 2, pp. 159-178, 2007.
[26] L. Kobbelt, S. Campagna, and H.-P. Seidel, "A General Framework for Mesh Decimation," Proc. Graphics Interface, pp. 43-50, 1998,
[27] M. Prasad, A. Zisserman, A.W. Fitzgibbon, M.P. Kumar, and P.H.S. Torr, "Learning Class-Specific Edges for Object Detection and Segmentation," Proc. Indian Conf. Computer Vision, Graphics and Image Processing, 2006.
[28] D. Dementhon and L. Davis, "Model-Based Object Pose in 25 Lines of Code," Int'l J. Computer Vision, vol. 15, nos. 1/2, pp. 123-141, 1995.
[29] E. Tosun, "Geometric Modeling Using High-Order Derivatives," PhD dissertation, New York Univ., 2008.
[30] M. Halstead, M. Kass, and T. DeRose, "Efficient, Fair Interpolation Using Catmull-Clark Surfaces," Proc. ACM Siggraph, J.T. Kajiya, ed., pp. 35-44, 1993.
[31] F. Grassia, "Practical Parameterization of Rotations Using the Exponential Map," J. Graphics Tools, vol. 3, no. 3, pp. 29-48, 1998.
[32] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, "Bundle Adjustment—A Modern Synthesis," Proc. Int'l Workshop Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds., pp. 298-372, 2000.
[33] B. Appleton and C. Sun, "Circular Shortest Paths by Branch and Bound," Pattern Recognition, vol. 36, no. 11, pp. 2513-2520, 2003.
[34] C.T. Loop, "Smooth Subdivision Surfaces Based on Triangles," master's thesis, Univ. of Utah, Aug. 1987.
[35] J. Stam, "Exact Evaluation of Catmull-Clark Subdivision Surfaces at Arbitrary Parameter Values," Proc. ACM Siggraph, M. Cohen, ed., pp. 395-404, 1998.
[36] J. Peters and U. Reif, Subdivision Surfaces. Springer, 2008.
[37] D. Zorin, "Constructing Curvature-Continuous Surfaces by Blending," Proc. Symp. Geometry Processing, K. Polthier and A. Sheffer, eds., pp. 31-40, 2006.
[38] H. Hoppe, T. DeRose, T. Duchamp, M. Halstead, H. Jin, J. McDonald, J. Schweitzer, and W. Stuetzle, "Piecewise Smooth Surface Reconstruction," Proc. ACM Siggraph, A. Glassner, ed., pp. 295-302, 1994.
[39] D. Anguelov, D. Koller, H.-C. Pang, P. Srinivasan, and S. Thrun, "Recovering Articulated Object Models from 3D Range Data," Proc. 20th Conf. Uncertainty in Artificial Intelligence, pp. 18-26, 2004.
54 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool