This Article 
 Bibliographic References 
 Add to: 
Plane-Based Optimization for 3D Object Reconstruction from Single Line Drawings
February 2008 (vol. 30 no. 2)
pp. 315-327
In previous optimization-based methods of 3D planar-faced object reconstruction from single 2D line drawings, the missing depths of the vertices of a line drawing (and other parameters in some methods) are used as the variables of the objective functions. A 3D object with planar faces is derived by finding values for these variables that minimize the objective functions. These methods work well for simple objects with a small number N of variables. As N grows, however, it is very difficult for them to find expected objects. This is because with the nonlinear objective functions in a space of large dimension N, the search for optimal solutions can easily get trapped into local minima. In this paper, we use the parameters of the planes that pass through the planar faces of an object as the variables of the objective function. This leads to a set of linear constraints on the planes of the object, resulting in a much lower dimensional nullspace where optimization is easier to achieve. We prove that the dimension of this nullspace is exactly equal to the minimum number of vertex depths which define the 3D object. Since a practical line drawing is usually not an exact projection of a 3D object, we expand the nullspace to a larger space based on the singular value decomposition of the projection matrix of the line drawing. In this space, robust 3D reconstruction can be achieved. Compared with two most related methods, our method not only can reconstruct more complex 3D objects from 2D line drawings, but also is computationally more efficient.

[1] K. Sugihara, Machine Interpretation of Line Drawings. MIT Press, 1986.
[2] P. Company, A. Piquer, M. Contero, and F. Naya, “A Survey on Geometrical Reconstruction as a Core Technology to Sketch-Based Modeling,” Computers & Graphics, vol. 29, pp. 892-904, 2005.
[3] W. Whiteley, “A Matroid on Hypergraphs with Applications in Scene Analysis and Geometry,” Discrete & Computational Geometry, vol. 4, pp. 75-95, 1989.
[4] M.C. Cooper, “Wireframe Projections: Physical Realisability of Curved Objects and Unambiguous Reconstruction of Simple Polyhedra,” Int'l J. Computer Vision, vol. 64, no. 1, pp. 69-88, 2005.
[5] M.C. Cooper, “The Interpretations of Line Drawings with Contrast Failure and Shadows,” Int'l J. Computer Vision, vol. 43, no. 2, pp. 75-97, 2001.
[6] A. Heyden, “On the Consistency of Line-Drawings, Obtained by Projections of Piecewise Planar Objects,” J. Math. Imaging and Vision, vol. 6, pp. 393-412, 1996.
[7] L. Ros and F. Thomas, “Overcoming Superstrictness in Line Drawing Interpretation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 456-466, Apr. 2002.
[8] H. Li, L. Zhao, and Y. Chen, “Polyhedral Scene Analysis Combining Parametric Propagation with Calotte Analysis,” Lecture Notes in Computer Science, vol. 3519, pp. 383-402, 2005.
[9] Y. Leclerc and M. Fischler, “An Optimization-Based Approach to the Interpretation of Single Line Drawings as 3D Wire Frames,” Int'l J. Computer Vision, vol. 9, no. 2, pp. 113-136, 1992.
[10] G. Markowsky and M. Wesley, “Fleshing Out Wire-Frames,” IBM J. Research and Development, vol. 24, no. 5, pp. 582-597, 1980.
[11] S. Courter and J. Brewer, “Automated Conversation of Curvilinear Wire-Frame Models to Surface Boundary Models: A Topological Approach,” Computer Graphics, vol. 20, no. 4, pp. 171-178, 1986.
[12] S. Agarwal and J. Waggenspack, “Decomposition Method for Extracting Face Topologies from Wireframe Models,” Computer-Aided Design, vol. 24, no. 3, pp. 123-140, 1992.
[13] M. Shpitalni and H. Lipson, “Identification of Faces in a 2D Line Drawing Projection of a Wireframe Object,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 10, pp. 1000-1012, Oct. 1996.
[14] J. Liu and Y. Lee, “A Graph-Based Method for Face Identification from a Single 2D Line Drawing,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1106-1119, Oct. 2001.
[15] J. Liu, Y. Lee, and W. Cham, “Identifying Faces in a 2D Line Drawing Representing a Manifold Object,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 12, pp. 1579-1593, Dec. 2002.
[16] J. Liu and X. Tang, “Evolutionary Search for Faces from Line Drawings,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 861-872, June 2005.
[17] H. Li, “nD Polyhedral Scene Reconstruction from Single 2D Line Drawing by Local Propagation,” Lecture Notes in Artifical Intelligence, vol. 3763, pp. 169-197, 2006.
[18] T. Marill, “Emulating the Human Interpretation of Line-Drawings as Three-Dimensional Objects,” Int'l J. Computer Vision, vol. 6, no. 2, pp. 147-161, 1991.
[19] E. Brown and P. Wang, “3D Object Recovery from 2D Images: A New Approach,” SPIE Proc. Robotics and Computer Vision, vol. 2904, pp. 138-145, 1996.
[20] K. Shoji, K. Kato, and F. Toyama, “3-D Interpretation of Single Line Drawings Based on Entropy Minimization Principle,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 90-95, 2001.
[21] H. Lipson and M. Shpitalni, “Optimization-Based Reconstruction of a 3D Object from a Single Freehand Line Drawing,” Computer-Aided Design, vol. 28, no. 8, pp. 651-663, 1996.
[22] A. Piquer, R. Martin, and P. Company, “Using Skewed Mirror Symmetry for Optimisation-Based 3D Line-Drawing Recognition,” Proc. Fifth IAPR Int'l Workshop Graphics Recognition, pp. 182-193, 2003.
[23] A. Shesh and B. Chen, “Smartpaper: An Interactive and User Friendly Sketching System,” Proc. Ann. Conf. European Assoc. Computer Graphics, 2004.
[24] L. Cao, J. Liu, and X. Tang, “3D Object Reconstruction from a Single 2D Line Drawing without Hidden Lines,” Proc. 10th IEEE Int'l Conf. Computer Vision, vol. 1, pp. 272-277, 2005.
[25] K. Sugihara, “An Algebraic Approach to Shape-from-Image Problems,” Artificial Intelligence, vol. 23, pp. 59-95, 1984.
[26] I. Shimshoni and J. Ponce, “Recovering the Shape of Polyhedra Using Line-Drawing Analysis and Complex Reflectance Models,” Computer Vision and Image Understanding, vol. 65, no. 2, pp. 296-310, 1997.
[27] H. Shimodaira, “A Shape-from-Shading Method of Polyhedral Objects Using Prior Information,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 612-624, Apr. 2006.
[28] A. Turner, D. Chapman, and A. Penn, “Sketching Space,” Computers & Graphics, vol. 24, pp. 869-879, 2000.
[29] P. Company, M. Contero, J. Conesa, and A. Piquer, “An Optimisation-Based Reconstruction Engine for 3D Modelling by Sketching,” Computers & Graphics, vol. 28, pp. 955-979, 2004.
[30] P.A.C. Varley and R.R. Martin, “Estimating Depth from Line Drawings,” Proc. Seventh ACM Symp. Solid Modeling and Applications, pp. 180-191, 2002.
[31] P.A.C. Varley, R.R. Martin, and H. Suzuki, “Frontal Geometry from Sketches of Engineering Objects: Is Line Labelling Necessary?” Computer-Aided Design, vol. 37, pp. 1285-1307, 2005.
[32] G. Strang, Introduction to Linear Algebra. Wellesley-Cambridge Press, 1998.
[33] W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in C++: The Art of Scientific Computing. Cambridge Univ. Press, 2002.
[34] Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs. Springer, 1996.

Index Terms:
3D object reconstruction, degree of reconstruction freedom, line drawing, nullspace, singular value decomposition
Jianzhuang Liu, Liangliang Cao, Zhenguo Li, Xiaoou Tang, "Plane-Based Optimization for 3D Object Reconstruction from Single Line Drawings," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 315-327, Feb. 2008, doi:10.1109/TPAMI.2007.1172
Usage of this product signifies your acceptance of the Terms of Use.