Subscribe
Issue No.01 - January (2008 vol.30)
pp: 131-146
ABSTRACT
In this paper, the duality in differential form is developed between a 3D primal surface and its dual manifold formed by the surface&#8217;s tangent planes, i.e., each tangent plane of the primal surface is represented as a four-dimensional vector which constitutes a point on the dual manifold. The iterated dual theorem shows that each tangent plane of the dual manifold corresponds to a point on the original 3D surface, i.e., the &#8220;dual&#8221; of the &#8220;dual&#8221; goes back to the &#8220;primal&#8221;. This theorem can be directly used to reconstruct 3D surface from image edges by estimating the dual manifold from these edges. In this paper we further develop the work in our original conference papers resulting in the robust differential dual operator. We argue that the operator makes good use of the information available in the image data, by using both points of intensity discontinuity and their edge directions; we provide a simple physical interpretation of what the abstract algorithm is actually estimating and why it makes sense in terms of estimation accuracy; our algorithm operates on all edges in the images, including silhouette edges, self occlusion edges, and texture edges, without distinguishing their types (thus resulting in improved accuracy and handling locally concave surface estimation if texture edges are present); the algorithm automatically handles various degeneracies; and the algorithm incorporates new methodologies for implementing the required operations such as appropriately relating edges in pairs of images, evaluating and using the algorithm&#8217;s sensitivity to noise to determine the accuracy of an estimated 3D point. Experiments with both synthetic and real images demonstrate that the operator is accurate, robust to degeneracies and noise, and general for reconstructing free-form objects from occluding edges and texture edges detected in calibrated images or video sequences.
INDEX TERMS
3D reconstruction robust to degeneracies and noise, duality in differential form, dual manifold, multi-view reconstruction, shape from silhouettes, shape from occlusions and textures, dynamic programming
CITATION
Shubao Liu, Kongbin Kang, Jean-Philippe Tarel, David B. Cooper, "Free-Form Object Reconstruction from Silhouettes, Occluding Edges and Texture Edges: A Unified and Robust Operator Based on Duality", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 1, pp. 131-146, January 2008, doi:10.1109/TPAMI.2007.1143
REFERENCES
 [1] R. Raskar, K.-H. Tan, R. Feris, J. Yu, and M. Turk, “Nonphotorealistic Camera: Depth Edge Detection and Stylized Rendering Using Multi-Flash Imaging,” ACM Trans. Graphics, vol. 23, no. 3, pp. 679-688, 2004. [2] D. Crispell, D. Lanman, P.G. Sibley, Y. Zhao, and G. Taubin, “Beyond Silhouettes: Surface Reconstruction Using Multi-Flash Photographing,” Proc. Int'l Symp. 3D Data Processing, Visualization, and Transmission, 2006. [3] A. Laurentini, “The Visual Hull Concept for Silhouette-Based Image Understanding,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 2, pp. 150-162, Feb. 1994. [4] E. Boyer and M.-O. Berger, “3D Surface Reconstruction Using Occluding Contours,” Int'l J. Computer Vision, vol. 22, pp. 219-233, Mar. 1997. [5] S. Sullivan and J. Ponce, “Automatic Model Construction and Pose Estimation from Photographs Using Triangular Splines,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 10, pp.1091-1096, Oct. 1998. [6] K.N. Kutulakos and S.M. Seitz, “A Theory of Shape by Space Carving,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 38, no. 3, pp. 199-218, Mar. 2000. [7] S. Lazebnik, E. Boyer, and J. Ponce, “On Computing Exact Visual Hulls of Solids Bounded by Smooth Surfaces,” Proc. Computer Vision and Pattern Recognition, pp. 156-161, 2001. [8] J.-S. Franco and E. Boyer, “Exact Polyhedral Visual Hulls,” Proc. 14th British Machine Vision Conf., pp. 329-338, Sept. 2003. [9] J.-S. Franco, M. Lapierre, and E. Boyer, “Visual Shapes of Silhouette Sets,” Proc. Int'l Symp. 3D Data Processing, Visualization, and Transmission, 2006. [10] L. Guan, S. Sinha, J.-S. Franco, and M. Pollefeys, “Visual Hull Construction in the Presence of Partial Occlusion,” Proc. Int'l Symp. 3D Data Processing, Visualization, and Transmission, June 2006. [11] R. Cipolla and A. Blake, “Surface Shape from Deformation of Apparent Contours,” Int'l J. Computer Vision, vol. 9, no. 2, pp. 83-112, 1992. [12] R. Cipolla and P. Giblin, Visual Motion of Curves and Surfaces. Cambridge Univ. Press, 2000. [13] P.J. Giblin and R.S. Weiss, “Reconstructions of Surfaces from Profiles,” Proc. Int'l Conf. Computer Vision, pp. 136-144, 1987. [14] R. Vaillant and O. Faugeras, “Using Extremal Boundaries for 3-D Object Modeling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 157-173, Feb. 1992. [15] K. Wong, P. Mendonca, and R. Cipolla, “Structure and Motion Estimation from Apparent Contours under Circular Motion,” Image and Vision Computing, vol. 20, pp. 441-448, Apr. 2002. [16] K. Kang, J.-P. Tarel, R. Fishman, and D.B. Cooper, “A Linear Dual-Space Approach to 3D Surface Reconstruction from Occluding Contours Using Algebraic Surface,” Proc. Int'l Conf. Computer Vision, vol. 1, pp. 198-204, 2001. [17] K. Kang, J.-P. Tarel, and D.B. Cooper, “A Unified Linear Fitting Approach for Singular and Non-Singular 3D Quadrics from Occluding Contours,” Proc. Int'l Workshop Higher-Level Knowledge, pp. 48-57, 2003. [18] K. Kang, “Three-Dimensional Free Form Surface Reconstruction from Occluding Contours in a Sequence Images or Video,” PhD dissertation, Division of Eng., Brown Univ., May 2004. [19] M. Brand, K. Kang, and D.B. Cooper, “Algebraic Solution for the Visual Hull,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 30-35, 2004. [20] C. Liang and K.-Y.K. Wong, “Complex 3D Shape Recovery Using a Dual-Space Approach,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 878-884, 2005. [21] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004. [22] H.H. Baker and R.C. Bolles, “Generalizing Epipolar-Plane Image Analysis on the Spatiotemporal Surface,” Int'l J. Computer Vision, vol. 3, no. 1, pp. 33-49, 1989. [23] Q.-T. Luong and O. Faugeras, The Geometry of Multiple Images. MIT Press, 2004. [24] C.L. Zitnick and T. Kanade, “A Cooperative Algorithm for Stereo Matching and Occlusion Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 7, pp. 675-684, July 2000. [25] Y. Nakamura, T. Matsuura, K. Satoh, and Y. Ohta, “Occlusion Detectable Stereo-Occlusion Patterns in Camera Matrix,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 371-378, 1996. [26] A.F. Bobick and S.S. Intille, “Large Occlusion Stereo,” Int'l J. Computer Vision, vol. 33, no. 3, pp. 181-200, 1999. [27] S.B. Kang, R. Szeliski, and J. Chai, “Handling Occlusions in Dense Multi-View Stereo,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 01, p. 103, 2001. [28] A.S. Ogale and Y. Aloimonos, “Shape and the Stereo Correspondence Problem,” Int'l J. Computer Vision, vol. 65, Oct. 2005. [29] A.S. Ogale, C. Fermuller, and Y. Aloimonos, “Motion Segmentation Using Occlusions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 988-992, June 2005. [30] R. Szeliski and R. Weiss, “Robust Shape Recovery from Occluding Contours Using a Linear Smoother,” Int'l J. Computer Vision, vol. 28, no. 1, pp. 27-44, 1998. [31] C. Esteban and F. Schmitt, “Silhouette and Stereo Fusion for 3D Object Modeling,” Computer Vision and Image Understanding, vol. 96, pp. 367-392, Dec. 2004. [32] J. Browne, Grassmann Algebra—Exploring Applications of Extended Vector Algebra with Mathematica, Internet draft, work in progress, Jan. 2001. [33] P. Giblin and R. Weiss, “Epipolar Fields on Surfaces,” Proc. Third European Conf. Computer Vision, vol. 1, pp. 14-23, May 1994. [34] S. Needleman and C. Wunsch, “A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins,” J. Molecular Biology, vol. 48, pp. 443-453, 1970. [35] G.H. Golub and C.F. VanLoan, Matrix Computations. Johns Hopkins Univ., 1996. [36] P. Wedin, “Perturbation Bounds in Connection with Singular Value Decomposition,” BIT, vol. 12, pp. 99-111, 1972. [37] G.W. Stewart, “Perturbation Theory for the Singular Value Decomposition,” Technical Report CS-TR-2539, 1990. [38] F.M. Dopico, “A Note on $\sin\theta$ Theorems for Singular Subspace Variations,” BIT, vol. 40, pp. 395-403, 2000. [39] F. Leymarie, “Three-Dimensional Shape Representation via Shock Flows,” PhD dissertation, Division of Eng., Brown Univ., May 2003. [40] Z. Zhang, “Flexible Camera Calibration by Viewing a Plane from Unknown Orientations,” Proc. Int'l Conf. Computer Vision, vol. 1, pp. 666-673, 1999. [41] H. Barton, “A Modern Presentation of Grassmann's Tensor Analysis,” Am. J. Math., vol. XLIX, pp. 598-614, 1927.