The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2012 vol.34)
pp: 889-901
Hoang-Hiep Vu , Ecole des Ponts, Paris
Patrick Labatut , Ecole des Ponts, Paris
Jean-Philippe Pons , Ecole des Ponts, Paris
Renaud Keriven , Ecole des Ponts, Paris
ABSTRACT
Since the initial comparison of Seitz et al. [48], the accuracy of dense multiview stereovision methods has been increasing steadily. A number of limitations, however, make most of these methods not suitable to outdoor scenes taken under uncontrolled imaging conditions. The present work consists of a complete dense multiview stereo pipeline which circumvents these limitations, being able to handle large-scale scenes without sacrificing accuracy. Highly detailed reconstructions are produced within very reasonable time thanks to two key stages in our pipeline: a minimum s-t cut optimization over an adaptive domain that robustly and efficiently filters a quasidense point cloud from outliers and reconstructs an initial surface by integrating visibility constraints, followed by a mesh-based variational refinement that captures small details, smartly handling photo-consistency, regularization, and adaptive resolution. The pipeline has been tested over a wide range of scenes: from classic compact objects taken in a laboratory setting, to outdoor architectural scenes, landscapes, and cultural heritage sites. The accuracy of its reconstructions has also been measured on the dense multiview benchmark proposed by Strecha et al. [59], showing the results to compare more than favorably with the current state-of-the-art methods.
INDEX TERMS
Dense multiview stereo, surface reconstruction, large-scale scenes, minimum s-t cut, deformable mesh.
CITATION
Hoang-Hiep Vu, Patrick Labatut, Jean-Philippe Pons, Renaud Keriven, "High Accuracy and Visibility-Consistent Dense Multiview Stereo", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 5, pp. 889-901, May 2012, doi:10.1109/TPAMI.2011.172
REFERENCES
[1] S. Agarwal, N. Snavely, I. Simon, S.M. Seitz, and R. Szeliski, "Building Rome in a Day," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[2] A. Banno, T. Masuda, T. Oishi, and K. Ikeuchi, "Flying Laser Range Sensor for Large-Scale Site-Modeling and Its Applications in Bayon Digital Archival Project," Int'l J. Computer Vision, vol. 78, nos. 2/3, pp. 207-222, 2008.
[3] J.L. Bentley, "Multidimensional Binary Search Trees Used for Associative Searching," Comm. ACM, vol. 18, no. 9, pp. 509-517, 1975.
[4] J.-D. Boissonnat, O. Devillers, M. Teillaud, and M. Yvinec, "Triangulations in CGAL," Proc. 16th Ann. Symp. Computational Geometry, pp. 11-18, 2000.
[5] Y. Boykov and V. Kolmogorov, "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124-1137, Sept. 2004.
[6] Y. Boykov and V. Lempitsky, "From Photohulls to Photoflux Optimization," Proc. British Machine Vision Conf., vol. 3, pp. 1149-1158, 2006.
[7] D. Bradley, T. Boubekeur, and W. Heidrich, "Accurate Multi-View Reconstruction Using Robust Binocular Stereo and Surface Meshing," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[8] A. Broadhurst, T.W. Drummond, and R. Cipolla, "A Probabilistic Framework for Space Carving," Proc. IEEE Int'l Conf. Computer Vision, pp. 388-393, 2001.
[9] N.D.F. Campbell, G. Vogiatzis, C. Hernández, and R. Cipolla, "Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo," Proc. European Conf. Computer Vision, pp. 766-779, 2008.
[10] F. Cazals and J. Giesen, "Delaunay Triangulation Based Surface Reconstruction, Mathematics and Visualization," Effective Computational Geometry for Curves and Surfaces, pp. 231-276, Springer, 2006.
[11] A. Delaunoy, E. Prados, P. Gargallo, J.-P. Pons, and P. Sturm, "Minimizing the Multi-View Stereo Reprojection Error for Triangular Surface Meshes," Proc. 19th British Machine Vision Conf., 2008.
[12] Y. Duan, L. Yang, H. Qin, and D. Samaras, "Shape Reconstruction from 3D and 2D Data Using PDE-Based Deformable Surfaces," Proc. European Conf. Computer Vision, vol. 3, pp. 238-251, 2004.
[13] I. Eckstein, J.-P. Pons, Y. Tong, C.C.J. Kuo, and M. Desbrun, "Generalized Surface Flows for Mesh Processing," Proc. Fifth Symp. Geometry Processing, pp. 183-192, 2007.
[14] O. Faugeras and R. Keriven, "Variational Principles, Surface Evolution, PDE's, Level Set Methods and the Stereo Problem," IEEE Trans. Image Processing, vol. 7, no. 3, pp. 336-344, Mar. 1998.
[15] L.R. Ford and D.R. Fulkerson, Flows in Networks. Princeton Univ. Press, 1962.
[16] J.-M. Frahm, P. Georgel, D. Gallup, T. Johnson, R. Raguram, C. Wu, Y.-H. Jen, E. Dumn, B. Clipp, S. Lazebnik, and M. Pollefeys, "Building Rome on a Cloudless Day," Proc. European Conf. Computer Vision, 2010.
[17] Y. Furukawa, B. Curless, S.M. Seitz, and R. Szeliski, "Towards Internet-Scale Multi-View Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[18] Y. Furukawa and J. Ponce, "Carved Visual Hulls for Image-Based Modeling," Proc. European Conf. Computer Vision, pp. 564-577, May 2006.
[19] Y. Furukawa and J. Ponce, "Accurate, Dense, and Robust Multi-View Stereopsis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[20] Y. Furukawa and J. Ponce, "Dense 3D Motion Capture from Synchronized Video Streams," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[21] Y. Furukawa and J. Ponce, "Accurate, Dense, and Robust Multi-View Stereopsis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 8, pp. 1362-1376, Aug. 2009.
[22] P. Gargallo and P. Sturm, "Bayesian 3D Modeling from Images Using Multiple Depth Maps," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 885-891, June 2005.
[23] M. Goesele, B. Curless, and S.M. Seitz, "Multi-View Stereo Revisited," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2402-2409, 2006.
[24] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S.M. Seitz, "Multi-View Stereo for Community Photo Collections," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[25] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[26] C. Hernández and F. Schmitt, "Silhouette and Stereo Fusion for 3D Object Modeling," Computer Vision and Image Understanding, vol. 96, no. 3, pp. 367-392, Dec. 2004.
[27] C. Hernández, G. Vogiatzis, and R. Cipolla, "Probabilistic Visibility for Multi-View Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[28] A. Hornung and L. Kobbelt, "Hierarchical Volumetric Multi-View Stereo Reconstruction of Manifold Surfaces Based on Dual Graph Embedding," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 503-510, 2006.
[29] A. Hornung and L. Kobbelt, "Robust and Efficient Photo-Consistency Estimation for Volumetric 3D Reconstruction," Proc. European Conf. Computer Vision, 2006.
[30] M. Jancosek and T. Pajdla, "Segmentation Based Multi-View Stereo," Proc. Computer Vision Winter Workshop, 2009.
[31] M. Jancosek, A. Shekhovtsov, and T. Pajdla, "Scalable Multi-View Stereo," Proc. IEEE Int'l Workshop 3D Digital Imaging and Modeling, 2009.
[32] H. Jin, S. Soatto, and A.J. Yezzi, "Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance," Int'l J. Computer Vision, vol. 63, no. 3, pp. 175-189, 2005.
[33] M. Kazhdan, M. Bolitho, and H. Hoppe, "Poisson Surface Reconstruction," Proc. Fourth Eurographics Symp. Geometry Processing, pp. 61-70, June 2006.
[34] L. Kobbelt, S. Campagna, J. Vorsatz, and H.-P. Seidel, "Interactive Multi-Resolution Modeling on Arbitrary Meshes," Proc. Int'l Conf. Computer Graphics and Interactive Techniques, pp. 105-114, 1998.
[35] K. Kolev, M. Klodt, T. Brox, and D. Cremers, "Continuous Global Optimization in Multiview 3D Reconstruction," Int'l J. Computer Vision, vol. 84, pp. 80-96, 2009.
[36] V. Kolmogorov and R. Zabih, "Multi-Camera Scene Reconstruction via Graph Cuts," Proc. European Conf. Computer Vision, vol. 3, pp. 82-96, May 2002.
[37] K.N. Kutulakos and S.M. Seitz, "A Theory of Shape by Space Carving," Int'l J. Computer Vision, vol. 38, no. 3, pp. 199-218, 2000.
[38] P. Labatut, J.-P. Pons, and R. Keriven, "Efficient Multi-View Reconstruction of Large-Scale Scenes Using Interest Points, Delaunay Triangulation and Graph Cuts," Proc. IEEE Int'l Conf. Computer Vision, Oct. 2007.
[39] P. Labatut, J.-P. Pons, and R. Keriven, "Hierarchical Shape-Based Surface Reconstruction for Dense Multi-View Stereo," Proc. IEEE Int'l Workshop 3D Digital Imaging and Modeling, 2009.
[40] P. Labatut, J.-P. Pons, and R. Keriven, "Robust and Efficient Surface Reconstruction from Range Data," Computer Graphics Forum, vol. 28, no. 8, pp. 2275-2290, 2009.
[41] A. Laurentini, "The Visual Hull Concept for Silhouette-Based Image Understanding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 2, pp. 150-162, Feb. 1994.
[42] V. Lempitsky and Y. Boykov, "Global Optimization for Shape Fitting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[43] V. Lempitsky, Y. Boykov, and D. Ivanov, "Oriented Visibility for Multiview Reconstruction," Proc. European Conf. Computer Vision, vol. 3, pp. 226-238, May 2006.
[44] M. Lhuillier and L. Quan, "A Quasi-Dense Approach to Surface Reconstruction from Uncalibrated Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 418-433, Mar. 2005.
[45] P. Merrell, A. Akbarzadeh, L. Wang, P. Mordohai, J.-M. Frahm, R. Yang, D. Nistér, and M. Pollefeys, "Real-Time Visibility-Based Fusion of Depth Maps," Proc. IEEE Int'l Conf. Computer Vision, Oct. 2007.
[46] J.-P. Pons, R. Keriven, and O. Faugeras, "Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score," Int'l J. Computer Vision, vol. 72, no. 2, pp. 179-193, 2007.
[47] N. Salman and M. Yvinec, "Surface Reconstruction from Multi-View Stereo," Proc. Int'l Workshop Representation and Modeling of Large-Scale 3D Environments, 2009.
[48] S.M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 519-526, June 2006.
[49] S.M. Seitz and C.R. Dyer, "Photorealistic Scene Reconstruction by Voxel Coloring," Int'l J. Computer Vision, vol. 35, no. 2, pp. 151-173, Nov. 1999.
[50] S.N. Sinha, P. Mordohai, and M. Pollefeys, "Multi-View Stereo via Graph Cuts on the Dual of an Adaptive Tetrahedral Mesh," Proc. IEEE Int'l Conf. Computer Vision, Oct. 2007.
[51] S.N. Sinha and M. Pollefeys, "Multi-View Reconstruction Using Photo-Consistency and Exact Silhouette Constraints: A Maximum-Flow Formulation," Proc. IEEE Int'l Conf. Computer Vision, pp. 349-356, Oct. 2005.
[52] G. Slabaugh and G. Unal, "Active Polyhedron: Surface Evolution Theory Applied to Deformable Meshes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 84-91, 2005.
[53] N. Snavely, S.M. Seitz, and R. Szeliski, "Modeling the World from Internet Photo Collections," Int'l J. Computer Vision, vol. 80, pp. 189-210, 2008.
[54] N. Snavely, S.M. Seitz, and R. Szeliski, "Skeletal Sets for Efficient Structure from Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[55] J. Starck, G. Miller, and A. Hilton, "Volumetric Stereo with Silhouette and Feature Constraints," Proc. British Machine Vision Conf., vol. 3, pp. 1189-1198, 2006.
[56] C. Strecha, R. Fransens, and L.V. Gool, "Wide-Baseline Stereo from Multiple Views: A Probabilistic Account," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 552-559, 2004.
[57] C. Strecha, R. Fransens, and L.V. Gool, "Combined Depth and Outlier Estimation in Multi-View Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2394-2401, 2006.
[58] C. Strecha, T. Tuytelaars, and L.V. Gool, "Dense Matching of Multiple Wide-Baseline Views," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1194-1201, 2003.
[59] C. Strecha, W. von Hansen, L.V. Gool, P. Fua, and U. Thoennessen, "On Benchmarking Camera Calibration and Multi-View Stereo for High Resolution Imagery," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[60] G. Taubin, "A Signal Processing Approach to Fair Surface Design," Proc. ACM Siggraph, pp. 351-358, 1995.
[61] S. Tran and L. Davis, "3D Surface Reconstruction Using Graph Cuts with Surface Constraints," Proc. European Conf. Computer Vision, vol. 2, pp. 219-231, 2006.
[62] R. Tylecek and R. Sara, "Depth Map Fusion with Camera Position Refinement," Proc. Computer Vision Winter Workshop, 2009.
[63] R. Tylecek and R. Sara, "Refinement of Surface Mesh for Accurate Multi-View Reconstruction," Int'l J. Virtual Reality, vol. 9, pp. 45-54, 2010.
[64] G. Vogiatzis, C. Hernández, P.H.S. Torr, and R. Cipolla, "Multi-View Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2241-2246, Dec. 2007.
[65] G. Vogiatzis, P.H.S. Torr, and R. Cipolla, "Multi-View Stereo via Volumetric Graph-Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 391-398, 2005.
[66] H.H. Vu, R. Keriven, P. Labatut, and J.-P. Pons, "Towards High-Resolution Large-Scale Multi-View Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009.
[67] R. Yang and M. Pollefeys, "Multi-Resolution Real-Time Stereo on Commodity Graphics Hardware," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 211-220, 2003.
[68] R. Yang, M. Pollefeys, and G. Welch, "Dealing with Textureless Regions and Specular Highlights: A Progressive Space Carving Scheme Using a Novel Photo-Consistency Measure," Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 576-584, 2003.
[69] T. Yu, N. Ahuja, and W.-C. Chen, "SDG Cut: 3D Reconstruction of Non-Lambertian Objects Using Graph Cuts on Surface Distance Grid," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2269-2276, 2006.
[70] C. Zach, T. Pock, and H. Bischof, "A Globally Optimal Algorithm for Robust TV-L1 Range Image Integration," Proc. IEEE Int'l Conf. Computer Vision, Oct. 2007.
[71] A. Zaharescu, E. Boyer, and R.P. Horaud, "TransforMesh: A Topology-Adaptive Mesh-Based Approach to Surface Evolution," Proc. Asian Conf. Computer Vision, pp. 166-175, Nov. 2007.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool