The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - August (2010 vol.32)
pp: 1362-1376
Yasutaka Furukawa , Google Inc., Seattle
Jean Ponce , Ecole Normale Supérieure, LIENS, and ENS/INRIA/CNRS, Paris
ABSTRACT
This paper proposes a novel algorithm for multiview stereopsis that outputs a dense set of small rectangular patches covering the surfaces visible in the images. Stereopsis is implemented as a match, expand, and filter procedure, starting from a sparse set of matched keypoints, and repeatedly expanding these before using visibility constraints to filter away false matches. The keys to the performance of the proposed algorithm are effective techniques for enforcing local photometric consistency and global visibility constraints. Simple but effective methods are also proposed to turn the resulting patch model into a mesh which can be further refined by an algorithm that enforces both photometric consistency and regularization constraints. The proposed approach automatically detects and discards outliers and obstacles and does not require any initialization in the form of a visual hull, a bounding box, or valid depth ranges. We have tested our algorithm on various data sets including objects with fine surface details, deep concavities, and thin structures, outdoor scenes observed from a restricted set of viewpoints, and “crowded” scenes where moving obstacles appear in front of a static structure of interest. A quantitative evaluation on the Middlebury benchmark [CHECK END OF SENTENCE] shows that the proposed method outperforms all others submitted so far for four out of the six data sets.
INDEX TERMS
Computer vision, 3D/stereo scene analysis, modeling and recovery of physical attributes, motion, shape.
CITATION
Yasutaka Furukawa, Jean Ponce, "Accurate, Dense, and Robust Multiview Stereopsis", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 8, pp. 1362-1376, August 2010, doi:10.1109/TPAMI.2009.161
REFERENCES
[1] S.M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "Multi-View Stereo Evaluation," http://vision.middlebury.edumview/, 2010.
[2] S.M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[3] O. Faugeras and R. Keriven, "Variational Principles, Surface Evolution, PDE's, Level Set Methods and the Stereo Problem," IEEE Trans. Image Processing, vol. 7, no. 3, pp. 336-344, Mar. 1998.
[4] S. Paris, F. Sillion, and L. Quan, "A Surface Reconstruction Method Using Global Graph Cut Optimization," Proc. Asian Conf. Computer Vision, http://artis.imag.fr/Publications/2004PSQ04 , Jan. 2004.
[5] J.-P. Pons, R. Keriven, and O.D. Faugeras, "Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score," Int'l J. Computer Vision, vol. 72, no. 2, pp. 179-193, 2007.
[6] S. Tran and L. Davis, "3D Surface Reconstruction Using Graph Cuts with Surface Constraints," Proc. European Conf. Computer Vision, 2006.
[7] G. Vogiatzis, P.H. Torr, and R. Cipolla, "Multi-View Stereo Via Volumetric Graph-Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[8] A. Hornung and L. Kobbelt, "Hierarchical Volumetric Multi-View Stereo Reconstruction of Manifold Surfaces Based on Dual Graph Embedding," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[9] S. Sinha, P. Mordohai, and M. Pollefeys, "Multi-View Stereo via Graph Cuts on the Dual of an Adaptive Tetrahedral Mesh," Proc. Int'l Conf. Computer Vision, 2007.
[10] C. Hernández Esteban and F. Schmitt, "Silhouette and Stereo Fusion for 3D Object Modeling," Computer Vision and Image Understanding, vol. 96, no. 3, pp. 367-392, 2004.
[11] A. Zaharescu, E. Boyer, and R. Horaud, "Transformesh: A Topology-Adaptive Mesh-Based Approach to Surface Evolution," Proc. Asian Conf. Computer Vision, Y. Yagi, S.B. Kang, I.-S. Kweon, and H. Zha, eds., pp. 166-175, 2007.
[12] Y. Furukawa and J. Ponce, "Carved Visual Hulls for Image-Based Modeling," Int'l J. Computer Vision, vol. 81, pp. 53-67, Mar. 2009.
[13] B. Baumgart, "Geometric Modeling for Computer Vision," PhD dissertation, Stanford Univ., 1974.
[14] M. Goesele, B. Curless, and S.M. Seitz, "Multi-View Stereo Revisited," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2402-2409, 2006.
[15] C. Strecha, R. Fransens, and L.V. Gool, "Combined Depth and Outlier Estimation in Multi-View Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2394-2401, 2006.
[16] D. Bradley, T. Boubekeur, and W. Heidrich, "Accurate Multi-View Reconstruction Using Robust Binocular Stereo and Surface Meshing," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[17] M. Lhuillier and L. Quan, "A Quasi-Dense Approach to Surface Reconstruction from Uncalibrated Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 418-433, Mar. 2005.
[18] M. Habbecke and L. Kobbelt, "Iterative Multi-View Plane Fitting," Proc. 11th Fall Workshop Vision, Modeling, and Visualization, 2006.
[19] L. Kobbelt and M. Botsch, "A Survey of Point-Based Techniques in Computer Graphics," Computers & Graphics, vol. 28, no. 6, pp. 801-814, 2004.
[20] R.L. Carceroni and K.N. Kutulakos, "Multi-View Scene Capture by Surfel Sampling: From Video Streams to Non-Rigid 3D Motion, Shape and Reflectance," Int'l J. Computer Vision, vol. 49, nos. 2-3, pp. 175-214, 2002.
[21] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S.M. Seitz, "Multi-View Stereo for Community Photo Collections," Proc. Int'l Conf. Computer Vision, 2007.
[22] G.P. Otto and T.K.W. Chau, "'Region-Growing' Algorithm for Matching of Terrain Images," Image Vision Computing, vol. 7, no. 2, pp. 83-94, 1989.
[23] V. Ferrari, T. Tuytelaars, and L. Van Gool, "Simultaneous Object Recognition and Segmentation by Image Exploration," Proc. European Conf. Computer Vision, 2004.
[24] A. Kushal and J. Ponce, "A Novel Approach to Modeling 3D Objects from Stereo Views and Recognizing them in Photographs," Proc. European Conf. Computer Vision, vol. 2, pp. 563-574, 2006.
[25] Y. Furukawa and J. Ponce, "PMVS," http://grail.cs.washington. edu/software pmvs, 2010.
[26] Y. Furukawa and J. Ponce, "Accurate, Dense, and Robust Multi-View Stereopsis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[27] E. Tola, V. Lepetit, and P. Fua, "A Fast Local Descriptor for Dense Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[28] W. Naylor and B. Chapman, "Wnlib," http://www.willnaylor. comwnlib.html, 2010.
[29] M. Kazhdan, M. Bolitho, and H. Hoppe, "Poisson Surface Reconstruction," Proc. Symp. Geometry Processing, 2006.
[30] H. Hoppe, T. DeRose, T. Duchamp, J. McDonald, and W. Stuetzle, "Mesh Optimization," Proc. SIGGRAPH, pp. 19-26, 1993.
[31] G. Vogiatzis, C.H. Esteban, P.H.S. Torr, and R. Cipolla, "Multiview Stereo Via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2241-2246, Dec. 2007.
[32] J.-S. Franco and E. Boyer, "Efficient Polyhedral Modeling from Silhouettes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 414-427, Mar. 2009.
[33] C. Zach, "Fast and High Quality Fusion of Depth Maps," Proc. 3D Data Processing, Visualization and Transmission, 2008.
[34] N. Campbell, G. Vogiatzis, C. Hernandez, and R. Cipolla, "Multiple Hypotheses Depth-Maps for Multi-View Stereo," Proc. European Conf. Computer Vision, 2008.
[35] C. Strecha, W. von Hansen, L.V. Gool, P. Fua, and U. Thoennessen, "On Benchmarking Camera Calibration and Multi-View Stereo for High Resolution Imagery," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[36] C. Strecha, "Multi-View Evaluation," http://cvlab.epfl.ch/~strechamultiview, 2010.
[37] C. Strecha, R. Fransens, and L.V. Gool, "Wide Baseline Stereo from Multiple Views: A Probabilistic Account," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[38] D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 7-42, 2002.
[39] C.H. Esteban, G. Vogiatzis, and R. Cipolla, "Probabilistic Visibility for Multi-View Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
23 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool