This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Match Propogation for Image-Based Modeling and Rendering
August 2002 (vol. 24 no. 8)
pp. 1140-1146

This paper presents a quasi-dense matching algorithm between images based on the match propagation principle. The algorithm starts from a set of sparse seed matches, then propagates to the neighboring pixels by the best-first strategy, and produces a quasi-dense disparity map. The quasi-dense matching aims at broad modeling and visualization applications which rely heavily on matching information. Our algorithm is robust to initial sparse match outliers due to the best-first strategy. It is efficient in time and space as it is only output sensitive. It handles half-occluded areas because of the simultaneous enforcement of newly introduced discrete 2D gradient disparity limit and the uniqueness constraint. The properties of the algorithm are discussed and empirically demonstrated. The quality of quasi-dense matching are validated through intensive real examples.

[1] P. Anandan, “A Computational Framework and an Algorithm for the Measurement of Visual Motion,” Int'l J. Computer Vision, vol. 2, pp. 283-310, 1989.
[2] S. Avidan and A. Shashua, “Novel View Synthesis in Tensor Space,” Proc. Conf. Computer Vision and Pattern Recognition pp. 1034-1040, June 1997.
[3] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43–77, 1994.
[4] Y. Boykov, O. Veksler, R. Zabih, "Disparity Component Matching for Visual Correspondence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[5] P. Burt and B. Julesz, “A Gradient Limit for Binocular Fusion,” Science, vol. 208, pp. 615-617, 1980.
[6] Q. Chen and G. Medioni, “A Volumetric Stereo Matching Method: Application to Image-Based Modeling,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 29-34, 1999.
[7] S.E. Chen and L. Williams, “View Interpolation for Image Synthesis,” Proc. ACM SIGGRAPH '93, pp. 279-288, Aug. 1993.
[8] U. Dhond and J.K. Aggarwal, "Structure From Stereo—A Review," IEEE Trans. Systems, Man, and Cybernetics, vol. 19, no. 6, pp. 1,489-1,510, Nov. 1989.
[9] L. Falkenhagen, “Depth Estimation from Stereoscopic Image Pairs Assuming Piecewise Continuous Surfaces,” Proc. Workshop Image Processing for Broadcast and Video Production, Y. Paker and S. Wilbur, eds., Series on Workshops in Computing, pp. 115-127, Nov. 1994.
[10] O. Faugeras, "What can be seen in three dimensions with an uncalibrated stereo rig?" Second European Conf. Computer Vision, pp. 563-578, 1992.
[11] A.W. Fitzgibbon, G. Cross, and A. Zisserman, “Automatic 3D Model Construction for Turn-Table Sequences,” Proc. 3D Structure from Multiple Images of Large-Scale Environments, European Workshop, SMILE '98, R. Koch and L. Van Gool, eds., vol. 1506, pp. 155-170, June 1998.
[12] P. Fua, “Combining Stereo and Monocular Information to Compute Dense Depth Maps that Preserve Discontinuities,” Proc. 12th Int'l Joint Conf. Artificial Intelligence, Aug. 1991.
[13] S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M.F. Cohen, “The Lumigraph,” Proc. SIGGRAPH '96, pp. 43-54, 1996.
[14] C. Harris and M. Stephens, “A Combined Corner and Edge Detector,” Proc. Alvey Vision Conf., pp. 147-151, 1988.
[15] R.I. Hartley, “Lines and Points in Three Views and the Trifocal Tensor,” Int'l J. Computer Vision, vol. 22, no. 2, pp. 125-140, 1997.
[16] R. Hartley, R. Gupta, and T. Chang, “Stereo from Uncalibrated Cameras,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 761-764, June 1992.
[17] S.S. Intille and A.F. Bobick, “Disparity-Space Images and Large Occlusion Stereo,” Proc. European Conf. Computer Vision, pp. 179-186, 1994.
[18] E. Izquierdo and S. Kruse, “Image Analysis for 3D Modeling, Rendering and Virtual View Generation,” Computer Vision and Image Understanding, vol. 71, no. 2, pp. 231-253, Aug. 1998.
[19] A. Koschan, “What is New in Computational Stereo since 1989: A Survey on Stereo Papers,” Technical Report 93-22, Dept. Computer Science, Univ. of Berlin, Aug. 1993.
[20] S. Laveau and O. Faugeras, “3D Scene Representation as a Collection of Images and Fundamental Matrices,” Technical Report 2205, INRIA, Feb. 1994.
[21] M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc. SIGGRAPH '96, pp. 31-42, 1996.
[22] M. Lhuillier, “Efficient Dense Matching for Textured Scenes Using Region Growing,” Proc. Ninth British Machine Vision Conf., pp. 700-709, 1998.
[23] M. Lhuillier and L. Quan, “Image Interpolation by Joint View Triangulation,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 139-145, 1999.
[24] M. Lhuillier and L. Quan, “Edge-Constrained Joint View Triangulation for Image Interpolation,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 218-224, 2000.
[25] M. Lhuillier and L. Quan, “Robust Dense Matching Using Local and Global Geometric Constraints,” Proc. 15th Int'l Conf. Pattern Recognition, vol. 1, pp. 968-972, 2000.
[26] B.D. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proc. Seventh Int'l Joint Conf. Artificial Intelligence, 1981.
[27] L. McMillan and G. Bishop, "Plenoptic Modeling: An Image-Based Rendering System," Siggraph 95 Conf. Proc., ACM Press, New York, 1995, pp. 39-46.
[28] H.P. Moravec, “Towards Automatic Visual Obstacle Avoidance,” Proc. Fifth Int'l Joint Conf. Artificial Intelligence, p. 584, Aug. 1977.
[29] P.J. Narayanan, P.W. Rander, and T. Kanade, “Constructing Virtual Worlds Using Dense Stereo,” Proc. Sixth Int'l Conf. Computer Vision, pp. 3-10, 1998.
[30] M. Okutomi and T. Kanade, “A Multiple-Baseline Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 4, pp. 353-363, Apr. 1993.
[31] G.P. Otto and T.K. Chau, “A Region-Growing Algorithm for Matching of Terrain Images,” Image and Vision Computing, vol. 7, no. 2, pp. 83-94, 1989.
[32] S.B. Pollard, J.E.W. Mayhew, and J.P. Frisby, “PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Constraint,” Perception, vol. 14, pp. 449-470, 1985.
[33] M. Pollefeys, R. Koch, and L. Van Gool, “Self-Calibration and Metric Reconstruction in Spite of Varying and Unknown Internal Camera Parameters,” Proc. Int'l Conf. Computer Vision, pp. 90-95, Jan. 1998.
[34] L. Quan, Invariants of Six Points and Projective Reconstruction from Three Uncalibrated Images IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 1, pp. 34-46, Jan. 1995.
[35] G.M. Quénot, “The Orthogonal Algorithm for Optical Flow Detection Using Dynamic Programming,” Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, 1992.
[36] D. Scharstein, “Stereo Vision for View Synthesis,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 852-858, June 1996.
[37] S.M. Seitz and C.R. Dyer, “View Morphing,” ACM SIGGRAPH, pp. 21-30, 1996.
[38] A. Shashua, “Trilinearity in Visual Recognition by Alignment” Proc. Third European Conf. Computer Vision, J.O. Eklundh, ed., pp. 479-484, May 1994.
[39] J. Shi and C. Tomasi, Good Features to Track Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[40] R. Szeliski and H.Y. Shum, “Motion Estimation with Quadtree Splines,” Proc. Fifth Int'l Conf. Computer Vision, pp. 757-763, June 1995.
[41] P. Torr and D. Murray, “The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix,” Int'l J. Computer Vision, vol. 3, no. 24, pp. 271-300, 1997.
[42] Z. Zhang, R. Deriche, O. Faugeras, and Q.T. Luong, “A Rubust Technique for Matching Two Uncalibrated Images through the Recovery of the Unknown Epipolar Geometry,” Artificial Intelligence J., vol. 78, pp. 87-119, 1995.
[43] Z. Zhang and Y. Shan, “A Progressive Scheme for Stereo Matching,” Proc. Second Workshop Structure from Multiple Image of Large Scale Environments, ECCV Workshop, pp. 26-31, July 2000.

Index Terms:
Quasi-dense matching, stereo vision, image-based modeling, rendering.
Citation:
Maxime Lhuillier, Long Quan, "Match Propogation for Image-Based Modeling and Rendering," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1140-1146, Aug. 2002, doi:10.1109/TPAMI.2002.1023810
Usage of this product signifies your acceptance of the Terms of Use.