This Article 
 Bibliographic References 
 Add to: 
Matching Two Perspective Views
August 1992 (vol. 14 no. 8)
pp. 806-825

A computational approach to image matching is described. It uses multiple attributes associated with each image point to yield a generally overdetermined system of constraints, taking into account possible structural discontinuities and occlusions. In the algorithm implemented, intensity, edgeness, and cornerness attributes are used in conjunction with the constraints arising from intraregional smoothness, field continuity and discontinuity, and occlusions to compute dense displacement fields and occlusion maps along the pixel grids. The intensity, edgeness, and cornerness are invariant under rigid motion in the image plane. In order to cope with large disparities, a multiresolution multigrid structure is employed. Coarser level edgeness and cornerness measures are obtained by blurring the finer level measures. The algorithm has been tested on real-world scenes with depth discontinuities and occlusions. A special case of two-view matching is stereo matching, where the motion between two images is known. The algorithm can be easily specialized to perform stereo matching using the epipolar constraint.

[1] E. H. Adelson and J. R. Bergen, "Spatiotemporal energy models for the perception of motion,"J. Opt. Soc. Amer. A, vol. 2, no. 2, pp. 284-299, 1985.
[2] P. Anandan and R. Weiss, "Introducing a smoothness constraint in a matching approach for the computation of optical flow fields," inProc. Workshop Comput. Vision: Representation Contr.(Bellaire, MI), 1985, pp. 186-194.
[3] N. Ayache and O. Faugeras, "Building, registration, and fusing noisy visual maps," inProc. First Int. Conf. Comput. vision(London), 1987, pp. 73-82.
[4] S. T. Barnard and W. B. Thompson, "Disparity analysis of images,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-2, no. 4, pp. 333-340, 1980.
[5] S. T. Barnard, "A stochastic approach to stereo vision," inProc. Fifth Nat. Conf. Artificial Intell.(Philadelphia, PA), 1986, pp. 676-680.
[6] M. Born and E. Wolf,Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light. Oxford: Pergamon, 1975.
[7] O. J. Braddick, "A short-range process in apparent motion,"Vision Res., vol. 14, pp. 519-527, 1974.
[8] O. J. Braddick, "Low-level and high-level processes in apparent motion,"Philo. Trans. Royal Soc. London B, vol. 290, pp. 137-151, 1980.
[9] L. Dreschler and H. -H. Nagel, "Volumetric model and 3-D trajectory of a moving car derived from monocular TV frame sequences of a street scene,"Comput. Graphics Image Processing, vol. 20, pp. 199-228, 1982.
[10] F. Glazer, G. Reynolds, and P. Anandan, "Scene matching by hierarchical correlation," inProc. IEEE Conf. Comput. Vision Patt. Recogn., 1983, pp. 432-441.
[11] W. E. L. Grimson, "Computational experiments with a feature based stereo algorithm,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-7, no. 1, pp. 17-34, 1985.
[12] W. K. Gu, J. Y. Yang, and T. S. Huang, "Matching perspective views of a polyhedron using circuits,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-9, no. 3, pp. 390-400, 1987.
[13] D. J. Heeger, "Optical flow from spatiotemporal filters," inProc. Int. Conf. Comput. Vision(London), 1987, pp. 181-190.
[14] E. Hildreth,The Measurement of Visual Motion, Cambridge, MA: MIT Press, 1983.
[15] W. Hoff and N. Ahuja, "Surfaces from stereo: Integrating feature matching, disparity estimation, and contour detection,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-11, no. 2, pp. 121-136, 1989.
[16] B. K. P. Horn and B. G. Schunck, "Determining optical flow,"Artificial Intell., vol. 17, pp. 185-203, 1981.
[17] J.K. Kearny, W.B. Thompson, and D.L. Boley, "Optical flow estimation: An error analysis of gradient-based methods with local optimization,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-9, pp. 229-244, Mar. 1987.
[18] R. Kingslake,Lens Design Fundamentals. New York: Academic, 1978.
[19] L. Kitchen and A. Rosenfeld, "Gray-level corner detection,"Patt. Recogn. Lett., vol. 1, pp. 95-102, 1982.
[20] H. S. Lim and T. O. Binford, "Stereo correspondence: A hierarchical approach," inProc. Image Understanding Workshop, 1987.
[21] D. Marr and T. Poggio, "A theory of human stereo vision," inProc. R. Soc. London, 1979, pp. 301-328, vol. B 204.
[22] J. E. W. Mayhew and J. P. Frisby, "Psychophysical and computational studies toward a theory of human stereopsis,"Artificial Intell., vol. 17, pp. 349-385, 1981.
[23] H. P. Moravec, "Obstacle avoidance and navigation in the real world by seeing a robot rover," Stanford Artif. Intell. Lab. Memo 340, 1980.
[24] H.-H. Nagel and W. Enkelmann, "An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-8, no. 5, pp. 565-593, Sept. 1986.
[25] Y. Ohta and T. Kanade, "Stereo by intra- and inter-scanline search using dynamic programming,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-7, pp. 139-154, 1985.
[26] L. H. Quam, "Hierarchical warp stereo," inProc. DARPA Image Understanding Workshop(New Orleans, LA), 1984, pp. 149-155.
[27] S. Ullman,The Interpretation of Visual Motion. Cambridge, MA: MIT Press, 1979.
[28] A. Verri and T. Poggio, "Against quantitative optical flow," inProc. First Int. Conf. Comput. Vision(London), 1987, pp. 171-180.
[29] A. M. Waxman, "An image flow paradigm," inProc. Workshop Comput. Vision: Representation Contr.(Annapolis MD), 1984, pp. 49-57.
[30] J. Weng, T. S. Huang, and N. Ahuja, "Motion and structure from two perspective views: Algorithm, error analysis and error estimation,"IEEE Trans. Patt. Anal. Machine Intell.vol. 11, no. 5, pp. 451-476, 1989.
[31] J. Weng, T. S. Huang, and N. Ahuja, "Motion from images: Image matching, parameter estimation, and intrinsic stability," inProc. IEEE Workshop Visual Motion, (Irvine, CA), Mar. 20-22, 1989, pp. 359-366.
[32] J. Weng, N. Ahuja, and T. S. Huang, "Optimal motion and structure estimation," inProc. IEEE Conf. Comput. Vision Patt. Recogn.(San Diego, CA), 1989, pp. 144-152.
[33] O. A. Zuniga and R. M. Haralick, "Corner detection using the facet model,"Proc. IEEE Conf. Comput. Vision Patt. Recogn., 1983, pp. 30-37.

Index Terms:
two perspective views matching; pattern recognition; image matching; structural discontinuities; occlusions; edgeness; cornerness attributes; intraregional smoothness; field continuity; discontinuity; dense displacement fields; pixel grids; multiresolution multigrid structure; blurring; depth discontinuities; stereo matching; pattern recognition; picture processing
J. Weng, N. Ahuja, T.S. Huang, "Matching Two Perspective Views," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 8, pp. 806-825, Aug. 1992, doi:10.1109/34.149592
Usage of this product signifies your acceptance of the Terms of Use.