This Article 
 Bibliographic References 
 Add to: 
A Correlation-Relaxation-Labeling Framework for Computing Optical Flow - Template Matching from a New Perspective
September 1995 (vol. 17 no. 9)
pp. 843-853

Abstract—Optical flow estimation is discussed based on a model for time-varying images more general than that implied in the work of Horn and Schunk [21]. The emphasis is on applications where low contrast imagery, non-rigid or evolving object patterns movement, as well as large interframe displacements are encountered. Template matching is identified as having advantages over point correspondence and the gradient-based approach in dealing with such applications. The two fundamental uncertainties in feature matching procedures, whether it is template matching or feature point correspondences, are discussed. Correlation template matching procedures are established based on likelihood measurement. A new method for determining optical flow is developed by combining template matching and relaxation labeling. In this method, a number of candidate displacements for each template and their respective likelihood measures are first determined. Then, relaxation labeling is employed to iteratively update each candidate’s likelihood by requiring smoothness within a motion field. Real cloud images taken from meteorological satellites are used to test the usefulness of this method. It is shown in this application that the new method can deal effectively with the uncertainty of multiple peak (multi-modal) correlation surfaces encountered in template matching. The results show significant improvement when compared to that of the maximum cross correlation (MCC), which has been operationally used for cloud tracking, and to that of the method of Barnard and Thompson, which estimates displacements based on combining point correspondences with relaxation labeling.

[1] Proc. First Int’l Wind Workshop, Workshop on wind extraction from operational meteorological satellite data. Washington, D.C., Sept.17-19, 1991, EUMETSAT.
[2] Proc. Second Int’l Wind Workshop, Workshop on wind extraction from operational meteorological satellite data. Tokyo, Japan, Dec.13-15, 1993, EUMETSAT.
[3] G. Adiv,“Determining 3D motion and structure from optical flows generated by several objects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 4, pp. 384-401, 1985.
[4] G. Adiv, “Inherent Ambiguities in Recovering 3-D Motion and Structure from a Noisy Flow Field,” Trans. Pattern Analysis and Machine Intelligence, vol. 11, pp. 477–489, 1989.
[5] E. H. Adelson,J. R. Bergen,“Spatiotemporal energy models for the perception of motion,” J. Opt. Soc. Am. A, vol. 2, no. 2, pp. 284-299, 1985.
[6] J. Aisbett, “Optical Flow with Intensity-Weighted Smoothing,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 5, pp. 512-522, May 1989.
[7] P. Anandan,“Measuring visual motion from image sequences,” PhD Thesis, COINS Department, Univ. of Massachusetts, Amherst, 1987.
[8] D. I. Barnea,H. F. Silverman,“A class of algorithms for fast digital image registration,” Trans. Computers, vol. 21, pp. 179-186, 1972.
[9] S. T. Barnard,W. B. Thompson,“Disparity analysis of images,” Trans. Pattern Analysis and Machine Intelligence, vol. 2, no. 4, pp. 333-340. 1980.
[10] O. J. Braddick,“A short-range process in apparent motion,” Vision Res., vol. 14, pp. 519-527, 1974.
[11] O. J. Braddick,“Low-level and high-level processes in apparent motion,” Philo. Trans. Royal Soc. London B, vol. 290, pp. 137-151, 1980.
[12] L. Dreschler,H. H. Nagel,“Volumetric model and 3D trajectory of a moving car derived from monocular TV frame sequences of a street scene,” Comput. Graphics Image Processing, vol. 20, pp. 199-288, 1982.
[13] W. J. Emery,A. C. Thomas,, and M. J. Collins,“An objective method for computing advective surface velocities from sequential infrared images,” J. Geophysical Research, vol. 91, no. C11, pp. 12865-12878, 1986.
[14] W. Enkelmann,“Investigations of multigrid algorithms for estimation of optical flow fields in image sequences,” Computer Vision Graphics, Image Processing, vol. 43, no. 2, pp. 150-177, 1988.
[15] F. Glazer,“Hierarchical gradient-based motion detection,” Proc. Image Understanding Workshop,Los Angeles, CA, 1987, pp. 733-748.
[16] W. E. L. Grimson,“Computational experiments with a feature based stereo algorithm,” Trans. Pattern Analysis and Machine Intelligence Intell., vol. 7, no. 1, pp. 17-34, 1985
[17] J. L. Harris,“Resolving power and decision theory,” J. Opt. Soc. Amer., vol. 54, pp. 606-611, 1964.
[18] D. J. Heeger,“Model for the extraction of image flow,” J. Opt. Soc. Am. A, vol. 2, no. 2, pp. 1455-1471, 1987.
[19] E. C. Hildreth,“The measurement of visual motion,” Cambridge, MA: MIT Press, 1984.
[20] W. Hoff and N. Ahuja, "Surfaces From Stereo: Integrating Feature Matching, Disparity Estimation, and Contour Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 2, pp. 121-136, Feb. 1989.
[21] B. K. P. Horn,B. G. Schunk,“Determining Optical Flow,” Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[22] R. A. Hummel,S. W. Zucker,“On the foundations of relaxation labeling processes,” Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 3, pp 276-287, 1983.
[23] M. Kamachi,“Advective surface velocities derived from sequential images for rotational flow field: limitations and applications of maximum cross-correlation method with rotational registration,” J. Geophysical Research, vol. 94, no. C12, pp 18227-18233, 1989.
[24] J. K. Kearney,W. B. Thompson,, and D. L. Boley,“Optical flow estimation: An error analysis of gradient-based methods with local optimization,” Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 2, pp 229-244, 1987.
[25] G. Kelly,“Satellite observations for global modeling,” Adv. Space Res. vol. 12, no. 7, pp. (7)263-(7)275, 1992.
[26] R. Kories,G. Zimmermann,“Motion detection in image sequences: an evaluation of feature detectors,” in Proc. 7th Int. Conf. on Pattern Recognition,Montreal, Canada, 1984, pp. 778-780.
[27] J. A. Leese,C. S. Novak,“An automated technique for obtaining cloud motion from geosynchronous satellite data using cross-correlation,” J. Appl. Meteorology, vol. 10, pp 118-132, 1971.
[28] J. A. Leese,C. S. Novak,, and V. R. Taylor,“The detection cloud pattern motions from geosynchronous satellite image data,” Pattern Recognition, vol. 2, pp. 279-292, 1970.
[29] J. O. Limb and J. A. Murphy,“Estimating the velocity of moving images in television signals,” Computer Graphics Image Processing, vol. 4, pp. 311-327, 1975.
[30] H. C. Longuet-Higgins,“A computer program for reconstructing a scene from two projection,” Nature, vol. 293, pp. 133-135, 1981.
[31] J. McGregor,C. J. Marks,“Making the most of GMS imagery,” Proc. 6th New Zealand Image Processing Workshop, August29-30, 1991, pp. 61-65.
[32] D. Marr,T. Poggio,“A computational theory of human stereo vision,” in Proc. R. Soc. London, 1979, pp. 301-324, vol. B204.
[33] H. P. Moravec,“Towards automatic visual obstacle avoidance,” in Proc. 5th Int. Joint Conf. Artificial Intell., pp. 584,Cambridge, MA, Aug. 1977.
[34] H. H. Nagel,W. Enkelmann,“An Investigation of smoothness constraints for the estimation of displacement vector fields from image sequences,” Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 5, pp. 565-593, 1986
[35] A. N. Netravali and J. D. Robbins,“Motion-compensated television coding: Part I,” Bell Syst. Tech. J., vol. 58, no. 3, Mar. 1979.
[36] A. Rosenfeld,R. A. Hummel,, and S. W. Zucker,“Scene labeling by relaxation operations,” Trans. Syst., Man, Cybern., vol. SMC-6, no. 6, pp. 420-433, 1976.
[37] A. Rosenfeld and A.C. Kak,Digital Picture Processing. Academic Press, 2nd ed., 1982
[38] T. W. Ryan,“The Prediction of Cross-Correlation Accuracy in Digital Stereo-Pair Images. PhD thesis, University of Arizona, 1981.
[39] J. Schmetz,K. Holmlund,J. Hoffman,B. Strauss,B. Mason,V. Gaertner,A. Koch,, and L. Van De Berg,“Operational cloud-motion winds from Meteosat infrared images,” J. Appl. Meteorol., vol. 32, no. 7, pp. 1206-1225, Jul. 1993.
[40] G. L. Scott,“Four-line method of locally estimating optic flow,” Image and Vision Computing, vol. 5, no. 2, 1986.
[41] M. A. Shuh,R. Jain,“Detecting time-varying corners,” in Proc 7th Int. Conf. on Pattern Recognition, pp. 2-5,Montreal, Canada, 1984.
[42] A. Singh,Optic Flow Computation, A unified perspective, IEEE Computer Society Press, 1991.
[43] E. A. Smith,D. R. Phillips,“Automated cloud tracking using precisely aligned digital ATS pictures,” Trans. Computers, vol. 21, pp. 715-729, 1972.
[44] R. Y. Tsai,T. S. Huang,“Uniqueness and estimation of 3D motion parameters of rigid bodies with curved surfaces,” Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 1, pp. 13-27, 1984.
[45] J. Weng,N. Ahuja,, and T. S. Huang,“Matching two perspective views,” Trans. Pattern Analysis and Machine Intelligence Intell., vol. 14, no. 8, pp. 806-825, 1992.
[46] Q. X. Wu and D. Pairman,“Computing sea surface velocities from sea surface temperature—A relaxation labeling approach,” Remote Sensing: Global Monitoring for Earth Management, vol. I,, pp. 157-161, Proc. International Geoscience and Remote Sensing Symposium, Helsinki Univ. of Tech nology, Espoo, Finland, June3-6, 1991.
[47] Q. X. Wu,D. Pairman,S. McNeill,, and E. J. Barnes,“Computing advective velocities from satellite images of sea surface temperature,” Trans. Geoscience and Remote Sensing, vol. 30, no. 1, pp. 166-176, 1992.
[48] Q. X. Wu,D. Pairman,E. J. Barnes,“Aspects of applying a relaxation labeling method to computing sea surface velocities from sea surface temperature images,” Proc. First Pacific Ocean Remote Sensing Conference, pp. 802-807, Okinawa, Japan, August25-31, 1992.
[49] Q. X. Wu,“A correlation-relaxation algorithm for computing velocity fields from time-lapsed satellite images,” in Proc. 6th Australasian Remote Sensing Conference, pp. 40-47,Wellington, New Zealand, 2-6, Nov. 1992, vol. 3.
[50] Q. X. Wu,“Computing velocity field from sequential satellite images,” in Satellite Remote Sensing of Oceanic Environment, Ed. Jones, Sugimori and Stewart, Publisher: Seibutsu Kenkyusha Co. Ltd., 1993, Chap. 2.4, pp. 38-47.
[51] Q. X. Wu,“Computing cloud motion using a correlation relaxation algorithm: Improving estimation by exploiting problem knowledge,” Proc. second Int’l wind workshop,Tokyo, Japan Dec.13-15 1993, EUMETSAT.
[52] S. Ullman,The interpretation of visual motion, Cambridge, MA: MIT Press, 1979.
[53] C. Zick and J. McGregor,Private Communication, Inst. of Geophysics, Victoria, University of Wellington, 1992.

Index Terms:
Optical flow, dynamic scene analysis, motion estimation, cloud motion fields, template matching, cross-correlation, relaxation labeling.
Qing X. Wu, "A Correlation-Relaxation-Labeling Framework for Computing Optical Flow - Template Matching from a New Perspective," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 9, pp. 843-853, Sept. 1995, doi:10.1109/34.406650
Usage of this product signifies your acceptance of the Terms of Use.