This Article 
 Bibliographic References 
 Add to: 
A Maximum Likelihood Framework for Determining Moving Edges
May 1989 (vol. 11 no. 5)
pp. 499-511

The determination of moving edges in an image sequence is discussed. An approach is proposed that relies on modeling principles and likely hypothesis testing techniques. A spatiotemporal edge in an image sequence is modeled as a surface patch in a 3-D spatiotemporal space. A likelihood ratio test enables its detection as well as simultaneous estimation of its related attributes. It is shown that the computation of this test leads to convolving the image sequence with a set of predetermined masks. The emphasis is on a restricted but widely relevant and useful case of surface patch, namely the planar one. In addition, an implementation of the procedure whose computation cost is merely equivalent to a spatial gradient operator is presented. This method can be of interest for motion-analysis schemes, not only for supplying spatiotemporal segmentation, but also for extracting local motion information. Moreover, it can cope with occlusion contours and important displacement magnitude. Experiments have been carried out with both synthetic and real images.

[1] B. L. Yen and T. S. Huang, "Determining 3-D motion and structure of a rigid body using straight line correspondences,"Image Sequence Processing and Dynamic Scene Analysis. Heidelberg, Germany: Springer-Verlag, 1983.
[2] H.-H. Nagel, "Image sequences-Ten (octal) years-From phenomenology towards a theoretical foundation," inProc. 8th Int. Conf. Pattern Recognition, Paris, France, Oct. 1986, pp. 1174-1185.
[3] J. Barron, "A survey of approaches for determining optical flow, environmental layout and egomotion," Dep. Comput. Sci., Univ. Toronto, Canada, RBCV-TR-84-5, Nov. 1984.
[4] A. Mitiche, "Computation of optical flow and rigid motion," inProc. 2nd IEEE Workshop Comput. Vision: Representation and Contr., Annapolis, MD, 1984, pp. 63-71.
[5] A. M. Waxman and S. Ullman, "Surface structure and three-dimensional motion from image flow kinematics,"Int. J. Robot. Res., vol. 4, no. 3, pp. 72-94, 1985.
[6] G. Adiv, "Determining three-dimensional motion and structure from optical flow generated by several moving objects,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-7, pp. 384-401, July 1985.
[7] H.-H. Nagel, "Overview on image sequence analysis," inImage Sequence Processing and Dynamic Scene Analysis, T. S. Huang, Ed., NATO-ASI Series, Vol. F2. New York: Springer-Verlag, 1983, pp. 2-39.
[8] R. Jain, "Dynamic scene analysis," inProgress in Pattern Recognition 2, L. Kanal and A. Rosenfeld, Eds.,Machine Intelligence and Pattern Recognition Series, Vol. 1. New York: North-Holland, 1985, pp. 125-167.
[9] D. C. Marr and S. Ullman, "Directional selectivity and its use in early visual processing,"Proc. Roy. Soc. London, vol. B211, pp. 151-180, Mar. 1981.
[10] B. K. P. Horn and B. G. Schunck, "Determining optical flow,"Artificial Intell., vol. 17, pp. 185-203, 1981.
[11] A. Rougée, B. Levy, and A. S. Willsky, "Optic flow estimation inside a bounded domain," Lab. for Inform. Decision Syst., M.I.T., Cambridge, Rep. LIDS-P-1589, Aug. 1986.
[12] H.-H. Nagel and W. Enkelmann, "An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-8, no. 5, pp. 565-593, Sept. 1986.
[13] G. R. Legters and T. Y. Young, "A mathematical model for computer image tracking,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-4, pp. 583-594, Nov. 1982.
[14] B. Espiau and P. Rives, "Closed-loop recursive estimation of 3D features for a mobile vision system," inProc. Conf. IEEE Robot. Automation, Raleigh, NC, Mar. 1987.
[15] R. Storey, "HDTV motion adaptive bandwidth reduction using DATV," inProc. Int. EURASIP Workshop Coding of HDTV, vol. 2, Italy, Nov. 1986.
[16] R. Jain, W. N. Martin, and J. K. Aggarwal, "Segmentation through the detection of changes due to motion,"Comput. Graphics Image Processing, vol. CGIP-11, pp. 13-34, 1979.
[17] Y. Z. Hsu, H.-H. Nagel, and G. Rekers, "New likelihood test methods for change detection in image sequences,"Comput. Vision, Graphics, Image Processing, vol. CVGIP-26, pp. 73-106, 1984.
[18] D. W. Murray and N. S. Williams, "Detecting the image boundaries between optical flow fields from several moving planar facets,"Pattern Recognition Lett., vol. 4, pp. 87-92, Apr. 1986.
[19] P. Bouthemy and J. Santillana Rivero, "A hierarchical likelihood approach for region segmentation according to motion-based criteria," inProc. 1st Int. Conf. Comput. Vision, London, England, June 1987, pp. 463-467.
[20] S. M. Haynes and R. Jain, "Time varying edge detection," inProc. 6th Int. Conf. Pattern Recognition, Munich, Germany, Oct. 1982, pp. 754-756.
[21] E. C. Hildreth, "The detection of intensity changes by computer and biological vision systems,"Comput. Vision, Graphics, Image Processing, vol. CVGIP-22, pp. 1-27, 1983.
[22] B. F. Buxton and H. Buxton, "Computation of optic flow from the motion of edge features in image sequences,"Image Vision Comput., vol. 2, pp. 59-75, May 1984.
[23] P. Kahn, "Local determination of a moving contrast edge,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-7, pp. 402-409, July 1985.
[24] B. G. Schunck, "The image flow contraint equation,"Comput. Vision Graphics Image Proc., vol. 35, pp. 20-46, 1986.
[25] E. Hildreth, "Computation underlying the measurement of visual motion,"Artificial Intell., vol. 23, pp. 309-354, 1984.
[26] P. Bouthemy, "A method of integrating motion information along contours including segmentation," inProc. 8th Int. Conf. Pattern Recognition, Paris, France, Oct. 1986, pp. 651-653.
[27] M. H. Hueckel, "An operator which localizes edges in digitized pictures," inJ. Assoc. Comput. Machine, vol. 18, pp. 113-125, 1971.
[28] R. M. Haralick, "Digital step edges from zero crossing of second directional derivatives,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-6, pp. 58-68, Jan. 1984.
[29] J. W. Modestino and R. W. Fries, "Edge detection in noisy images using recursive digital filtering,"Comput. Graphics Image Processing, vol. CGIP-6, pp. 409-433, 1977.
[30] D. B. Cooper and H. Elliott, "A maximum likelihood framework for boundary estimation in noisy images," inProc. IEEE Conf. Pattern Recognition Image Processing, Chicago, IL, 1978, pp. 25-31.
[31] Y. Yakimovsky, "Boundary and object detection in real world images,"JACM, vol. 23, no. 4, pp. 598-619, Oct. 1976.
[32] J. F. Canny, "Finding lines and edges in images," Artificial Intell. Lab., Massachusetts Inst. Technol., Tech. Rep. TM-720, 1983.
[33] P. Bouthemy, "Estimation of edge motion based on local modeling," inProc. SPIE Conf. Comput. Vision for Robots, vol. 595, Cannes, France, Dec. 1985, pp. 162-169.
[34] C. Labit and A. Benveniste, "Motion estimation in a sequence of television pictures," inImage Sequence Processing and Dynamic Scene Analysis, T. S. Huang, Ed.,NATO-ASI Series, Vol. F2. New York: Springer-Verlag, 1983, pp. 292-306.
[35] F. Glazer, "Computing optic flow," inProc. 7th Int. Joint Conf. Artificial Intell., Vancouver, Canada, 1981, pp. 644-647.
[36] S. W. Zucker and R. A. Hummel, "A three-dimensional edge operator,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-3, pp. 324-331, May 1981.
[37] R. Nevatia and K. R. Babu, "Linear feature extraction and description,"Comput. Graphics Image Processing, vol. CGIP-13, pp. 257- 269, 1980.
[38] L. Marcéand P. Bouthemy, "Determination of a depth map from an image sequence," inProc. 3rd Int. Conf. Adv. Robot., Paris, France, Oct. 1987, pp. 221-232.
[39] D. J. Fleet and A. D. Jepson, "On the hierarchical construction of orientation and velocity selective filters," Dep. Comput. Sci., Univ. Toronto, Canada, RBCV-TR-85-8, Nov. 1985.
[40] D. J. Heeger, "Model for the extraction of image flow,"J. Opt. Soc. Amer. A, vol. 4, pp. 1455-1471, Aug. 1987.
[41] L. Jacobson and H. Wechsler, "Derivation of optical flow using a spatiotemporal-frequency approach,"Comput. Vision, Graphics, Image Processing, vol. CVGIP-38, pp. 29-65, 1987.
[42] S. G. Mallat, "A theory for multiresolution signal decomposition: The wavelet representation," Dep. Comput. Inform. Sci., Univ. Pennsylvania, Philadelphia, MS-CIS-87-22, May 1987.
[43] P. Bouthemy, "Détermination du movement apparent dans une séquence d'images: Extraction de primitives locales, structuration intermédiaire, estimation du champ des vitesses,"Traitement du Signal, vol. 4, no. 3, pp. 239-257, 1987.

Index Terms:
picture processing; information extraction; maximum likelihood framework; moving edges; hypothesis testing; surface patch; 3-D spatiotemporal space; spatiotemporal segmentation; occlusion contours; displacement magnitude; picture processing
P. Bouthemy, "A Maximum Likelihood Framework for Determining Moving Edges," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, no. 5, pp. 499-511, May 1989, doi:10.1109/34.24782
Usage of this product signifies your acceptance of the Terms of Use.