This Article 
 Bibliographic References 
 Add to: 
Image Flow Segmentation and Estimation by Constraint Line Clustering
October 1989 (vol. 11 no. 10)
pp. 1010-1027

Image flow is the velocity field in the image plane caused by the motion of the observer, objects in the scene, or apparent motion, and can contain discontinuities due to object occlusion in the scene. An algorithm that can estimate the image flow velocity field when there are discontinuities due to occlusions is described. The constraint line clustering algorithm uses a statistical test to estimate the image flow velocity field in the presence of step discontinuities in the image irradiance or velocity field. Particular emphasis is placed on motion estimation and segmentation in situations such as random dot patterns where motion is the only cue to segmentation. Experimental results on a demanding synthetic test case and a real image are presented. A smoothing algorithm for improving the velocity field estimate is also described. The smoothing algorithm constructs a smooth estimate of the velocity field by approximating a surface between step discontinuities. It is noted that the velocity field estimate can be improved using surface reconstruction between velocity field boundaries.

[1] B. G. Schunck, "The image flow contraint equation,"Comput. Vision Graphics Image Proc., vol. 35, pp. 20-46, 1986.
[2] B. G. Schunck, "The motion constraint equation for optical flow," inProc. Int. Joint Conf. Pattern Recogn., 1984, pp. 20-22.
[3] B. G. Schunck, "Motion segmentation and estimation," Ph.D. thesis, Dep. Elec. Eng. Comput. Sci., M.I.T., Cambridge, MA, 1983.
[4] B. G. Schunck, "Image flow continuity equations for motion and density," inProc. Workshop Motion, 1986.
[5] A. Verri and T. Poggio, "Qualitative information in the optical flow," inProc. DARPA Image Understanding Workshop, 1987, pp. 825- 834.
[6] O. Braddick, "A short-range process in apparent motion,"Vision Res., vol. 14, pp. 519-527, 1974.
[7] H. Nagel, "From digital picture processing to image analysis," inProc. Int. Conf. Image. Anal. Processing, 1980.
[8] T. Poggio, "Early vision: From computational structure to algorithms and parallel hardware,"CVGIP, vol. 31, pp. 139-155, 1985.
[9] B. K. P. Horn and B. G. Schunck, "Determining optical flow,"Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[10] T. Poggio and W. Reichardt, "Visual control of orientation behavior in the fly,"Quart. Rev. Biophys., vol. 9, pp. 377-438, 1976.
[11] W. Reichardt and T. Poggio, "Figure-ground discrimination by relative movement in the visual system of the fly,"Bio. Cybern., vol. 35, pp. 81-100, 1979.
[12] E. Buckner, "Elementary movement detectors in an insect visual system,"Bio. Cybern., vol. 24, pp. 85-101, 1976.
[13] M. Schetzen,The Volterra and Wiener Theories of Nonlinear Systems. New York: Wiley, 1980.
[14] B. G. Haskell, "Frame-to-frame coding of television pictures using two-dimensional Fourier transforms,"IEEE Trans. Inform. Theory, vol. 20, pp. 119-120, 1974.
[15] J. O. Limb and J. A. Murphy, "Measuring the speed of moving objects from television,"IEEE Trans. Commun., vol. COM-23, pp. 474-478, 1975.
[16] J. O. Limb and J. A. Murphy, "Estimating the velocity of moving images in television signals,"Comp. Graph. Image Proc., vol. 4, pp. 311-327, 1975.
[17] C. Cafforio and F. Rocca, "Methods for measuring small displacements of television images,"IEEE Trans. Inform. Theory, vol. IT-22, pp. 573-579, 1976.
[18] A. N. Netravali and J. D. Robbins, "Motion-compensated television coding: Part I,"Bell Syst. Tech. J., vol. 58, pp. 631-670, 1979.
[19] J. A. Stuller and A. N. Netravali, "Transform domain motion estimation,"Bell Syst. Tech. J., vol. 58, pp. 1673-1702, 1979.
[20] J. A. Stuller, A. N. Netravali, and J. D. Robbins, "Interframe television coding using gain and displacement compensation,"Bell Syst. Tech. J., vol. 59, pp. 1227-1240, 1980.
[21] R. A. Jones and H. Rashid, "Residual recursive displacement estimation," inProc. Conf. Pattern Recog. Image Processing, 1981, pp. 508-509.
[22] R. Paquin and E. Dubois, "A spatio-temporal gradient method for estimating the displacement field in time-varying imagery,"Comp. Vision, Graph. Image Proc., vol. 21, pp. 205-221, 1983.
[23] L. S. Davis, Z. Wu, and H. Sun, "Contour-based motion estimation," Tech. Rep., Comput. Vision Lab., Univ. Maryland, 1981.
[24] E. C. Hildreth, "The computation of the velocity field,"Proc. Roy. Soc. Lond. B, vol. 221, pp. 189-220, 1984.
[25] E. Hildreth,The Measurement of Visual Motion, Cambridge, MA: MIT Press, 1983.
[26] E. C. Hildreth and S. Ullman, "The measurement of visual motion," Memo 699, Artificial Intelligence Lab., M.I.T., Cambridge, MA, 1982.
[27] S. M. Anstis, "Phi movement as a subtraction process,"Vision Res., vol. 10, pp. 1411-1430, 1970.
[28] S. M. Anstis and B. J. Rogers, "Illusory reversal of visual depth and movement during changes of contrast,"Vision Res., vol. 15, pp. 957- 961, 1975.
[29] S. Ullman,The Interpretation of Visual Motion. Cambridge, MA: M.I.T. Press, 1979.
[30] D. Marr and E. C. Hildreth, "Theory of edge detection,"Proc. Roy. Soc. Lond. B, vol. 207, pp. 187-217, 1980.
[31] K. Nakayama and J. Loomis, "Optical velocity patterns, velocity-sensitive neurons, and space perception: A hypothesis,"Perception, vol. 3, pp. 63-80, 1974.
[32] J. Batali and S. Ullman, "Motion detection and analysis," inProc. DARPA Image Understand. Workshop, Nov. 1979.
[33] K. M. Mutch and W. B. Thompson, "Analysis of accretion and deletion at boundaries in dynamic scenes,"IEEE Trans. Pattern Anal. Machine Intel., vol. 7, pp. 133-138, Mar. 1985.
[34] W. B. Thompson, K. M. Mutch, and V. A. Berzins, "Dynamic occlusion analysis in optical flow fields,"IEEE Trans. Pattern Anal. Machine Intel., vol. 7, pp. 374-383, July 1985.
[35] C. L. Fennema and W. B. Thompson, "Velocity determination in scenes containing several moving objects,"Comp. Graph Image Proc., vol. 9, pp. 301-315, 1979.
[36] B. G. Schunck, "Motion segmentation and estimation by constraint line clustering," inProc. Workshop Comput. Vision, 1984, pp. 58- 62.
[37] D. Marr and S. Ullman, "Directional selectivity and its use in early visual processing,"Proc. Roy. Soc. Lond. B, vol. 211, pp. 151-180, 1981.
[38] B. Noble and J. W. Daniel,Applied Linear Algebra. Englewood Cliffs, NJ: Prentice-Hall, 1977, 2nd ed.
[39] A. Ben-Israel and T. N. E. Greville,Generalized Inverses: Theory and Applications. New York: Wiley-Interscience, 1974.
[40] T. L. Boullion and P. L. Odell,Generalized Inverse Matrices. New York: Wiley-Interscience, 1971.
[41] C. L. Lawson and R. J. Hanson,Solving Least Squares Problems. Englewood Cliffs, NJ: Prentice-Hall, 1974.
[42] M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,"Commun. ACM, vol. 24, no. 6, pp. 381-395, 1981.
[43] H. A. David,Order Statistics. New York: Wiley, 1981, 2nd ed.
[44] P. J. Huber,Robust Statistics. New York: Wiley, 1981.
[45] R. J. Serfling,Approximation Theorems of Mathematical Statistics. New York: Wiley, 1980.
[46] J. F. Canny, "A computational approach to edge detection,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, pp. 679-697, 1986.
[47] R. Nevatia,Machine Perception. Englewood Cliffs, NJ: Prentice-Hall, 1982.
[48] W. E. L. Grimson,From Images to Surfaces: A Computational Study of the Human Early visual System. Cambridge, MA: MIT Press, 1981.
[49] W. E. L. Grimson, "A computational theory of visual surface interpolation," Memo 613, Artificial Intelligence Lab., M.I.T., 1981.
[50] W. E. L. Grimson, "The implicit constraints of the primal sketch," Memo 663, Artificial Intelligence Lab., M.I.T., 1981.
[51] D. Terzopoulos, "Multi-level reconstruction of visual surfaces: Variational principles and finite element representations," AI Memo 671, MIT, Apr. 1982.
[52] A. Blake, "Reconstructing a visible surface," inProc. Nat. Conf. Artificial Intelligence, 1984, pp. 23-26.
[53] M. Brady and B. K. P. Horn, "Rotationally symmetric operators for surface interpolation," Memo 654, Artificial Intelligence Lab., M.I.T., Nov. 1981.
[54] D. Terzopoulos, "Multilevel computational processes for visual surface reconstruction,"Comp. Vision, Graph. Image Proc., vol. 24, pp. 52-96, 1983.
[55] J. L. Marroquin, "Surface reconstruction preserving discontinuities," Massachusetts Inst. Technol., Cambridge, A.I. Memo 792, 1984.
[56] S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,"IEEE Trans. Pattern Anal. Machine Intel., vol. 6, pp. 721-741, 1984.
[57] T. Poggio and V. Torre, "Ill-posed problems and regularization analysis in early vision," Memo 773, MIT AI Lab., Apr. 1984.
[58] J. M. Hutchinson and C. Koch, "Simple analog and hybrid networks for surface interpolation," inAIP Conf. Proc. 151 Neural Networks for Computing, Snowbird, UT. New York: AIP, 1986, pp. 235-240.
[59] A. Blake and A. Zisserman,Visual Reconstruction. Cambridge, MA: MIT Press, 1987.
[60] R. Courant and D. Hilbert,Methods of Mathematical Physics. New York: Wiley, 1937, vol. 1.
[61] W. F. Ames,Numerical Methods for Partial Differential Equations. New York: Academic, 1977, 2nd ed.

Index Terms:
computerised picture processing; pattern recognition; segmentation; line clustering; image flow velocity field; statistical test; smoothing algorithm; surface reconstruction; computerised pattern recognition; computerised picture processing; statistical analysis
B.G. Schunck, "Image Flow Segmentation and Estimation by Constraint Line Clustering," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, no. 10, pp. 1010-1027, Oct. 1989, doi:10.1109/34.42834
Usage of this product signifies your acceptance of the Terms of Use.