This Article 
 Bibliographic References 
 Add to: 
Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations
October 1996 (vol. 18 no. 10)
pp. 972-986

Abstract—This paper presents a new model for estimating optical flow based on the motion of planar regions plus local deformations. The approach exploits brightness information to organize and constrain the interpretation of the motion by using segmented regions of piecewise smooth brightness to hypothesize planar regions in the scene. Parametric flow models are estimated in these regions in a two step process which first computes a coarse fit and estimates the appropriate parameterization of the motion of the region (two, six, or eight parameters). The initial fit is refined using a generalization of the standard area-based regression approaches. Since the assumption of planarity is likely to be violated, we allow local deformations from the planar assumption in the same spirit as physically-based approaches which model shape using coarse parametric models plus local deformations. This parametric+deformation model exploits the strong constraints of parametric approaches while retaining the adaptive nature of regularization approaches. Experimental results on a variety of images indicate that the parametric+deformation model produces accurate flow estimates while the incorporation of brightness segmentation provides precise localization of motion boundaries.

[1] G. Adiv, "Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 4, pp. 384-401, July 1985.
[2] P. Anandan, "A Computational Framework and an Algorithm for the Measurement of Visual Motion," Int'l J. Computer Vision, vol. 2, pp. 283-310, 1989.
[3] S. Ayer and H. Sawhney, "Layered Representation of Motion Video Using Robust Maximum-Likelihood Estimation of Mixture Models and mdl Encoding," Int'l Conf. Computer Vision, pp. 777-784,Cambridge, Mass., June 1995.
[4] S. Ayer, P. Schroeter, and J. Bigün, "Segmentation of Moving Objects by Robust Motion Parameter Estimation Over Multiple Frames," Proc. European Conf. Computer Vision, ECCV-94, J. Eklundh, ed., vol. 801of LNCS-Series, pp. 317-327,Stockholm: Springer-Verlag, 1994..
[5] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43–77, 1994.
[6] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hiercharchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
[7] J.R. Bergen, P.J. Burt, R. Hingorani, and S. Peleg, "A Three-Frame Algorithm for Estimating Two-Component Image Motion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, pp. 886-895, Sept. 1992.
[8] P.J. Besl and R.C. Jain,“Segmentation through variable-order surface fitting,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 2, pp. 167-191, Mar. 1988.
[9] M.J. Black, "Combining Intensity and Motion for Incremental Segmentation and Tracking Over Long Image Sequences," Proc. Second European Conf. Computer Vision, ECCV-92, G. Sandini, ed., vol. 588of LNCS-Series, pp. 485-493. Springer-Verlag, May 1992.
[10] M.J. Black, "Recursive Non-Linear Estimation of Discontinuous Flow Fields," Proc. European Conf. Computer Vision, ECCV-94, J. Eklundh, ed., vol. 800of LNCS-Series, pp. 138-145,Stockholm: Springer-Verlag, 1994..
[11] M.J. Black and P. Anandan,“A framework for the robust estimation of optical flow,” Proc. Int’l Conf. on Computer Vision, ICCV-93, Berlin, pp. 231-236, May 1993.
[12] M. Black and P. Anandan, The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields J. Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
[13] M.J. Black and A. Rangarajan, “On the Unification of Line Processes, Outlier Rejection, and Robust Statistics with Applications in Early Vision,” Int'l J. Computer Vision, vol. 19, no. 1, pp. 57-91, 1996.
[14] T. Darrell and A. Pentland, "Cooperative Robust Estimation Using Layers of Support," Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 5, pp. 474-487, May 1995.
[15] M. Dubuisson and A.K. Jain, "Object Contour Extraction Using Color and Motion," Proc. Computer Vision and Pattern Recognition, CVPR-93, pp. 471-476,New York, June 1993.
[16] C.L. Fennema and W.B. Thompson, "Velocity Determination in Scenes Containing Several Moving Objects," Computer Graphics and Image Processing, vol. 9, pp. 301-315, 1979.
[17] D.J. Fleet and A.D. Jepson, “Computation of Component Image Velocity from Local Phase Information,” Int'l J. Computer Vision, vol. 5, no. 1, pp. 77-104, 1990.
[18] D. Geman,S. Geman,C. Graffigne,, and P. Dong,“Boundary detection by constrained optimization,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 609-628, July 1990.
[19] F.R. Hampel, E.M. Ronchetti, P.J. Rousseeuw, and W.A. Stahel, Robust Statistics: The Approach Based on Influence Functions.New York: John Wiley and Sons, 1986.
[20] F. Heitz and P. Bouthemy, Multimodal Estimation of Discontinuous Optical Flow Using Markov Random Fields IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 12, pp. 1217-1232, Dec. 1993.
[21] B.K.P. Horn and B.G. Schunck, "Determining Optical Flow," Artificial Intelligence, vol. 17, nos. 1-3, pp. 185-203, Aug. 1981.
[22] M. Irani, B. Rousso, and S. Peleg, “Detecting and Tracking Multiple Moving Objects Using Temporal Integration,” Proc. European Conf. Computer Vision, pp. 282-287, May 1992.
[23] A. Jepson and M.J. Black, "Mixture Models for Optical Flow Computation," Partitioning Data Sets: With Applications to Psychology, Vision and Target Tracking, I. Cox, P. Hansen, and B. Julesz, eds., pp. 271-286, DIMACS Workshop, Apr. 1993.Providence, R.I.: AMS Pub.
[24] I.A. Kakadiaris, D. Metaxas, and R. Bajcsy, "Active Part-Decomposition, Shape and Motion Estimations of Articulated Objects: A Physics-Based Approach," Computer Vision and Pattern Recognition, CVPR-94, pp. 980-984,Seattle, 1994.
[25] J. K. Kearney,W. B. Thompson,, and D. L. Boley,“Optical flow estimation: An error analysis of gradient-based methods with local optimization,” Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 2, pp 229-244, 1987.
[26] R. Koch, "Automatic Reconstruction of Buildings from Stereoscopic Image Sequences," EUROGRPHICS '93, vol. 12, no. 3, pp. 339-350, 1993.
[27] R. Kumar, P. Anandan, and K. Hanna, "Shape Recovery from Multiple Views: A Parallax Based Approach," Proc. ARPA Image Understanding Workshop, 1994.
[28] R. Kumar and A.R. Hanson, "Analysis of Different Robust Methods for Pose Refinement," Proc. Int'l Workshop Robust Computer Vision, pp. 167-182,Seattle, Oct. 1990.
[29] A. Leonardis, A. Gupta, and R. Bajcsy, "Segmentation as the Search for the Best Description of the Image in Terms of Primitives," Technical Report MS-CIS-90-30, GRASP LAB 215, May 1990.
[30] G. Li, "Robust Regression," Exploring Data, Tables, Trends and Shapes F. Mosteller and J.W. Tukey, eds. New York: John Wiley and Sons, 1985.
[31] B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. Seventh IJCAI, pp. 674-679,Vancouver, B.C., Canada, 1981.
[32] W. Luo and H. Maitre, “Using Surface Model to Correct and Fit Disparity Data in Stereo Vision,” Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 60–64, 1990.
[33] W.J. MacLean, A.D. Jepson, and R.C. Frecker, "Recovery of Egomotion and Segmentation of Independent Object Motion Using the Em Algorithm," Proc. British Machine Vision Conf.,York, U.K., 1994.
[34] G.J. McLachlan and K.E. Basford, Mixture Models: Inference and Applications to Clustering.New York: Marcel Dekker, 1988.
[35] F. Meyer and P. Bouthemy, “Region-Based Tracking Using Affine Motion Models in Long Image Sequences,” CVGIP: Image Understanding, vol. 60, no. 2, Sept. 1994.
[36] H.-H. Nagel, “On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results,” Artificial Intelligence, vol. 33, pp. 299-324, 1987.
[37] M. Otte and H.-H. Nagel, "Optical Flow Estimation: Advances and Comparisons," Proc. Third European Conf. Computer Vision, Springer-Verlag, New York, 1994, pp. 51-60.
[38] A. Pentland and S. Sclaroff, "Closed-Form Solutions for Physically-Based Shape Modeling and Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 715-729, July 1991.
[39] P. Rousseeuw and A. Leory, Robust Regression and Outlier Detection. Wiley Series in Probability and Statistics, 1987.
[40] H.S. Sawhney, "3D Geometry From Planar Parallax," Proc. CVPR '94, pp. 929-934, 1994.
[41] A. Singh, Optic Flow Computation: A Unified Perspective.Los Alamitos, Calif.: IEEE CS Press, 1992.
[42] S. Sull and N. Ahuja, "Segmentation, Matching and Estimation of Structure and Motion of Textured Piecewise Planar Surfaces," Proc. IEEE Workshop Visual Motion, pp. 274-279,Princeton, N.J., Oct. 1991.
[43] D. Terzopoulos and D. Metaxas, “Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 703-714, July 1991.
[44] W.B. Thompson, "Combining Motion and Contrast for Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 2, pp. 543-549, 1980.
[45] S. Uras, F. Girosi, A. Verri, and V. Torre, "A Computational Approach to Motion Perception," Biological Cybernetics, vol. 60, pp. 79-97, 1989.
[46] J.Y.A. Wang and E.H. Adelson, Representing Moving Images with Layers IEEE Trans. Image Processing, vol. 3, no. 5, pp. 625-638, Sept. 1994.
[47] A. Waxman, "An Image Flow Paradigm," Proc. IEEE Workshop Computer Vision: Representation and Control, pp. 49-55,Annapolis, Md., 1984.
[48] A.M. Waxman, B. Kamgar-Parsi, and M. Subbarao, "Close-Form Solutions to Image Flow Equations for 3D-Structure and Motion," Int'l J. Computer Vision, no. 3, pp. 239-258, 1987.
[49] A.M. Waxman and K. Wohn, "Contour Evolution, Neighbourhood Deformation and Global Image Flow: Planar Surfaces in Motion," Int'l J. Robotics Research, vol. 4, pp. 95-108, 1985.
[50] J. Weber and J. Malik, “Robust Computation of Optical Flow in a Multi-Scale Differential Framework,” Proc. Int'l Conf. Computer Vision, 1993.

Index Terms:
Optical flow, segmentation, robust regression, parameterized flow models, local deformation.
Michael J. Black, Allan D. Jepson, "Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models With Local Deformations," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 10, pp. 972-986, Oct. 1996, doi:10.1109/34.541407
Usage of this product signifies your acceptance of the Terms of Use.