This Article 
 Bibliographic References 
 Add to: 
Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields
December 1993 (vol. 15 no. 12)
pp. 1217-1232

The estimation of dense velocity fields from image sequences is basically an ill-posed problem, primarily because the data only partially constrain the solution. It is rendered especially difficult by the presence of motion boundaries and occlusion regions which are not taken into account by standard regularization approaches. In this paper, the authors present a multimodal approach to the problem of motion estimation in which the computation of visual motion is based on several complementary constraints. It is shown that multiple constraints can provide more accurate flow estimation in a wide range of circumstances. The theoretical framework relies on Bayesian estimation associated with global statistical models, namely, Markov random fields. The constraints introduced here aim to address the following issues: optical flow estimation while preserving motion boundaries, processing of occlusion regions, fusion between gradient and feature-based motion constraint equations. Deterministic relaxation algorithms are used to merge information and to provide a solution to the maximum a posteriori estimation of the unknown dense motion field. The algorithm is well suited to a multiresolution implementation which brings an appreciable speed-up as well as a significant improvement of estimation when large displacements are present in the scene. Experiments on synthetic and real world image sequences are reported.

[1] G. Adiv, "Determining three-dimensional motion and structure from optical flow generated by several moving objects,"IEEE Trans. Pattern Anal. Machine Intell., vol. 7, pp. 384-401, July 1985.
[2] J. K. Aggarwal and N. Nandhakumar, "On the computation of motion from sequences of images--A review,"Proc. IEEE, vol. 76, no. 8, pp. 917-935, 1988.
[3] S. T. Barnard, "Stochastic stereo matching over scale,"Int. J. Comput. Vision, vol. 3, pp. 17-32, 1989.
[4] J. Bergen, P. Burt, R. Hingorani, and S. Peleg, "Computing two motions from three frames," inProc. 3rd Int. Conf. Comput. Vision, Osaka, Dec. 1990, pp. 27-32.
[5] J. Besag, "On the statistical analysis of dirty pictures,"J. Roy. Statist. Soc., vol. 48, ser. B, no 3, pp. 259-302, 1986.
[6] M. J. Black and P. Anandan, "A model for the detection of motion over time," inProc. Third Int. Conf. Comput. Vision, 1990, pp. 33-37.
[7] P. Bouthemy, "A maximum-likelihood framework for determining moving edges,"IEEE Trans. Pattern Anal. Machine Intell., vol. 11, no. 5, pp. 499-511, May 1989.
[8] P. Bouthemy and E. François, "Motion segmentation and qualitative dynamic scene analysis from an image sequence,"Int. J. Comput. Vision, vol. 10, no. 2, pp. 157-182, 1993.
[9] P. Bouthemy and P. Lalande, "Detection and tracking of moving objects based on a statistical regularization method in space and time," inProc. First European Conf. Comput. Vision, Antibes, France, Apr. 1990, pp. 307-311.
[10] R. Deriche, "Using Canny's criteria to derive a recursively implemented optimal edge detector,"Int. J. Comput. Vision, pp. 167-187, 1987.
[11] W. Enkelmann, "Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences,"Comput. Vision Graphics Image Processing, vol. 43, pp. 150-177, 1988.
[12] E. B. Gamble and T. Poggio, "Visual integration and detection of discontinuities: The key role of intensity edges," Artificial Intell. Lab., Massachusetts Inst. Technol., AI Memo 970, Oct. 1987.
[13] D. Geman, S. Geman, C. Graffigne, and D. Pong, "Boundary detection by constrained optimization,"IEEE Trans. Pattern Anal. Machine Intell., vol. 12, no. 7, pp. 609-628, July 1990.
[14] S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions and the bayesian restoration of images,"IEEE Trans. Pattern Anal. Machine Intell., vol. 6, no. 6, pp. 721-741, Nov. 1984.
[15] F. Heitz and P. Bouthemy, "Multimodal motion estimation and segmentation using Markov random fields," inProc. IEEE Int. Conf. Patt. Recogn., June 1990, pp. 378-383.
[16] F. Heitz and P. Bouthemy, "Multimodal estimation of discontinuous optical flow using Markov random fields," Tech. Rep. 1367, INRIA-Rennes, Jan. 1991.
[17] F. Heitz, P. Perez, and P. Bouthemy, "Multiscale minimization of global energy functions in some visual recovery problems,"CVGIP: Image Understanding, accepted for publication, 1993.
[18] B. K. P. Horn and B. G. Schunck, "Determining optical flow,"Artificial Intell., vol. 17, pp. 185-203, 1981.
[19] J. Hutchinson, C. Koch, J. Luo, and C. Mead, "Computing motion using analog and binary resistive networks,"Comput., vol. 21, pp. 52-63, Mar. 1988.
[20] J.K. Kearny, W.B. Thompson, and D.L. Boley, "Optical flow estimation: An error analysis of gradient-based methods with local optimization,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-9, pp. 229-244, Mar. 1987.
[21] J. Konrad and E. Dubois, "Estimation of image motion fields: Bayesian formulation and stochastic solution," inProc. IEEE Int. Conf. Acoust. Speech Signal Processing, Apr. 1988, pp. 1072-1075.
[22] J. Konrad and E. Dubois, "Bayesian estimation of motion vector fields,"IEEE Trans. Pattern Anal. Machine Intell., vol. 14, no. 9, pp. 910-927, 1992.
[23] J. J. Little and W. E. Gillett, "Direct evidence for occlusion in stereo and motion," inProc. First European Conf. Comput. Vision, Antibes, France, Apr. 1990, pp. 336-340.
[24] A. Mitiche, Y. F. Wang, and J. K. Aggarwal, "Experiments in computing optical flow with the gradient-based, multiconstraint methods,"Patt. Recogn., vol. 20, no. 2, 1987.
[25] D. W. Murray and B. F. Buxton, "Scene segmentation from visual motion using global optimization,"IEEE Trans. Patt. Analy. Machine Intell., vol. PAMI-9, pp. 161-180, 1987.
[26] Y.C. Lee et al., "Internal Thermal Resistance of a Multi-Chip Packaging Design for VLSI Based Systems,"IEEE Trans. Components, Hybrids, and Manufacturing Technology, Vol. 12, No. 2, June 1989, pp. 163- 169.
[27] H.-H. Nagel and W. Enkelmann, "An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-8, no. 5, pp. 565-593, Sept. 1986.
[28] S. Pele and H. Rom, "Motion based segmentation," inProc. Int. Conf. Patt. Recogn.(Atlantic City, NJ), June 1990, pp. 109-113.
[29] B. G. Schunck, "Image flow segmentation and estimation by constraint line clustering,"IEEE Trans. Pattern Anal. Machine Intell., vol. 11, no. 10, pp. 1010-1027, Oct. 1989.
[30] M. Shizawa and K. Mase, "Simultaneous multiple optical flow estimation," inProc. Int. Conf. Patt. Recogn. (Atlantic City, NJ), June 1990, pp. 274-278.
[31] A. Singh, "An estimation-theoretic framework for image flow computation," inProc. 3rd Int. Conf. Comput. Vision, Osaka, Dec. 1990, pp. 168-177.
[32] A. Spoerri and S. Ullman, "The early detection of motion boundaries," inProc. First Int. Conf. Comput. Vision, London, U.K., June 1987, pp. 209-218.
[33] D. Terzopoulos, "Image analysis using multigrid relaxation methods,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, pp. 129-139, Mar. 1986.
[34] A. Verri and T. Poggio, "Motion field and optical flow: Qualitative properties,"IEEE Trans. Pattern Anal. Machine Intell., vol. 11, no. 5, pp. 490-498, May 1989.
[35] K. Wohn and A. Waxman, "The analytic structure of image flows: Segmentation,"Comput. Vision Graphics Image Proc., vol. 49, pp. 127-151, Feb. 1990.

Index Terms:
multimodal estimation; discontinuous optical flow; Markov random fields; dense velocity fields; ill-posed problem; motion boundaries; occlusion regions; motion estimation; visual motion; flow estimation; Bayesian estimation; global statistical models; gradient-based motion constraint equations; feature-based motion constraint equations; deterministic relaxation algorithms; real world image sequences; synthetic image sequences; Bayes methods; image sequences; Markov processes; motion estimation; statistics
F. Heitz, P. Bouthemy, "Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 12, pp. 1217-1232, Dec. 1993, doi:10.1109/34.250841
Usage of this product signifies your acceptance of the Terms of Use.