This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Computing Optical Flow with Physical Models of Brightness Variation
June 2001 (vol. 23 no. 6)
pp. 661-673

Abstract—Although most optical flow techniques presume brightness constancy, it is well-known that this constraint is often violated, producing poor estimates of image motion. This paper describes a generalized formulation of optical flow estimation based on models of brightness variations that are caused by time-dependent physical processes. These include changing surface orientation with respect to a directional illuminant, motion of the illuminant, and physical models of heat transport in infrared images. With these models, we simultaneously estimate the 2D image motion and the relevant physical parameters of the brightness change model. The estimation problem is formulated using total least squares (TLS), with confidence bounds on the parameters. Experiments in four domains, with both synthetic and natural inputs, show how this formulation produces superior estimates of the 2D image motion.

[1] P. Anandan, “A Computational Framework and an Algorithm for the Measurement of Visual Motion,” Int'l J. Computer Vision, vol. 2, pp. 283–310, 1989.
[2] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43–77, 1994.
[3] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hiercharchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
[4] M.J. Black and P. Anandan, “Robust Dynamic Motion Estimation Over Time,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 296–302, June 1991.
[5] M.J. Black, D.J. Fleet, and Y. Yacoob, “Robustly Estimating Changes in Image Appearance,” Computer Vision and Image Understanding, vol. 78, no. 1, pp. 8–31, 2000.
[6] N. Cornelius and T. Kanade, “Adapting Optical Flow to Measure Object Motion in Reflectance and X-Ray Image Sequences,” ACM Workshop Motion: Representation and Perception, pp. 50–58, 1983.
[7] B. Duc, “Feature Design: Applications to Motion Analysis and Identity Verification,” PhD thesis,École Polytechnique Fédérale de Lausanne, 1997.
[8] D.J. Fleet, M.J. Black, Y. Yacoob, and A.D. Jepson, “Design and Use of Linear Models for Image Motion Analysis,” Int'l J. Computer Vision, vol. 36, no. 3, pp. 171–193, 2000.
[9] D.J. Fleet and K. Langley, “Computational Analysis of Nonfourier Motion,” Vision Research, vol. 22, pp. 3057–3079, 1994.
[10] D.J. Fleet and K. Langley, “Recursive Filters for Optical Flow,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, pp. 61–67, 1995.
[11] D.J. Fleet and A.D. Jepson, “Computation of Component Image Velocity from Local Phase Information,” Int'l J. Computer Vision, vol. 5, no. 1, pp. 77-104, 1990.
[12] D.J. Fleet and A.D. Jepson, “Stability of Phase Information,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, pp. 1253–1268, 1993.
[13] L. Florac, W. Niessen, and M. Nielsen, “The Intrinsic Structure of Optical Flow Incorporating Measurement Duality,” Int'l J. Computer Vision, vol. 27, no. 3, pp. 263–286, 1998.
[14] G. Hager and P. Belhumeur, “Efficient Region Tracking with Parametric Models of Geometry and Illumination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 10, pp. 1025-1039, Oct. 1998.
[15] H.W. Haussecker, “Physics from IR Sequences: Quantitative Analysis of Transport Models and Parameters of Air-Sea Gas Transfer,” Proc. Intl. Symp. Gas Transfer at Water Surfaces, 2000.
[16] H.W. Haussecker and H. Spies, “Motion,” Handbook of Computer Vision and Applications, B. Jähne, H.W. Haussecker, and P. Geissler, eds., pp. 309–396, 1999.
[17] B.K. Horn and B.G. Schunck, “Determining Optical Flow,” Aritifial Intelligence, vol. 17, pp. 185–204, 1981.
[18] B.K. Horn, Robot Vision. Cambridge, Mass.: MIT Press, 1986.
[19] B. Jähne, H.W. Haussecker, H. Spies, D. Schmundt, and U. Schurr, ”Study of Dynamical Processes with Tensor-Based Spatiotemporal Image Processing Techniques,” Proc. European Conf. Computer Vision, vol. II, H. Burkhardt and B. Neumann, eds., pp. 322–335, 1998.
[20] B. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proc. DARPA Image Understanding Workshop, pp. 121–130, 1981.
[21] W.J. MacLean, “Removal of Translation Bias When Using Subspace Methods,” Proc. IEEE Int'l Conf. Computer Vision, vol. II, pp. 753–758, 1999.
[22] B. Matei and P. Meer, “A General Method for Errors-in-Variables Problems in Computer Vision,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 18-25, June 2000.
[23] J. Michel, N. Nandhakumar, and V. Velten, “Thermophysical Algebraic Invariants from Infrared Imagery for Object Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 1, pp. 41–51, 1997.
[24] M. Mühlich and R. Mester, “The Role of Total Least Squares in Motion Analysis,” Proc. Fifth European Conf. Computer Vision, H. Burkhardt and B. Neumann, eds., vol. 2, pp. 305-321, June 1998.
[25] M. Mühlich and R. Mester, ”A Considerable Improvement in Pure Parameter Estimation Using TLS and Equilibration,” Pattern Recognition Letters, 2000.
[26] N. Mukawa, “Estimation of Shape, Reflection Coefficients and Illumination Direction from Image Sequences,” Proc. IEEE Int'l Conf. Computer Vision, pp. 507–512, 1990.
[27] D.W. Murray and B.F. Buxton, Experiments in the Machine Interpretation of Visual Motion. Cambridge, Mass.: MIT Press, 1990.
[28] H.-H. Nagel, “On a Constraint Equation for the Estimation of Displacement Rates in Image Sequences,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 1, pp. 13–30, 1989.
[29] H.-H. Nagel, “Extending the 'Oriented Smoothness Constraint' into the Temporal Domain and the Estimation of Derivatives of Optical Flow,” Proc. European Conf. Computer Vision, O. Faugeras, ed., pp. 139–148, 1990.
[30] H.-H. Nagel, “Optical-Flow Estimation and the Interaction between Measurement Errors at Adjacent Pixel Positions,” Int'l J. Computer Vision, vol. 15, no. 3, pp. 271–288, July 1995.
[31] S.K. Nayar and G. Narasimhan, “Vision in Bad Weather,” Proc. IEEE Int'l Conf. Computer Vision, vol. II, pp. 820–827, 1999.
[32] S. Negahdaripour, “Revised Definition of Optical Flow: Integration of Radiometric and Geometric Clues for Dynamic Scene Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 9, pp. 961–979, Sept. 1998.
[33] O. Nestares, D.J. Fleet, and D.J. Heeger, “Likelihood Functions and Confidence Bounds for Total-Least-Squares Estimation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 523–530, 2000.
[34] N. Ohta, “Optical Flow Detection Using a General Noise Model,” IEICE Trans. Information and Systems, vol. 79, no. 7, pp. 951–957, 1996.
[35] H. Scharr, S. Körkel, and B. Jähne, ”Numerische Isotropieoptimierung von FIR-Filtern mittels Querglättung,” Mustererkennung 1997, E. Paulus and F.M. Wahl, eds., pp. 367–374, 1997.
[36] G.W. Stewart, “Errors in Variables for Numerical Analysts,” Recent Advances in Total Least Squares Techniques, S. Van Huffel, ed., pp. 3–10, 1997.
[37] S. Van Huffel and S. Vandewalle, The Total Least Squares Problem: Computational Aspects and Analysis. Philadelphia, Penn.: SIAM Press, 1991.
[38] A. Verri and T. Poggio, "Motion field and optical flow: Qualitative properties," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 5, pp. 490-498, May 1989.
[39] S. Wang, V. Markandey, and A. Reid, “Total Least Squares Fitting Spatiotemporal Derivatives to Smooth Optical Flow Fields,” Proc. SPIE: Signal and Data Processing of Small Targets, pp. 42–55, 1992.
[40] J. Weber and J. Malik, “Robust Computation of Optical Flow in a Multiscale Differential Framework,” Int'l J. Computer Vision, vol. 15, no. 1, pp. 67–81, 1995.
[41] R.P. Wildes, M.J. Amabile, A. Lanzillotto, and T. Leu, “Recovering Estimates of Fluid Flow from Image Sequence Data,” Computer Vision and Image Understanding, vol. 80, pp. 246–266, 2000.

Index Terms:
Optical flow, physics-based brightness variation, total least squares.
Citation:
Horst W. Haussecker, David J. Fleet, "Computing Optical Flow with Physical Models of Brightness Variation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 661-673, June 2001, doi:10.1109/34.927465
Usage of this product signifies your acceptance of the Terms of Use.