This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Adjusting Shape Parameters Using Model-Based Optical Flow Residuals
June 2002 (vol. 24 no. 6)
pp. 814-823

We present a method for estimating the shape of a deformable model using the least-squares residuals from a model-based optical flow computation. This method is built on top of an estimation framework using optical flow and image features, where optical flow affects only the motion parameters of the model. Using the results of this computation, our new method adjusts all of the parameters so that the residuals from the flow computation are minimized. We present face tracking experiments that demonstrate that this method obtains a better estimate of shape compared to related frameworks.

[1] J. Barron, “Computing Motion and Structure from Noisy Time-Varying Image Velocity Information,” Technical Report RBCV-TR-88-24, Dept. of Computer Science, Univ. of Toronto, 1988.
[2] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hiercharchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
[3] M.J. Black and Y. Yacoob, "Tracking and Recognizing Rigid and Non-Rigid Facial Motions Using Local Parametric Model of Image Motion," Proc. Int'l Conf. Computer Vision, pp. 374-381,Cambridge, Mass., 1995.
[4] A. Bryson and Y. Ho, Applied Optimal Control. Halsted Press, 1975.
[5] J. Cassell, “Embodied Conversational Interface Agents,” Comm. ACM, vol. 43, no. 4, pp. 70-78, 2000.
[6] T.F. Cootes, G.J. Edwards, and C.J. Taylor, “Active Appearance Models,” Proc. European Conf. Computer Vision, pp. 484-498, June 1998.
[7] D. DeCarlo and D. Metaxas, Optical Flow Constraints on Deformable Models with Applications to Face Tracking Int'l J. Computer Vision, vol. 38, no. 2, pp. 99-127, July 2000.
[8] H.F. Durrant-Whyte, “Consistent Integration and Propagation of Disparate Sensor Observations,” Int'l J. Robotics Research, vol. 6, no. 3, pp. 3-24, 1987.
[9] P. Fua, “Regularized Bundle-Adjustment to Model Heads from Image Sequences without Calibration Data,” Int'l J. Computer Vision, vol. 38, no. 2, July 2000.
[10] P. Gill, Practical Optimization. Academic Press, 1982.
[11] J. Heinzmann and A. Zelinsky, “Building Human-Friendly Robot Systems,” Proc. Int'l Symp. Robotics Research (ISRR' 99), 1999.
[12] B.K. Horn, Robot Vision. Cambridge, Mass.: MIT Press, 1986.
[13] T. Huang and A. Netravali,“Motion and structure from feature correspondences: A review,” Proc. IEEE, vol. 82, pp. 252-268, Feb. 1994.
[14] T. Jebara, A. Azarbayejani, and A.P. Pentland, “3D Structure from 2D Motion,” IEEE Signal Processing Magazine, vol. 16, 1999.
[15] R. Koch, "Dynamic 3D Scene Analysis through Synthesis Feedback Control," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 15, No. 6, June 1993, pp. 556-568.
[16] H. Li, P. Roivainen, and R. Forchheimer, "3D Motion Estimation in Model-Based Facial Image Coding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 545-555, June 1993.
[17] R. McKendall and M. Mintz, “Data Fusion Techniques Using Robust Statistics,” Data Fusion in Robotics and Machine Intelligence, M. A. Abidi and R.C. Gonzalez, eds., 1992.
[18] D. Metaxas and D. Terzopoulos, “Shape and Non-Rigid Motion Estimation through Physics-Based Synthesis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 580-591, June 1993.
[19] Y. Moses, D. Reynard, and A. Blake, “Robust Real Time Tracking and Classificiation of Facial Expressions,” Proc. Int'l Conf. Computer Vision '95, pp. 296-301, 1995.
[20] S. Negahdaripour, “Revised Definition of Optical Flow: Integration of Radiometric and Geometric Clues for Dynamic Scene Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 9, pp. 961–979, Sept. 1998.
[21] S. Negahdaripour and B.K.P. Horn, “Direct Passive Navigation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 1, pp. 168-176, Jan. 1987.
[22] A. Netravali and J. Salz, “Algorithms for Estimation of Three-Dimensional Motion,” AT&T Technical J., vol. 64, pp. 335-346, 1985.
[23] A. Pentland and S. Sclaroff, "Closed-Form Solutions for Physically-Based Shape Modeling and Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 715-729, July 1991.
[24] J. Rehg, M. Loughlin, and K. Waters, “Vision for a Smart Kiosk,” Computer Vision and Pattern Recognition, 1997.
[25] D. Reynard, A. Wildenberg, A. Blake, and J. Marchant, “Learning Dynamics of Complex Motions from Image Sequences,” Proc. Fourth European Conf. Computer Vision, B.F. Buxton and R. Cipolla, eds., pp. 357-368, Apr. 1996.
[26] G. Strang, Linear Algebra and Its Applications. Harcourt, Brace, Jova novich, 1988.
[27] H. Tao et al., "Compression of MPEG-4 Facial Animation Parameters for Transmission of Talking Heads," IEEE Trans. Circuits and Systems for Video Technology, vol. 9, no. 3, Mar. 1999, pp. 264-276.
[28] D. Terzopoulos and K. Waters, "Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 569-579, 1993.
[29] D. Terzopolous, A. Witkin, and M. Kass, "Constraints on deformable models: Recovering 3D shape and nonrigid motion, AI, no. 36, pp. 91-123, 1988.

Index Terms:
Nonrigid shape and motion estimation, model-based optical flow, deformable models.
Citation:
Douglas DeCarlo, Dimitris Metaxas, "Adjusting Shape Parameters Using Model-Based Optical Flow Residuals," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 814-823, June 2002, doi:10.1109/TPAMI.2002.1008387
Usage of this product signifies your acceptance of the Terms of Use.