This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Dynamic Shape and Appearance Models
December 2006 (vol. 28 no. 12)
pp. 2006-2019
We propose a model of the joint variation of shape and appearance of portions of an image sequence. The model is conditionally linear, and can be thought of as an extension of active appearance models to exploit the temporal correlation of adjacent image frames. Inference of the model parameters can be performed efficiently using established numerical optimization techniques borrowed from finite-element analysis and system identification techniques.

[1] K.S. Arun and S.Y. Kung, “Balanced Approximation of Stochastic Systems,” SIAM J. Matrix Analysis and Applications, vol. 11, no. 1, pp. 42-68, 1990.
[2] S. Baker, R. Gross, and I. Matthews, “Lucas-Kanade 20 Years On: A Unifying Framework: Part 4.” Technical Report CMU-RI-TR-04-14, CMU Robotics Inst., Pittsburgh, Penn., Feb. 2004.
[3] S. Baker and I. Matthews, “Equivalence and Efficiency of Image Alignment Algorithms,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 1090-1097, 2001.
[4] S. Baker, I. Matthews, and J. Schneider, “Automatic Construction of Active Appearance Models as an Image Coding Problem,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp.1380-1384, Oct. 2004.
[5] N. Campbell, C. Dalton, D. Gibson, and B. Thomas, “Practical Generation of Video Textures Using the Autoregressive Process,” Proc. British Machine Vision Conf., pp. 434-443, 2002.
[6] T.K. Carne, “The Geometry of Shape Spaces,” Proc. London Math. Soc., vol. 3, no. 61, pp. 407-432, 1990.
[7] T.F. Cootes, G.J. Edwards, and C.J. Taylor, “Active Appearance Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, June 2001.
[8] T.F. Cootes, S. Marsland, C.J. Twining, K. Smith, and C.J. Taylor, “Groupwise Diffeomorphic Non-Rigid Registration for Automatic Model Building,” Proc. European Conf. Computer Vision, pp. 316-327, 2004.
[9] G. Doretto, A. Chiuso, Y.N. Wu, and S. Soatto, “Dynamic Textures,” Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, 2003.
[10] G. Doretto, D. Cremers, P. Favaro, and S. Soatto, “Dynamic Texture Segmentation,” Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1236-1242, 2003.
[11] G. Doretto, E. Jones, and S. Soatto, “Spatially Homogeneous Dynamic Textures,” Proc. European Conf. Computer Vision, vol. 2, pp. 591-602, 2004.
[12] A.W. Fitzgibbon, “Stochastic Rigidity: Image Registration for Nowhere-Static Scenes,” Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 662-669, 2001.
[13] T.J.R. Hughes, The Finite Element Method—Linear Static and Dynamic Finite Element Analysis. Dover Publications, Inc., 2000.
[14] J.D. Jackson, A.J. Yezzi, and S. Soatto, “Dynamic Shape and Appearance Modeling via Moving and Deforming Layers,” Proc. Workshop Energy Minimization in Computer Vision and Pattern Recognition, pp. 427-448, 2005.
[15] H. Jin, S. Soatto, and A.J. Yezzi, “Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance,” Int'l J. Computer Vision, vol. 63, no. 3, pp. 175-189, 2005.
[16] Y. Ma, S. Soatto, J. Kosecká, and S.S. Sastry, An Invitation to 3D Vision: From Images to Geometric Models. Springer-Verlag, 2004.
[17] J.R. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics, second ed. John Wiley and Sons Inc., 1999.
[18] I. Matthews and S. Baker, “Active Appearance Models Revised,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 135-164, 2004.
[19] P. Saisan, G. Doretto, Y.N. Wu, and S. Soatto, “Dynamic Texture Recognition,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 58-63, 2001.
[20] A. Schödl, R. Szeliski, D.H. Salesin, and I. Essa, “Video Textures,” Proc. SIGGRAPH, pp. 489-498, 2000.
[21] B. Schölkopf and A. Smola, Learning with Kernels: SVM, Regularization, Optimization, and Beyond. MIT Press, 2002.
[22] S. Soatto, G. Doretto, and Y.N. Wu, “Dynamic Textures,” Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 439-446, 2001.
[23] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J.Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[24] P. Van Overschee and B. De Moor, “Subspace Algorithms for the Stochastic Identification Problem,” Automatica, vol. 29, no. 3, pp.649-660, 1993.
[25] P. Van Overschee and B. De Moor, “N4SID: Subspace Algorithms for the Identification of Combined Deterministic-Stochastic Systems,” Automatica, vol. 30, no. 1, pp. 75-93, 1994.
[26] T. Vetter and T. Poggio, “Linear Object Classes and Image Synthesis from a Single Example Image,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 733-742, July 1997.
[27] J.Y.A. Wang and E.H. Adelson, “Representing Moving Images with Layers,” IEEE Trans. Image Processing, vol. 3, no. 5, pp. 625-638, 1994.
[28] Y.Z. Wang and S.C. Zhu, “A Generative Method for Textured Motion: Analysis and Synthesis,” Proc. European Conf. Computer Vision, pp. 583-598, 2002.

Index Terms:
Active appearance models, linear dynamical systems, video analysis, image motion, dynamic textures.
Citation:
Gianfranco Doretto, Stefano Soatto, "Dynamic Shape and Appearance Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2006-2019, Dec. 2006, doi:10.1109/TPAMI.2006.243
Usage of this product signifies your acceptance of the Terms of Use.