The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - August (2011 vol.33)
pp: 1681-1688
Yilei Xu , Navteq Corp., Chicago
Amit K. Roy-Chowdhury , University of California, Riverside, Riverside
ABSTRACT
Linear and multilinear models (PCA, 3DMM, AAM/ASM, and multilinear tensors) of object shape/appearance have been very popular in computer vision. In this paper, we analyze the applicability of these heuristic models from the fundamental physical laws of object motion and image formation. We prove that under suitable conditions, the image appearance space can be closely approximated to be multilinear, with the illumination and texture subspaces being trilinearly combined with the direct sum of the motion and deformation subspaces. This result provides a physics-based understanding of many of the successes and limitations of the linear and multilinear approaches existing in the computer vision literature, and also identifies some of the conditions under which they are valid. It provides an analytical representation of the image space in terms of different physical factors that affect the image formation process. Numerical analysis of the accuracy of the physics-based models is performed, and tracking results on real data are presented.
INDEX TERMS
Image appearance models, theoretical analysis, multilinear, deformation, face tracking.
CITATION
Yilei Xu, Amit K. Roy-Chowdhury, "A Physics-Based Analysis of Image Appearance Models", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 8, pp. 1681-1688, August 2011, doi:10.1109/TPAMI.2010.216
REFERENCES
[1] R. Basri and D. Jacobs, "Lambertian Reflectance and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[2] P. Belhumeur and D. Kriegman, "What Is the Set of Images of an Object under All Possible Lighting Conditions?" Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1996.
[3] V. Blanz and T. Vetter, "Face Recognition Based on Fitting a 3D Morphable Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[4] T. Broida and R. Chellappa, "Estimating the Kinematics and Structure of a Rigid Object from a Sequence of Monocular Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 6, pp. 497-513, June 1991.
[5] T. Cootes, G. Edwards, and C. Taylor, "Active Appearance Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, June 2001.
[6] R. Costantini, L. Sbaiz, and S. Ssstrunk, "Higher Order SVD Analysis for Dynamic Texture Synthesis," IEEE Trans. Image Processing, vol. 17, no. 1, pp. 42-52, Jan. 2008.
[7] G. Doretto, A. Chiuso, Y. Wu, and S. Soatto, "Dynamic Textures," Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, 2003.
[8] G. Doretto and S. Soatto, "Dynamic Shape and Appearance Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2006-2019, Dec. 2006.
[9] R. Frankot and R. Challappa, "A Method for Enforcing Integrability in Shape from Shading Algorithms," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp. 439-451, July 1988.
[10] B. Horn and M. Brooks, "The Variational Approach to Shape from Shading," Computer Vision, Graphics, and Image Processing, vol. 33, no. 2, pp. 174-208, 1986.
[11] B. Horn and B. Schunck, "Determining Optical Flow," Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[12] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active Contour Models," Int'l J. Computer Vision, pp. 321-331, 1988.
[13] L.D. Landau and E.M. Lifschitz, Mechanics, third ed., Pergamon Press, 1976.
[14] L.D. Lathauwer, B.D. Moor, and J. Vandewalle, "A Multillinear Singular Value Decomposition," SIAM J. Matrix Analysis and Applications, vol. 21, no. 4, pp. 1253-1278, 2000.
[15] C. Lee and A. Elgammal, "Nonlinear Shape and Appearance Models for Facial Expression Analysis and Synthesis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 313-320, 2003.
[16] K. Lee, J. Ho, M. Yang, and D. Kriegman, "Video-Based Face Recognition Using Probabilistic Appearance Manifolds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 313-320, 2003.
[17] I. Matthews and S. Baker, "Active Appearance Models Revisited," Int'l J. Computer Vision, vol. 60, no. 2, pp. 135-164, Nov. 2004.
[18] Y. Moses, "Face Recognition: Generalization to Novel Images," PhD thesis, Weizmann Inst. of Sciences, 1993.
[19] J. Oliensis and P. Dupuis, "Direct Method for Reconstructing Shape from Shading," Proc. SPIE Conf. 1570 on Geometric Methods in Computer Vision, 1991.
[20] R. Ramamoorthi and P. Hanrahan, "On the Relationship between Radiance and Irradiance: Determining the Illumination from Images of a Convex Lambertian Object," J. Optical Soc. of Am., vol. 18, no. 10, Oct. 2001.
[21] S. Roweis and L. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, no. 5500, pp. 2323-2326, Dec. 2000.
[22] G. Sapiro, Geometric Partial Differential Equations and Image Analysis. Cambridge Univ. Press, 2001.
[23] A. Shashua, "On Photometric Issues in 3D Visual Recognition from a Single 2D Image," Int'l J. Computer Vision, vol. 21, nos. 1/2, pp. 99-122, 1997.
[24] S. Shirdhonkar and D. Jacobs, "Non-Negative Lighting and Specular Object Recognition," Proc. 10th IEEE Int'l Conf. Computer Vision, vol. I, pp. 1323-1330, Oct. 2005.
[25] J.B. Tenenbaum, V. de Silva, and J.C. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, no. 5500, pp. 2319-2323, Dec. 2000.
[26] J.B. Tenenbaum and W.T. Freeman, "Separating Style and Content with Bilinear Models," Neural Computation, vol. 12, no. 6, pp. 1247-1283, 2000.
[27] C. Tomasi and T. Kanade, "Shape and Motion from Image Streams under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[28] L. Torresani and C. Bregler, "Space-Time Tracking," Proc. European Conf. Computer Vision, 2002.
[29] M. Vasilescu and D. Terzopoulos, "Multilinear Independent Components Analysis," IEEE CS Conf. Computer Vision and Pattern Recognition, June 2005.
[30] A. Veeraraghavan, A. Roy-Chowdhury, and R. Chellappa, "Matching Shape Sequences in Video with Applications in Human Motion Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1896-1909, Dec. 2005.
[31] Y. Xu and A. Roy-Chowdhury, "Integrating Motion, Illumination and Structure in Video Sequences, with Applications in Illumination-Invariant Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 793-806, May 2007.
[32] Y. Xu and A. Roy-Chowdhury, "A Theoretical Analysis of Linear and Multilinear Moels of Image Appearance," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[33] Y. Xu and A. Roy-Chowdhury, "Learning a Geometry Integrated Image Appearance Manifold from a Small Training Set," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[34] H. Yang, M. Pollefeys, G. Welch, J.-M. Frahm, and A. Ilie, "Differential Camera Tracking through Linearizing the Local Appearance Manifold," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007.
[35] L. Zhang, B. Curless, A. Hertzmann, and S. Seitz, "Shape and Motion under Varying Illumination: Unifying Structure from Motion, Photometric Stereo, and Multiview Stereo," Proc. IEEE Int'l Conf. Computer Vision, 2003.
27 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool