The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - November (2011 vol.33)
pp: 2273-2286
Pavan Turaga , University of Maryland, College Park
Ashok Veeraraghavan , Mitsubishi Electrical Research Labs, Cambridge
Anuj Srivastava , Florida State University, Tallahassee
Rama Chellappa , University of Maryland, College Park
ABSTRACT
In this paper, we examine image and video-based recognition applications where the underlying models have a special structure—the linear subspace structure. We discuss how commonly used parametric models for videos and image sets can be described using the unified framework of Grassmann and Stiefel manifolds. We first show that the parameters of linear dynamic models are finite-dimensional linear subspaces of appropriate dimensions. Unordered image sets as samples from a finite-dimensional linear subspace naturally fall under this framework. We show that an inference over subspaces can be naturally cast as an inference problem on the Grassmann manifold. To perform recognition using subspace-based models, we need tools from the Riemannian geometry of the Grassmann manifold. This involves a study of the geometric properties of the space, appropriate definitions of Riemannian metrics, and definition of geodesics. Further, we derive statistical modeling of inter and intraclass variations that respect the geometry of the space. We apply techniques such as intrinsic and extrinsic statistics to enable maximum-likelihood classification. We also provide algorithms for unsupervised clustering derived from the geometry of the manifold. Finally, we demonstrate the improved performance of these methods in a wide variety of vision applications such as activity recognition, video-based face recognition, object recognition from image sets, and activity-based video clustering.
INDEX TERMS
Image and video models, feature representation, statistical models, manifolds, Stiefel, Grassmann.
CITATION
Pavan Turaga, Ashok Veeraraghavan, Anuj Srivastava, Rama Chellappa, "Statistical Computations on Grassmann and Stiefel Manifolds for Image and Video-Based Recognition", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 11, pp. 2273-2286, November 2011, doi:10.1109/TPAMI.2011.52
REFERENCES
[1] P. Turaga, A. Veeraraghavan, and R. Chellappa, "Statistical Analysis on Stiefel and Grassmann Manifolds with Applications in Computer Vision," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2008.
[2] G. Doretto, A. Chiuso, Y.N. Wu, and S. Soatto, "Dynamic Textures," Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, Feb. 2003.
[3] A.B. Chan and N. Vasconcelos, "Modeling, Clustering, and Segmenting Video with Mixtures of Dynamic Textures," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 909-926, May 2008.
[4] A. Bissacco, A. Chiuso, Y. Ma, and S. Soatto, "Recognition of Human Gaits," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 52-57, Dec. 2001.
[5] A. Veeraraghavan, A. Roy-Chowdhury, and R. Chellappa, "Matching Shape Sequences in Video with an Application to Human Movement Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1896-1909, Dec. 2005.
[6] G. Aggarwal, A. Roy-Chowdhury, and R. Chellappa, "A System Identification Approach for Video-Based Face Recognition," Proc. Int'l Conf. Pattern Recognition, Aug. 2004.
[7] C.R. Goodall and K.V. Mardia, "Projective Shape Analysis," J. Computational and Graphical Statistics, vol. 8, no. 2, pp. 143-168, June 1999.
[8] V. Patrangenaru and K.V. Mardia, "Affine Shape Analysis and Image Analysis," Proc. 22nd Leeds Ann. Statistics Research Workshop, July 2003.
[9] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, "From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[10] A. Edelman, T.A. Arias, and S.T. Smith, "The Geometry of Algorithms with Orthogonality Constraints," SIAM J. Matrix Analysis and Application, vol. 20, no. 2, pp. 303-353, Apr. 1999.
[11] P.-A. Absil, R. Mahony, and R. Sepulchre, "Riemannian Geometry of Grassmann Manifolds with a View on Algorithmic Computation," Acta Applicandae Mathematicae, vol. 80, no. 2, pp. 199-220, Jan. 2004.
[12] D. Lin, S. Yan, and X. Tang, "Pursuing Informative Projection on Grassmann Manifold," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 1727-1734, June 2006.
[13] E. Begelfor and M. Werman, "Affine Invariance Revisited," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2087-2094, June 2006.
[14] J. Hamm and D.D. Lee, "Grassmann Discriminant Analysis: A Unifying View on Subspace-Based Learning," Proc. Int'l Conf. Machine Learning, pp. 376-383, June 2008.
[15] Y.M. Lui and J.R. Beveridge, "Grassmann Registration Manifolds for Face Recognition," Proc. European Conf. Computer Vision, pp. 44-57, Oct. 2008.
[16] A. Srivastava and E. Klassen, "Bayesian and Geometric Subspace Tracking," Advances in Applied Probability, vol. 36, no. 1, pp. 43-56, Mar. 2004.
[17] Y.M. Lui, J.R. Beveridge, and M. Kirby, "Canonical Stiefel Quotient and Its Application to Generic Face Recognition in Illumination Spaces," Proc. IEEE Third Int'l Conf. Biometrics: Theory, Applications, and Systems, Aug. 2009.
[18] W.M. Boothby, An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 1975.
[19] R. Bhattacharya and V. Patrangenaru, "Large Sample Theory of Intrinsic and Extrinsic Sample Means on Manifolds-I," Annals of Statistics, vol. 31, no. 1, pp. 1-29, 2003.
[20] B. Pelletier, "Kernel Density Estimation on Riemannian Manifolds," Statistics and Probability Letters, vol. 73, no. 3, pp. 297-304, July 2005.
[21] X. Pennec, "Intrinsic Statistics on Riemannian Manifolds: Basic Tools for Geometric Measurements," J. Math. Imaging and Vision, vol. 25, no. 1, pp. 127-154, July 2006.
[22] P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization Algorithms on Matrix Manifolds. Princeton Univ. Press, 2008.
[23] Y. Chikuse, Statistics on Special Manifolds: Lecture Notes in Statistics. Springer, 2003.
[24] O. Tuzel, F. Porikli, and P. Meer, "Region Covariance: A Fast Descriptor for Detection and Classification," Proc. European Conf. Computer Vision, pp. 589-600, May 2006.
[25] O. Tuzel, F. Porikli, and P. Meer, "Pedestrian Detection via Classification on Riemannian Manifolds," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1713-1727, Oct. 2008.
[26] F. Porikli, O. Tuzel, and P. Meer, "Covariance Tracking Using Model Update Based on Lie Algebra," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 728-735, June 2006.
[27] R. Subbarao and P. Meer, "Nonlinear Mean Shift for Clustering over Analytic Manifolds," Int'l J. Computer Vision, vol. 84, no. 1, pp. 1-20, Aug. 2009.
[28] A. Srivastava, S.H. Joshi, W. Mio, and X. Liu, "Statistical Shape Analysis: Clustering, Learning, and Testing," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 590-602, Apr. 2005.
[29] A. Veeraraghavan, A. Srivastava, A.K. Roy Chowdhury, and R. Chellappa, "Rate-Invariant Recognition of Humans and Their Activities," IEEE Trans. Image Processing, vol. 18, no. 6, pp. 1326-1339, June 2009.
[30] P.V. Overschee and B.D. Moor, "Subspace Algorithms for the Stochastic Identification Problem," Automatica, vol. 29, no. 3, pp. 649-660, May 1993.
[31] K.D. Cock and B.D. Moor, "Subspace Angles between ARMA Models," Systems and Control Letters, vol. 46, pp. 265-270, July 2002.
[32] P. Turaga and R. Chellappa, "Locally Time-Invariant Models of Human Activities Using Trajectories on the Grassmannian," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2435-2441, June 2009.
[33] P. Saisan, G. Doretto, Y.N. Wu, and S. Soatto, "Dynamic Texture Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 58-63, Dec. 2001.
[34] T.K. Kim, J. Kittler, and R. Cipolla, "Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1005-1018, June 2007.
[35] O. Arandjelovic, G. Shakhnarovich, J. Fisher, R. Cipolla, and T. Darrell, "Face Recognition with Image Sets Using Manifold Density Divergence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 581-588, June 2005.
[36] S.K. Zhou and R. Chellappa, "From Sample Similarity to Ensemble Similarity: Probabilistic Distance Measures in Reproducing Kernel Hilbert Space," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 917-929, June 2006.
[37] R. Basri and D.W. Jacobs, "Lambertian Reflectance and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[38] K.-C. Lee, J. Ho, and D.J. Kriegman, "Acquiring Linear Subspaces for Face Recognition under Variable Lighting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684-698, May 2005.
[39] H. Karcher, "Riemannian Center of Mass and Mollifier Smoothing," Comm. Pure and Applied Math., vol. 30, pp. 509-541, 1977.
[40] X. Pennec, "Statistical Computing on Manifolds: From Riemannian Geometry to Computational Anatomy," Proc. Emerging Trends in Visual Computing, pp. 347-386, 2008.
[41] K. Gallivan, A. Srivastava, X. Liu, and P. VanDooren, "Efficient Algorithms for Inferences on Grassmann Manifolds," Proc. IEEE 12th Workshop Statistical Signal Processing, Oct. 2003.
[42] D. Weinland, R. Ronfard, and E. Boyer, "Free Viewpoint Action Recognition Using Motion History Volumes," Computer Vision and Image Understanding, vol. 104, no. 2, pp. 249-257, Nov. 2006.
[43] "NIST Multiple Biometric Grand Challenge," http://face.nist. govmbgc/, 2011.
[44] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression Database," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[45] Y. Wang, H. Jiang, M.S. Drew, Z.N. Li, and G. Mori, "Unsupervised Discovery of Action Classes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1654-1661, 2006.
[46] K.C. Lee, J. Ho, M.H. Yang, and D. Kriegman, "Video-Based Face Recognition Using Probabilistic Appearance Manifolds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 313-320, June 2003.
[47] J. Hamm, "Subspace-Based Learning with Grassmann Kernels," PhD thesis, Univ. of Pennsylvania, 2008.
[48] K. Fukui and O. Yamaguchi, "Face Recognition Using Multi-Viewpoint Patterns for Robot Vision," Proc. Int'l Symp. Robotics Research, pp. 192-201, 2003.
[49] O. Yamaguchi, K. Fukui, and K. Maeda, "Face Recognition Using Temporal Image Sequence," Proc. Third Int'l Conf. Face and Gesture Recognition, pp. 318-323, Apr. 1998.
[50] P. Turaga, A. Veeraraghavan, and R. Chellappa, "Unsupervised View and Rate Invariant Clustering of Video Sequences," Computer Vision and Image Understanding, vol. 113, no. 3, pp. 353-371, Mar. 2009.
45 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool