CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2014 vol.36 Issue No.03 - March

Subscribe

Issue No.03 - March (2014 vol.36)

pp: 592-605

Ravishankar Sivalingam , Dept. of Comput. Sci. & Eng., Univ. of Minnesota, Twin Cities, Minneapolis, MN, USA

Daniel Boley , Dept. of Comput. Sci. & Eng., Univ. of Minnesota, Twin Cities, Minneapolis, MN, USA

Vassilios Morellas , Dept. of Comput. Sci. & Eng., Univ. of Minnesota, Twin Cities, Minneapolis, MN, USA

Nikolaos Papanikolopoulos , Dept. of Comput. Sci. & Eng., Univ. of Minnesota, Twin Cities, Minneapolis, MN, USA

ABSTRACT

In recent years, there has been extensive research on sparse representation of vector-valued signals. In the matrix case, the data points are merely vectorized and treated as vectors thereafter (for example, image patches). However, this approach cannot be used for all matrices, as it may destroy the inherent structure of the data. Symmetric positive definite (SPD) matrices constitute one such class of signals, where their implicit structure of positive eigenvalues is lost upon vectorization. This paper proposes a novel sparse coding technique for positive definite matrices, which respects the structure of the Riemannian manifold and preserves the positivity of their eigenvalues, without resorting to vectorization. Synthetic and real-world computer vision experiments with region covariance descriptors demonstrate the need for and the applicability of the new sparse coding model. This work serves to bridge the gap between the sparse modeling paradigm and the space of positive definite matrices.

INDEX TERMS

Covariance matrices, Sparse matrices, Vectors, Symmetric matrices, Dictionaries, Encoding, Tin,optimization, Sparse coding, positive definite matrices, region covariance descriptors, computer vision

CITATION

Ravishankar Sivalingam, Daniel Boley, Vassilios Morellas, Nikolaos Papanikolopoulos, "Tensor Sparse Coding for Positive Definite Matrices",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.36, no. 3, pp. 592-605, March 2014, doi:10.1109/TPAMI.2013.143REFERENCES

- [1] R. Tibshirani, "Regression Shrinkage and Selection via the Lasso,"
J. Royal Statistical Soc. (Series B), vol. 58, pp. 267-288, 1996.- [2] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least Angle Regression,"
The Annals of Statistics, vol. 32, no. 2, pp. 407-451, 2004.- [3] J. Tropp and A. Gilbert, "Signal Recovery from Random Measurements via Orthogonal Matching Pursuit,"
IEEE Trans. Information Theory, vol. 53, no. 12, pp. 4655-4666, Dec. 2007.- [4] D. Donoho, "Compressed Sensing,"
IEEE Trans. Information Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.- [5] E. Candes and M. Wakin, "An Introduction to Compressive Sampling,"
IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 21-30, Mar. 2008.- [6] J. Wright, Y. Ma, J. Mairal, G. Sapiro, T. Huang, and S. Yan, "Sparse Representation for Computer Vision and Pattern Recognition,"
Proc. IEEE, vol. 98, no. 6, pp. 1031-1044, June 2010.- [7] T. Hazan, S. Polak, and A. Shashua, "Sparse Image Coding Using a 3D Non-Negative Tensor Factorization,"
Proc. 10th IEEE Int'l Conf. Computer Vision, vol. 1, pp. 50-57, Oct. 2005.- [8] K.-R. Muller, S. Mika, G. Ratsch, K. Tsuda, and B. Scholkopf, "An Introduction to Kernel-Based Learning Algorithms,"
IEEE Trans. Neural Networks, vol. 12, no. 2, pp. 181-201, Mar. 2001.- [9] O. Tuzel, F. Porikli, and P. Meer, "Region Covariance: A Fast Descriptor for Detection and Classification,"
Proc. European Conf. Computer Vision (ECCV), pp. 589-600, 2006.- [10] F. Porikli, O. Tuzel, and P. Meer, "Covariance Tracking Using Model Update Based on Lie Algebra,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 728-735, June 2006.- [11] O. Tuzel, F. Porikli, and P. Meer, "Pedestrian Detection via Classification on Riemannian Manifolds,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1713-1727, Oct. 2008.- [12] S. Smith, "Covariance, Subspace, and Intrinsic Cramer-Rao Bounds,"
IEEE Trans. Signal Processing, vol. 53, no. 5, pp. 1610-1630, May 2005.- [13] X. Pennec, P. Fillard, and N. Ayache, "A Riemannian Framework for Tensor Computing,"
Int'l J. Computer Vision, vol. 66, pp. 41-66, Jan. 2006.- [14] X. Pennec, "Statistical Computing on Manifolds: From Riemannian Geometry to Computational Anatomy,"
Emerging Trends in Visual Computing, pp. 347-386, Springer-Verlag, 2009.- [15] L. Vandenberghe, S. Boyd, and S.-P. Wu, "Determinant Maximization with Linear Matrix Inequality Constraints,"
SIAM J. Matrix Analysis and Applications, vol. 19, pp. 499-533, Apr. 1998.- [16] R. Sivalingam, V. Morellas, D. Boley, and N. Papanikolopoulos, "Metric Learning for Semi-Supervised Clustering of Region Covariance Descriptors,"
Proc. Third ACM/IEEE Int'l Conf. Distributed Smart Cameras, pp. 1-8, Sept. 2009.- [17] V. Arsigny, P. Fillard, X. Pennec, and N. Ayache, "Log-Euclidean Metrics for Fast and Simple Calculus on Diffusion Tensors,"
Magnetic Resonance in Medicine, vol. 56, no. 2, pp. 411-421, Aug. 2006.- [18] R. Bhatia,
Positive Definite Matrices. Princeton Univ. Press, 2007.- [19] I.L. Dryden, A. Koloydenko, and D. Zhou, "Non-Euclidean Statistics for Covariance Matrices, with Applications to Diffusion Tensor Imaging,"
The Annals of Statistics, vol. 3, no. 3, pp. 1102-1123, 2009.- [20] F. Porikli and O. Tuzel, "Fast Construction of Covariance Matrices for Arbitrary Size Image Windows,"
Proc. IEEE Int'l Conf. Image Processing, pp. 1581-1584, Oct. 2006.- [21] H. Palaio and J. Batista, "Multi-Object Tracking Using an Adaptive Transition Model Particle Filter with Region Covariance Data Association,"
Proc. 19th Int'l Conf. Pattern Recognition, pp. 1-4, Dec. 2008.- [22] H. Wildenauer, B. Mičušík, and M. Vincze, "Efficient Texture Representation Using Multi-Scale Regions,"
Proc. Eighth Asian Conf. Computer Vision (ACCV '07), vol. Part I, pp. 65-74, 2007.- [23] J.Y. Tou, Y.H. Tay, and P.Y. Lau,
Gabor Filters as Feature Images for Covariance Matrix on Texture Classification Problem, pp. 745-751. Springer-Verlag, 2009.- [24] F. Porikli and T. Kocak, "Robust License Plate Detection Using Covariance Descriptor in a Neural Network Framework,"
Proc. IEEE Int'l Conf. Video and Signal Based Surveillance, pp. 107-107, Nov. 2006.- [25] A. Ruta, F. Porikli, S. Watanabe, and Y. Li, "In-Vehicle Camera Traffic Sign Detection and Recognition,"
Machine Vision and Applications, pp. 1-17, 2009.- [26] Y. Pang, Y. Yuan, and X. Li, "Gabor-Based Region Covariance Matrices for Face Recognition,"
IEEE Trans. Circuits and Systems for Video Technology, vol. 18, no. 7, pp. 989-993, July 2008.- [27] K. Guo, P. Ishwar, and J. Konrad, "Action Change Detection in Video by Covariance Matching of Silhouette Tunnels,"
Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 1110-1113, Mar. 2010.- [28] P.C. Cargill, C.U. Rius, D.M. Quiroz, and A. Soto, "Performance Evaluation of the Covariance Descriptor for Target Detection,"
Proc. Int'l Conf. Chilean Computer Science Soc., pp. 133-141, Nov. 2009.- [29] S. Zheng, H. Qiao, B. Zhang, and P. Zhang, "The Application of Intrinsic Variable Preserving Manifold Learning Method to Tracking Multiple People with Occlusion Reasoning,"
Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, pp. 2993-2998, Oct. 2009.- [30] J. Wang and Y. Wu, "Visual Tracking via Incremental Covariance Model Learning,"
Proc. Second Int'l Conf. Computer Modeling and Simulation, vol. 1, pp. 277-280, Jan. 2010.- [31] F. Porikli, "Learning on Manifolds,"
Proc. Joint IAPR Int'l Conf. Structural, Syntactic, and Statistical Pattern Recognition, pp. 20-39, 2010.- [32] F.R. Bach, G.R.G. Lanckriet, and M.I. Jordan, "Multiple Kernel Learning, Conic Duality, and the SMO Algorithm,"
Proc. Int'l Conf. Machine Learning. p. 6, 2004.- [33] F.R. Bach, G.R.G. Lanckriet, and M.I. Jordan, "Fast Kernel Learning Using Sequential Minimal Optimization," Technical Report UCB/CSD-04-1307, EECS Dept., Univ. of California, Berkeley, Feb. 2004.
- [34] Y. Ying, K. Huang, and C. Campbell, "Enhanced Protein Fold Recognition through a Novel Data Integration Approach,"
BMC Bioinformatics, vol. 10, no. 1, p. 267, 2009.- [35] K. Tsuda, G. Rätsch, and M.K. Warmuth, "Matrix Exponentiated Gradient Updates for On-Line Learning and Bregman Projection,"
J. Machine Learning Research, vol. 6, pp. 995-1018, Dec. 2005.- [36] G. Meyer, S. Bonnabel, and R. Sepulchre, "Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach,"
J. Machine Learning Research, vol. 12, pp. 593-625, Feb. 2011.- [37] Y.E. Nesterov and M. Todd, "On the Riemannian Geometry Defined by Self-Concordant Barriers and Interior-Point Methods,"
Foundations of Computational Math., vol. 2, no. 4, pp. 333-361, 2002.- [38] J.V. Davis, B. Kulis, P. Jain, S. Sra, and I.S. Dhillon, "Information-Theoretic Metric Learning,"
Proc. 24th Int'l Conf. Machine Learning (ICML '07), pp. 209-216, 2007.- [39] K. Guo, P. Ishwar, and J. Konrad, "Action Recognition Using Sparse Representation on Covariance Manifolds of Optical Flow,"
Proc. Seventh IEEE Int'l Conf. Advanced Video and Signal Based Surveillance, pp. 188-195, Sept. 2010.- [40] Z. Wang and B. Vemuri, "DTI Segmentation Using an Information Theoretic Tensor Dissimilarity Measure,"
IEEE Trans. Medical Imaging, vol. 24, no. 10, pp. 1267-1277, Oct. 2005.- [41] S. Sra and A. Cherian, "Generalized Dictionary Learning for Symmetric Positive Definite Matrices with Application to Nearest Neighbor Retrieval,"
Proc. European Conf. Machine Learning and Knowledge Discovery in Databases—Vol. Part III, pp. 318-332, 2011.- [42] G. Pfander, H. Rauhut, and J. Tanner, "Identification of Matrices Having a Sparse Representation,"
IEEE Trans. Signal Processing, vol. 56, no. 11, pp. 5376-5388, Nov. 2008.- [43] H. Wang, A. Banerjee, and D. Boley, "Modeling Time Varying Covariance Matrices in Low Dimensions," Technical Report TR-10-017, Dept. of Computer Science and Eng., Univ. of Minnesota, Aug. 2010.
- [44] B. Kulis, M. Sustik, and I. Dhillon, "Learning Low-Rank Kernel Matrices,"
Proc. 23rd Int'l Conf. Machine Learning (ICML '06), pp. 505-512, 2006.- [45] L.M. Bregman, "The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming,"
USSR Computational Math. and Math. Physics, vol. 7, no. 3, pp. 200-217, 1967.- [46] A. Banerjee, S. Merugu, I.S. Dhillon, and J. Ghosh, "Clustering with Bregman Divergences,"
J. Machine Learning Research, vol. 6, pp. 1705-1749, Dec. 2005.- [47] J. Wishart, "The Generalized Product Moment Distribution in Samples from a Normal Multivariate Population,"
Biometrika, vol. 20A, nos. 1/2, pp. 32-52, 1928.- [48] G. Wang, Y. Liu, and H. Shi, "Covariance Tracking via Geometric Particle Filtering,"
Proc. Second Int'l Conf. Intelligent Computation Technology & Automation, vol. 1, pp. 250-254, Oct. 2009.- [49] A. Gelman, J.B. Carlin, H.S. Stern, and D.S. Rubin,
Bayesian Data Analysis, second ed. Chapman & Hall/CRC, 2003.- [50] J. Tropp, "Just Relax: Convex Programming Methods for Identifying Sparse Signals in Noise,"
IEEE Trans. Information Theory, vol. 52, no. 3, pp. 1030-1051, Mar. 2006.- [51] D.L. Donoho and J. Tanner, "Sparse Nonnegative Solution of Underdetermined Linear Equations Bylinear Programming,"
Proc. Nat'l Academy of Sciences USA, vol. 102, no. 27, pp. 9446-9451, 2005.- [52] D. Donoho and V. Stodden, "When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts?"
Proc. Advances in Neural Information Processing Systems 16, 2004.- [53] D.D. Lee and H.S. Seung, "Algorithms for Non-Negative Matrix Factorization,"
Proc. Advances in Neural Information Processing Systems 16, T. Leen, T. Dietterich, and V. Tresp, eds., vol. 13, pp. 556-562, 2000.- [54] J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, "Robust Face Recognition via Sparse Representation,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210-227, Feb. 2009.- [55] P. Phillips, H. Moon, S. Rizvi, and P. Rauss, "The FERET Evaluation Methodology for Face-Recognition Algorithms,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090-1104, Oct. 2000.- [56] T. Randen and J.H. Husøy, "Filtering for Texture Classification: A Comparative Study,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 4, pp. 291-310, Apr. 1999.- [57] B. Kulis, M.A. Sustik, and I.S. Dhillon, "Low-Rank Kernel Learning with Bregman Matrix Divergences,"
J. Machine Learning Research, vol. 10, pp. 341-376, June 2009.- [58] J. Löfberg, "YALMIP : A Toolbox for Modeling and Optimization in MATLAB,"
Proc. IEEE Int'l Symp. Computer Aided Control Systems Design (CACSD), 2004.- [59] R.H. Tutuncu, K.C. Toh, and M.J. Todd, "Solving Semidefinite-Quadratic-Linear Programs Using SDPT3,"
Math. Programming, vol. 95, no. 2, pp. 189-217, 2003.- [60] C.-C. Chang and C.-J. Lin,
LIBSVM: A Library for Support Vector Machines, http://www.csie.ntu.edu.tw/cjlinlibsvm, 2001. |