The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2012 vol.34)
pp: 1992-2004
Yiqun Hu , The University of Western Australia, Crawley
Ajmal S. Mian , The University of Western Australia, Crawley
Robyn Owens , The University of Western Australia, Crawley
ABSTRACT
We propose an efficient and robust solution for image set classification. A joint representation of an image set is proposed which includes the image samples of the set and their affine hull model. The model accounts for unseen appearances in the form of affine combinations of sample images. To calculate the between-set distance, we introduce the Sparse Approximated Nearest Point (SANP). SANPs are the nearest points of two image sets such that each point can be sparsely approximated by the image samples of its respective set. This novel sparse formulation enforces sparsity on the sample coefficients and jointly optimizes the nearest points as well as their sparse approximations. Unlike standard sparse coding, the data to be sparsely approximated are not fixed. A convex formulation is proposed to find the optimal SANPs between two sets and the accelerated proximal gradient method is adapted to efficiently solve this optimization. We also derive the kernel extension of the SANP and propose an algorithm for dynamically tuning the RBF kernel parameter while matching each pair of image sets. Comprehensive experiments on the UCSD/Honda, CMU MoBo, and YouTube Celebrities face datasets show that our method consistently outperforms the state of the art.
INDEX TERMS
Approximation methods, Optimization, Data models, Vectors, Hidden Markov models, Adaptation models, convex optimization., Image set classification, face recognition, sparse modeling
CITATION
Yiqun Hu, Ajmal S. Mian, Robyn Owens, "Face Recognition Using Sparse Approximated Nearest Points between Image Sets", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 10, pp. 1992-2004, Oct. 2012, doi:10.1109/TPAMI.2011.283
REFERENCES
[1] A. Beck and M. Teboulle, "A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems," SIAM J. Imaging Sciences, vol. 2, no. 1, pp. 183-202, 2009.
[2] A.W Fitzgibbon and A. Zisserman, "Joint Manifold Distance: A New Approach to Appearance Based Clustering," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 26-33, 2003.
[3] B. Schölkopf, A. Smola, and K.R. Müller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem," Neural Computation, vol. 10, no. 5, pp. 1299-1319, 1998.
[4] D.A. Ross, J. Lim, R.S. Lin, and M.H. Yang, "Incremental Learning for Robust Visual Tracking," Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 125-141, 2008.
[5] E. Oja, Subspace Methods of Pattern Recognition. Research Studies Press, 1983.
[6] G. Shakhnnarvovich, J.W Fisher, and T. Darrell, "Face Recognition from Long-Term Observations," Proc. European Conf. Computer Vision, pp. 851-865, 2002.
[7] H. Cevikalp and B. Triggs, "Face Recognition Based on Image Sets," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 2567-2573, 2010.
[8] H. Hotelling, "Relations between Two Sets of Variates," Biometrika, vol. 28, nos. 3/4, pp. 321-377, 1936.
[9] H. Lee, A. Battle, R. Raina, and A.Y. Ng, "Efficient Sparse Coding Algorithms," Proc. Ann. Conf. Neural Information Processing Systems, pp. 801-808, 2006.
[10] J.R Beveridge, B.A Draper, J.M. Chang, M. Kirby, H. Kley, and C. Peterson, "Principal Angles Separate Subject Illumination Spaces in YDB and CMU-PIE," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 351-363, Feb. 2009.
[11] J. Weng, C.H. Evans, and W.S. Hwang, "An Incremental Learning Method for Face Recognition under Continuous Video Stream," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 251-256, 2000.
[12] J. Wright, A.Y. Yang, A. Ganesh, S.S. Sastry, and Y. Ma, "Robust Face Recognition via Sparse Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210-227, Feb. 2009.
[13] K.C. Lee, J. Ho, M.H. Yang, and D. Kriegman, "Video-Based Face Recognition Using Probabilistic Appearance Manifolds," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 313-320, 2003.
[14] K. Fukui and O. Yamaguchi, "The Kernel Orthogonal Mutual Subspace Method and Its Application to 3D Object Recognition," Proc. Asian Conf. Computer Vision, pp. 467-476, 2007.
[15] K. Fukui, B. Stenger, and O. Yamaguchi, "A Framework for 3D Object Recognition Using the Kernel Constrained Mutual Subspace Method," Proc. Asian Conf. Computer Vision, pp. 315-324, 2006.
[16] L. Wolf and A. Shashua, "Learning over Sets Using Kernel Principal Angles," J. Machine Learning Research, vol. 4, no. 10, pp. 913-931, 2003.
[17] M.J. Lyons, J. Budynek, and S. Akamatsu, "Automatic Classification of Single Facial Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357-1362, Dec. 1999.
[18] M. Kim, S. Kumar, V. Pavlovic, and H. Rowley, "Face Tracking and Recognition with Visual Constraints in Real-World Videos," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[19] M. Nishiyama, M. Yuasa, T. Shibata, T. Wakasugi, T. Kawahara, and O. Yamaguchi, "Recognizing Faces of Moving People by Hierarchical Image-Set Matching," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[20] M. Nishiyama, O. Yamaguchi, and K. Fukui, "Face Recognition with the Multiple Constrained Mutual Subspace Method," Proc. Int'l Conf. Audio- and Video-Based Biometric Person Authentication, pp. 71-80, 2005.
[21] O. Arandjelovic and R. Cipolla, "A Pose-Wise Linear Illumination Manifold Model for Face Recognition Using Video," Computer Vision and Image Understanding, vol. 113, no. 1, pp. 113-125, 2009.
[22] O. Arandjelovic, G. Shakhnarovich, J. Fisher, R. Cipolla, and T. Darrell, "Face Recognition with Image Sets Using Manifold Density Divergence," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 581-588, 2005.
[23] O. Boiman, E. Shechtman, and M. Irani, "In Defense of Nearest-Neighbor Based Image Classification," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[24] O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee, "Choosing Multiple Parameters for Support Vector Machines," Machine Learning, vol. 46, pp. 131-159, 2002.
[25] O. Yamaguchi, K. Fukui, and K.i. Maeda, "Face Recognition Using Temporal Image Sequence," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 318-323, 1998.
[26] P.N. Belhumeur, J.P. Hespanha, and D.J Kriegman, "Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[27] P. Viola and M.J. Jones, "Robust Real-Time Face Detection," Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[28] R. Gross and J. Shi, "The CMU Motion of Body (MoBo) Database," Technical Report CMU-RI-TR-01-18, Robotics Inst., 2001.
[29] R. Wang and X. Chen, "Manifold Discriminant Analysis," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 429-436, 2009.
[30] R. Wang, S. Shan, X. Chen, and W. Gao, "Manifold-Manifold Distance with Application to Face Recognition based on Image Set," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[31] S. Mika, G. Rätsch, and K.R. Müller, "A Mathematical Programming Approach to the Kernel Fisher Algorithm," Proc. Ann. Conf. Neural Information Processing Systems, pp. 801-808, 2000.
[32] S. Yan, D. Xu, B. Zhang, H.J. Zhang, Q. Yang, and S. Lin, "Graph Embedding and Extensions: A General Framework for Dimensionality Reduction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 40-51, Jan. 2007.
[33] S. Zhou and R. Chellappa, "Probabilistic Human Recognition from Video," Proc. European Conf. Computer Vision, pp. 681-697, 2002.
[34] J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge Univ. Press, 2004.
[35] T. Ahonen, A. Hadid, and M. Pietikainen, "Face Description with Local Binary Patterns: Application to Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, Dec. 2006.
[36] T.J. Chin, K. Schindler, and D. Suter, "Incremental Kernel SVD for Face Recognition with Image Sets," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 461-466, 2006.
[37] T.K. Kim and R. Cipolla, "On-Line Learning for Maximizing Orthogonality between Subspaces and Its Application to Image Set-Based Face Recognition," IEEE Trans. Image Processing, vol. 19, no. 4, pp. 1067-1074, Apr. 2009.
[38] T.K. Kim, J. Kittler, and R. Cipolla, "Incremental Learning of Locally Orthogonal Subspaces for Set-Based Object Recognition," Proc. British Machine Vision Conf., pp. 559-568, 2006.
[39] T.K. Kim, O. Arandjelovic, and R. Cipolla, "Learning over Sets Using Boosted Manifold Principle Angles (BoMPA)," Proc. British Machine Vision Conf., pp. 779-788, 2005.
[40] T.K. Kim, O. Arandjelovic, and R. Cipolla, "Boosted Manifold Principal Angles for Image Set-Based Recognition," Pattern Recognition, vol. 40, no. 9, pp. 2475-2484, 2007.
[41] T.K. Kim, O. Arandjelovic, and R. Cipolla, "Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1005-1018, June 2007.
[42] T. Wang and P. Shi, "Kernel Grassmannian Distances and Discriminant Analysis for Face Recognition from Image Sets," Pattern Recognition Letters, vol. 30, no. 13, pp. 1161-1165, 2009.
[43] V.N. Vapnik, Statistical Learning Theory. Wiley-Interscience, 1998.
[44] W. Fan and D.Y. Yeung, "Locally Linear Models on Face Appearance Manifolds with Application to Dual-Subspace Based Classification," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1384-1390, 2006.
[45] X. Li, K. Fukui, and N. Zheng, "Boosting Constrained Mutual Subspace Method for Robust Image-Set Based Object Recognition," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 1132-1137, 2009.
[46] X. Li, K. Fukui, and N. Zheng, "Image-Set Based Face Recognition Using Boosted Global and Local Principal Angles," Proc. Asian Conf. Computer Vision, pp. 323-332, 2009.
[47] X. Liu and T. Cheng, "Video-Based Face Recognition Using Adaptive Hidden Markov Models," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 340-348, 2003.
[48] Y. Hu, A.S. Mian, and R. Owens, "Sparse Approximated Nearest Points for Image Set Classification," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2011.
[49] Y. Nesterov, "Gradient Methods for Minimizing Composite Objective Function," Technical Report 2007076, Université Catholique de Louvain, Center for Operations Research and Econometrics (CORE), 2007.
44 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool