The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2012 vol.34)
pp: 480-492
Andrea Vedaldi , Oxford University, Oxford
Andrew Zisserman , Oxford University, Oxford
ABSTRACT
Large scale nonlinear support vector machines (SVMs) can be approximated by linear ones using a suitable feature map. The linear SVMs are in general much faster to learn and evaluate (test) than the original nonlinear SVMs. This work introduces explicit feature maps for the additive class of kernels, such as the intersection, Hellinger's, and χ² kernels, commonly used in computer vision, and enables their use in large scale problems. In particular, we: 1) provide explicit feature maps for all additive homogeneous kernels along with closed form expression for all common kernels; 2) derive corresponding approximate finite-dimensional feature maps based on a spectral analysis; and 3) quantify the error of the approximation, showing that the error is independent of the data dimension and decays exponentially fast with the approximation order for selected kernels such as χ². We demonstrate that the approximations have indistinguishable performance from the full kernels yet greatly reduce the train/test times of SVMs. We also compare with two other approximation methods: Nystrom's approximation of Perronnin et al. [1], which is data dependent, and the explicit map of Maji and Berg [2] for the intersection kernel, which, as in the case of our approximations, is data independent. The approximations are evaluated on a number of standard data sets, including Caltech-101 [3], Daimler-Chrysler pedestrians [4], and INRIA pedestrians [5].
INDEX TERMS
Kernel methods, feature map, large scale learning, object recognition, object detection.
CITATION
Andrea Vedaldi, Andrew Zisserman, "Efficient Additive Kernels via Explicit Feature Maps", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 3, pp. 480-492, March 2012, doi:10.1109/TPAMI.2011.153
REFERENCES
[1] F. Perronnin, J. Sánchez, and Y. Liu, “Large-Scale Image Categorization with Explicit Data Embedding,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[2] S. Maji and A.C. Berg, “Max-Margin Additive Classifiers for Detection,” Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[3] L. Fei-Fei, R. Fergus, and P. Perona, “A Bayesian Approach to Unsupervised One-Shot Learning of Object Categories,” Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[4] S. Munder and D.M. Gavrila, “An Experimental Study on Pedestrian Classification,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1863-1868, Nov. 2006.
[5] N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2005.
[6] T. Joachims, “Training Linear SVMs in Linear Time,” Proc. 12th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, 2006.
[7] A. Rahimi and B. Recht, “Random Features for Large-Scale Kernel Machines,” Proc. Neural Information Processing Information Systems Conf., 2007.
[8] J. Zhang, M. Marszałek, S. Lazebnik, and C. Schmid, “Local Features and Kernels for Classification of Texure and Object Categories: An in-Depth Study,” technical report, INRIA, 2005.
[9] G. Csurka, C.R. Dance, L. Dan, J. Willamowski, and C. Bray, “Visual Categorization with Bags of Keypoints,” Proc. ECCV Workshop Statistical Learning in Computer Vision, 2004.
[10] J. Sivic and A. Zisserman, “Video Google: A Text Retrieval Approach to Object Matching in Videos,” Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[11] K. Grauman and T. Darrel, “The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features,” Proc. 10th IEEE Int'l Conf. Computer Vision, 2005.
[12] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond Bag of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[13] K.-W. Chang, C.-J. Hsieh, and C.-J. Lin, “Coordinate Descent Method for Large-Scale L2-Loss Linear Support Vector Machines,” J. Machine Learning Research, vol. 9, pp. 1369-1398, 2008.
[14] S. Shalev-Shwartz, Y. Singer, and N. Srebro, “Pegasos: Primal Estimated Sub-GrAdient SOlver for SVM,” Proc. 24th Int'l Conf. Machine Learning, 2007.
[15] L. Bottou and C.-J. Lin, “Support Vector Machine Solvers,” Large Scale Kernel Machines, L. Bottou, O. Chappelle, D. DeCoste, and J. Weston, eds., MIT Press, 2007.
[16] L. Bottou and O. Bousquet, “The Tradeoffs of Large Scale Learning,” Proc. Neural Information Processing Systems, 2008.
[17] J. Duchi and Y. Singer, “Efficient Learning Using Forward-Backward Splitting,” Proc. Neural Information Processing Systems, 2009.
[18] B. Schölkopf and A.J. Smola, Learning with Kernels. MIT Press, 2002.
[19] C.K.I. Williams and M. Seeger, “The Effect of the Input Density Distribution on Kernel-Based Classifiers,” Proc. 17th Int'l Conf. Machine Learning, 2000.
[20] C.K.I. Williams and M. Seeger, “Using the Nyström Method to Speed Up Kernel Machines,” Proc. Neural Information Processing Systems, 2001.
[21] A.J. Smola and B. Schölkopf, “Sparse Greedy Matrix Approximation for Machine Learning,” Proc. 17th Int'l Conf. Machine Learning, 2000.
[22] S. Fine and K. Scheinberg, “Efficient SVM Training Using Low-Rank Kernel Representations,” J. Machine Learning Research, vol. 2, pp. 243-264, 2001.
[23] F.R. Bach and M.I. Jordan, “Kernel Independent Component Analysis,” J. Machine Learning Research, vol. 3, no. 1, pp. 1-48, 2002.
[24] F.R. Bach and M.I. Jordan, “Predictive Low-Rank Decomposition for Kernel Methods,” Proc. 22nd Int'l Conf. Machine Learning, 2005.
[25] E. Snelson and Z. Ghaharamani, “Sparse Gaussian Processes Using Pseudo-Inputs,” Proc. Neural Information Processing Systems, 2006.
[26] L. Bo and C. Sminchisescu, “Efficient Match Kernels between Sets of Features for Visual Recognition,” Proc. Neural Information Processing Systems, 2009.
[27] M. Raginsky and S. Lazebnik, “Locality-Sensitive Binary Codes from Shift-Invariant Kernels,” Proc. Neural Information Processing Systems, 2009.
[28] F. Li, C. Ionescu, and C. Sminchisescu, “Random Fourier Approximations for Skewed Multiplicative Histogram Kernels,” Proc. 32nd DAGM Conf. Pattern Recognition, 2010.
[29] M. Hein and O. Bousquet, “Hilbertian Metrics and Positive Definite Kernels on Probability Measures,” Proc. Workshop Artificial Intelligence and Statistics, 2005.
[30] V. Sreekanth, A. Vedaldi, C.V. Jawahar, and A. Zisserman, “Generalized RBF Feature Maps for Efficient Detection,” Proc. British Machine Vision Conf., 2010.
[31] A. Vedaldi and B. Fulkerson, “VLFeat: An Open and Portable Library of Computer Vision Algorithms,” http:/www.vlfeat. org/, 2008.
[32] B. Schölkopf, “The Kernel Trick for Distances,” Proc. Neural Information Processing Systems, 2001.
[33] A. Barla, F. Odone, and A. Verri, “Histogram Intersection Kernel for Image Classification,” Proc. Int'l Conf. Image Processing, 2003.
[34] J. Puzicha, Y. Rubner, C. Tomasi, and J. Buhmann, “Empirical Evaluation of Dissimilarity Measures for Color and Texture,” Proc. Seventh IEEE Int'l Conf. Computer Vision, 1999.
[35] D.R. Martin, C. Fowlkes, and J. Malik, “Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 5, pp. 530-549, May 2004.
[36] A. Vedaldi, V. Gulshan, M. Varma, and A. Zisserman, “Multiple Kernels for Object Detection,” Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[37] B. Schölkopf, “Kernel Means,” lecture slides, 2007.
[38] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, “LIBLINEAR: A Library for Large Linear Classification,” J. Machine Learning Research, vol. 9, pp. 1871-1874, 2008.
[39] S. Maji, A.C. Berg, and J. Malik, “Classification Using Intersection Kernel Support Vector Machines Is Efficient,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[40] P. Ott and M. Everingham, “Implicit Color Segmentation Features for Pedestrian and Object Detection,” Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[41] P.F. Felzenszwalb, R.B. Grishick, D. McAllester, and D. Ramanan, “Object Detection with Discriminatively Trained Part Based Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Sept. 2010.
[42] M.B. Blaschko and C.H. Lampert, “Learning to Localize Objects with Structured Output Regression,” Proc. European Conf. Computer Vision, 2008.
[43] T. Joachims, T. Finley, and C.-N. J. Yu, “Cutting-Plane Training of Structural SVMs,” Machine Learning, vol. 77, no. 1, pp. 27-59, 2009.
[44] X. Wang, T.X. Han, and S. Yan, “An HOG-LBP Human Detector with Partial Occlusion Handling,” Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[45] S. Maji, A.C. Berg, and J. Malik http://www.cs.berkeley.edu/ smaji/projects ped-detector/, 2011.
33 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool