The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan. (2013 vol.35)
pp: 66-77
S. Maji , Toyota Technol. Inst. at Chicago, Chicago, IL, USA
A. C. Berg , Comput. Sci. Dept., Stony Brook Univ., Stony Brook, NY, USA
J. Malik , Univ. of California at Berkeley, Berkeley, CA, USA
ABSTRACT
We show that a class of nonlinear kernel SVMs admits approximate classifiers with runtime and memory complexity that is independent of the number of support vectors. This class of kernels, which we refer to as additive kernels, includes widely used kernels for histogram-based image comparison like intersection and chi-squared kernels. Additive kernel SVMs can offer significant improvements in accuracy over linear SVMs on a wide variety of tasks while having the same runtime, making them practical for large-scale recognition or real-time detection tasks. We present experiments on a variety of datasets, including the INRIA person, Daimler-Chrysler pedestrians, UIUC Cars, Caltech-101, MNIST, and USPS digits, to demonstrate the effectiveness of our method for efficient evaluation of SVMs with additive kernels. Since its introduction, our method has become integral to various state-of-the-art systems for PASCAL VOC object detection/image classification, ImageNet Challenge, TRECVID, etc. The techniques we propose can also be applied to settings where evaluation of weighted additive kernels is required, which include kernelized versions of PCA, LDA, regression, k-means, as well as speeding up the inner loop of SVM classifier training algorithms.
INDEX TERMS
Kernel, Additives, Histograms, Support vector machines, Complexity theory, Piecewise linear approximation, Training,additive kernels, Image classification, support vector machines, efficient classifiers
CITATION
S. Maji, A. C. Berg, J. Malik, "Efficient Classification for Additive Kernel SVMs", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 1, pp. 66-77, Jan. 2013, doi:10.1109/TPAMI.2012.62
REFERENCES
[1] S. Agarwal and D. Roth, "Learning a Sparse Representation for Object Detection," Proc. European Conf. Computer Vision, 2002.
[2] S. Belongie, C. Fowlkes, F. Chung, and J. Malik, "Spectral Partitioning with Indefinite Kernels Using the Nystrom Extension," Proc. European Conf. Computer Vision, 2002.
[3] S. Boughorbel, J.-P. Tarel, and N. Boujemaa, "Generalized Histogram Intersection Kernel for Image Recognition," Proc. IEEE Conf. Image Processing, 2005.
[4] L. Bourdev and J. Brandt, "Robust Object Detection via Soft Cascade," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[5] C. Burges and B. Schölkopf, "Improving the Accuracy and Speed of Support Vector Machines," Proc. Neural Information Processing Systems, 1997.
[6] C.J.C. Burges, "Simplified Support Vector Decision Rules," Proc. Int'l Conf. Machine Learning, 1996.
[7] O. Chapelle, P. Haffner, and V. Vapnik, "Support Vector Machines for Histogram-Based Image Classification," IEEE Trans. Neural Networks, vol. 10, no. 5, pp. 1055-1064, Sept. 1999.
[8] O. Chum and A. Zisserman, "Presented at Pascal Visual Recognition Challenge Workshop," 2007.
[9] C. Cortes and V. Vapnik, "Support-Vector Networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
[10] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[11] D. Decoste and B. Schölkopf, "Training Invariant Support Vector Machines," Machine Learning, vol. 46, nos. 1-3, pp. 161-190, 2002.
[12] M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, and A. Zisserman, "The Pascal Visual Object Classes (VOC) Challenge," Int'l J. Computer Vision, vol. 88, no. 2, pp. 303-338, June 2010.
[13] R. Fan, K. Chang, C. Hsieh, X. Wang, and C. Lin, "Liblinear: A Library for Large Linear Classification," J. Machine Learning Research, vol. 9, pp. 1871-1874, 2008.
[14] L. Fei-Fei, R. Fergus, and P. Perona, "Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[15] L. Fei-Fei, R. Fergus, and P. Perona, "One-Shot Learning of Object Categories," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 594-611, Apr. 2006.
[16] P. Felzenszwalb, D. McAllester, and D. Ramanan, "A Discriminatively Trained Multiscale, Deformable Part Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[17] R. Fergus, P. Perona, and A. Zisserman, "Object Class Recognition by Unsupervised Scale-Invariant Learning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[18] J. Friedman and W. Stuetzle, "Projection Pursuit Regression," J. Am. Statistical Assoc., vol. 76, pp. 817-823, 1981.
[19] A. Garg, S. Agarwal, and T.S. Huang, "Fusion of Global and Local Information for Object Detection," Proc. Int'l Conf. Pattern Recognition, 2002.
[20] K. Grauman and T. Darrell, "The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[21] K. Grauman and T. Darrell, "Unsupervised Learning of Categories from Sets of Partially Matching Image Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[22] K. Grauman and T. Darrell, "The Pyramid Match Kernel: Efficient Learning with Sets of Features," J. Machine Learning Research, vol. 8, pp. 725-760, 2007.
[23] T. Hastie and R. Tibshirani, Generalized Additive Models. Chapman & Hall/CRC, 1990.
[24] B. Heisele, T. Serre, S. Prentice, and T. Poggio, "Hierarchical Classification and Feature Reduction for Fast Face Detection with Support Vector Machines," Pattern Recognition, vol. 36, no. 11, pp. 2007-2017, Sept. 2003.
[25] M. Herbster, "Learning Additive Models Online with Fast Evaluating Kernels," Proc. 14th Ann. Conf. Computational Learning Theory and Fifth European Conf. Computational Learning, pp. 444-460, 2001.
[26] D. Hoiem, A. Efros, and M. Hebert, "Putting Objects in Perspective," Int'l J. Computer Vision, vol. 80, no. 1, pp. 3-15-15, Oct. 2008.
[27] P. Indyk and N. Thaper, "Fast Image Retrieval via Embeddings," Proc. Third Int'l Workshop Statistical and Computational Theories of Vision, 2003.
[28] W.J. Deng, R. Dong, L.-J. Socher, K.L. Li, and L. Fei-Fei, "Imagenet: A Large-Scale Hierarchical Image Database," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[29] T. Joachims, "Training Linear svms in Linear Time," Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, 2006.
[30] S.S. Keerthi, O. Chapelle, and D. DeCoste, "Building Support Vector Machines with Reduced Classifier Complexity," J. Machine Learning Research, vol. 7, pp. 1493-1515, 2006.
[31] C.H. Lampert, M.B. Blaschko, and T. Hofmann, "Beyond Sliding Windows: Object Localization by Efficient Subwindow Search," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[32] S. Lazebnik, C. Schmid, and J. Ponce, "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[33] B. Leibe, A. Leonardis, and B. Schiele, "Combined Object Categorization and Segmentation with an Implicit Shape Model," Proc. European Conf. Computer Vision Workshop Statistical Learning in Computer Vision, pp. 17-32, 2004.
[34] S. Maji and A.C. Berg, "Max Margin Additive Classifiers for Detection," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[35] S. Maji, A.C. Berg, and J. Malik, "Classification Using Intersection Kernel Support Vector Machines is Efficient," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[36] S. Maji and J. Malik, "Fast and Accurate Digit Classification," Technical Report UCB/EECS-2009-159, EECS Department, Univ. of California, Berkeley, Nov. 2009.
[37] S. Maji and J. Malik, "Object Detection Using a Max-Margin Hough Transform," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[38] S. Mika, G. Ratsch, J. Weston, B. Scholkopf, and K. Mullers, "Fisher Discriminant Analysis with Kernels," Proc. IEEE Signal Processing Soc. Workshop Neural Networks for Signal Processing, 1999.
[39] S. Munder and D.M. Gavrila, "An Experimental Study on Pedestrian Classification," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1863-1868, Nov. 2006.
[40] J. Mutch and D.G. Lowe, "Multiclass Object Recognition with Sparse Localized Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[41] F. Odone, A. Barla, and A. Verri, "Building Kernels from Binary Strings for Image Matching," IEEE Trans. Image Processing, vol. 14, no. 2, pp. 169-180, Feb. 2005.
[42] E. Osuna, R. Freund, and F. Girosi, "Training Support Vector Machines: An Application to Face Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[43] E.E. Osuna and F. Girosi, "Reducing the Run-Time Complexity in Support Vector Machines," Advances in Kernel Methods: Support Vector Learning, pp. 271-283, The MIT Press, 1999.
[44] C. Papageorgiou and T. Poggio, "A Trainable System for Object Detection," Int'l J. Computer Vision, vol. 38, no. 1, pp. 15-33, 2000.
[45] F. Perronnin, J. Sandnchez, and Y. Liu, "Large-Scale Image Categorization with Explicit Data Embedding," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[46] A. Rahimi and B. Recht, "Random Features for Large-Scale Kernel Machines," Proc. Neural Information Processing Systems, 2007.
[47] S. Romdhani, P. Torr, B. Scholkopf, and A. Blake, "Computationally Efficient Face Detection," Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[48] R. Schapire, "A Brief Introduction to Boosting," Proc. Int'l Joint Conf. Artificial Intelligence, 1999.
[49] B. Schölkopf, A. Smola, and K. Müller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem," Neural Computation, vol. 10, no. 5, pp. 1299-1319, 1998.
[50] B. Scholkopf and A.J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2001.
[51] S. Shalev-Shwartz, Y. Singer, and N. Srebro, "Pegasos: Primal Estimated Sub-Gradient Solver for SVM," Proc. Int'l Conf. Machine Learning, 2007.
[52] L. Shang, L. Yang, F. Wang, K.-P. Chan, and X.-S. Hua, "Real-Time Large Scale Near-Duplicate Web Video Retrieval," Proc. ACM Int'l Conf. Multimedia, 2010.
[53] A.F. Smeaton, P. Over, and W. Kraaij, "Evaluation Campaigns and Trecvid," Proc. Eighth ACM Int'l Workshop Multimedia Information Retrieval, pp. 321-330, 2006.
[54] M.J. Swain and D.H. Ballard, "Color Indexing," Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[55] A. Torralba, K. Murphy, and W. Freeman, "Sharing Features: Efficient Boosting Procedures for Multiclass Object Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[56] K. Van de Sande, J. Uijlings, T. Gevers, and A. Smeulders, "Segmentation as Selective Search for Object Recognition," Proc. 13th IEEE Int'l Conf. Computer Vision, 2011.
[57] M. Varma and D. Ray, "Learning the Discriminative Power-Invariance Trade-Off," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007.
[58] A. Vedaldi, V. Gulshan, M. Varma, and A. Zisserman, "Multiple Kernels for Object Detection," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[59] A. Vedaldi and A. Zisserman, "Efficient Additive Kernels via Explicit Feature Maps," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[60] A. Vedaldi and A. Zisserman, "Efficient Additive Kernels via Explicit Feature Maps," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 3, pp. 480-492, Mar. 2012.
[61] P. Viola and M.J. Jones, "Robust Real-Time Face Detection," Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[62] S. Walk, N. Majer, K. Schindler, and B. Schiele, "New Features and Insights for Pedestrian Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[63] G. Wang, D. Hoiem, and D. Forsyth, "Learning Image Similarity from Flickr Groups Using Stochastic Intersection Kernel Machines," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[64] W. Zhang, G. Zelinsky, and D. Samaras, "Real-Time Accurate Object Detection Using Multiple Resolutions," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007.
[65] J. Wu and J.M. Rehg, "Beyond the Euclidean Distance: Creating Effective Visual Codebooks Using the Histogram Intersection Kernel," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[66] C. Yang, R. Duraiswami, N.A. Gumerov, and L. Davis, "Improved Fast Gauss Transform and Efficient Kernel Density Estimation," Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[67] J. Zhang, M. Marszalek, S. Lazebnik, and C. Schmid, "Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study," Int'l J. Computer Vision, vol. 73, no. 2, pp. 213-238, 2007.
49 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool