The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2013 vol.35)
pp: 1135-1148
A. J. Ma , Dept. of Comput. Sci., Hong Kong Baptist Univ., Hong Kong, China
P. C. Yuen , Dept. of Comput. Sci., Hong Kong Baptist Univ., Hong Kong, China
Jian-Huang Lai , Sch. of Inf. Sci. & Technol., Sun Yat-Sen Univ., Guangzhou, China
ABSTRACT
This paper addresses the independent assumption issue in fusion process. In the last decade, dependency modeling techniques were developed under a specific distribution of classifiers or by estimating the joint distribution of the posteriors. This paper proposes a new framework to model the dependency between features without any assumption on feature/classifier distribution, and overcomes the difficulty in estimating the high-dimensional joint density. In this paper, we prove that feature dependency can be modeled by a linear combination of the posterior probabilities under some mild assumptions. Based on the linear combination property, two methods, namely, Linear Classifier Dependency Modeling (LCDM) and Linear Feature Dependency Modeling (LFDM), are derived and developed for dependency modeling in classifier level and feature level, respectively. The optimal models for LCDM and LFDM are learned by maximizing the margin between the genuine and imposter posterior probabilities. Both synthetic data and real datasets are used for experiments. Experimental results show that LCDM and LFDM with dependency modeling outperform existing classifier level and feature level combination methods under nonnormal distributions and on four real databases, respectively. Comparing the classifier level and feature level fusion methods, LFDM gives the best performance.
INDEX TERMS
Joints, Mathematical model, Computational modeling, Kernel, Vectors, Linear programming, Optimization, multiple feature fusion, Linear dependency modeling, feature dependency, classifier level fusion, feature level fusion
CITATION
A. J. Ma, P. C. Yuen, Jian-Huang Lai, "Linear Dependency Modeling for Classifier Fusion and Feature Combination", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 5, pp. 1135-1148, May 2013, doi:10.1109/SNPD.2012.116
REFERENCES
[1] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[2] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 886-8930, 2005.
[3] X. He, S. Yan, Y. Hu, P. Niyogi, and H.J. Zhang, "Face Recognition Using Laplacianfaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 328-340, Mar. 2005.
[4] A. Ross, K. Nandakumar, and A.K. Jain, Handbook of Multibiometrics. Springer, 2006.
[5] P. Gehler and S. Nowozin, "On Feature Combination for Multiclass Object Classification," Proc. IEEE Int'l Conf. Computer Vision, pp. 221-228, 2009.
[6] J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas, "On Combining Classifiers," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 226-239, Mar. 1998.
[7] A. Jain, K. Nandakumar, and A. Ross, "Score Normalization in Multimodal Biometric Systems," Pattern Recognition, vol. 38, no. 12, pp. 2270-2285, 2005.
[8] K. Nandakumar, Y. Chen, S.C. Dass, and A.K. Jain, "Likelihood Ratio Based Biometric Score Fusion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 342-347, Feb. 2008.
[9] S. Prabhakar and A.K. Jain, "Decision-Level Fusion in Fingerprint Verification," Pattern Recognition, vol. 35, no. 4, pp. 861-874, 2002.
[10] B.W. Silverman, Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986.
[11] O.R. Terrades, E. Valveny, and S. Tabbone, "Optimal Classifier Fusion in a Non-Bayesian Probabilistic Framework," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 9, pp. 1630-1644, Sept. 2009.
[12] A. Demiriz, K.P. Bennett, and J. Shawe-Taylor, "Linear Programming Boosting via Column Generation," J. Machine Learning Research, vol. 46, nos. 1-3, pp. 225-254, 2002.
[13] A. Saffari, M. Godec, T. Pock, C. Leistner, and H. Bischof, "Online Multi-Class LPBoost," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 3570-3577, 2010.
[14] T.M. Cover and J.A. Thomas, Elements of Information Theory. Wiley-Interscience, 2006.
[15] F. Bach, G.R.G. Lanckriet, and M.I. Jordan, "Multiple Kernel Learning, Conic Duality, and the SMO Algorithm," Proc. Int'l Conf. Machine Learning, 2004.
[16] S. Sonnenburg, G. Rätsch, C. Schäfer, and B. Schölkopf, "Large Scale Multiple Kernel Learning," J. Machine Learning Research, vol. 7, pp. 1531-1565, 2006.
[17] A. Rakotomamonjy, F. Bach, S. Canu, and Y. Grandvalet, "SimpleMKL," J. Machine Learning Reseach, vol. 9, pp. 2491-2521, 2008.
[18] J. Yang, Y. Li, Y. Tian, L. Duan, and W. Gao, "Group-Sensitive Multiple Kernel Learning for Object Categorization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 436-443, 2009.
[19] M. Guillaumin, J. Verbeek, and C. Schmid, "Multimodal Semi-Supervised Learning for Image Classication," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 902-909, 2010.
[20] F. Orabona, J. Luo, and B. Caputo, "Online-Batch Strongly Convex Multi Kernel Learning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 787-794, 2010.
[21] A.J. Ma and P.C. Yuen, "Linear Dependency Modeling for Feature Fusion," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[22] M.A.T. Figueiredo and A.K. Jain, "Unsupervised Learning of Finite Mixture Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 381-396, Mar. 2002.
[23] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification. Wiley, 2000.
[24] D.G. Luenberger and Y. Ye, Linear and Nonlinear Programming, third ed. Springer, 2008.
[25] M.-E. Nilsback and A. Zisserman, "A Visual Vocabulary for Flower Classification," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[26] D.T.M.V. Breukelen, R.P.W. Duin, and J. Hartog, "Handwritten Digit Recognition by Combined Classifiers," Kybernetika, vol. 34, no. 4, pp. 381-386, 1998.
[27] L. Gorelick, M. Blank, E. Shechtman, M. Irani, and R. Basri, "Actions as Space-Time Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2247-2253, Dec. 2007.
[28] C. Schuldt, I. Laptev, and B. Caputo, "Recognizing Human Actions: A Local SVM Approach," Proc. IEEE Int'l Conf. Pattern Recognition, 2004.
[29] F.M. Alkoot and J. Kittler, "Experimental Evaluation of Expert Fusion Strategies," Pattern Recognition Letters, vol. 20, nos. 11-3, pp. 1361-1369, 1999.
[30] Z.I. Botev, J.F. Grotowski, and D.P. Kroese, "Kernel Density Estimation via Diffusion," Annals of Statistics, vol. 38, no. 5, pp. 2916-2957, 2010.
[31] M.-E. Nilsback and A. Zisserman, "Automated Flower Classification over a Large Number of Classes," Proc. IEEE Indian Conf. Computer Vision, Graphics, and Image Processing, 2008.
[32] S. Canu, Y. Grandvalet, V. Guigue, and A. Rakotomamonjy, "SVM and Kernel Methods Matlab Toolbox," Perception Systèmes et Information, INSA de Rouen, Rouen, France, 2005.
[33] A.W. Bowman and A. Azzalini, Applied Smoothing Techniques for Data Analysis: The Kernel Approach with S-Plus Illustrations. Oxford Univ. Press, 1997.
[34] P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior Recognition via Sparse Spatio-Temporal Features," Proc. IEEE Joint Int'l Workshop Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005.
[35] I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld, "Learning Realistic Human Actions from Movies," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[36] W. Feller, An Introduction to Probability Theory and Its Applications, vol. 1. Wiley, 1968.
[37] M.L.R.G.J. Székely and N.K. Bakirov, "Measuring and Testing Dependence by Correlation of Distances," Ann. of Statistics, vol. 35, no. 6, pp. 2769-2794, 2007.
852 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool