The Community for Technology Leaders
RSS Icon
Issue No.07 - July (2010 vol.32)
pp: 1342-1343
Kiran S. Balagani , Louisiana Tech University, Ruston
We derive the feature selection criterion presented in [CHECK END OF SENTENCE] and [CHECK END OF SENTENCE] from the multidimensional mutual information between features and the class. Our derivation: 1) specifies and validates the lower-order dependency assumptions of the criterion and 2) mathematically justifies the utility of the criterion by relating it to Bayes classification error.
Feature selection, entropy, mutual information, Bayes classification error, entropy estimation.
Kiran S. Balagani, "On the Feature Selection Criterion Based on an Approximation of Multidimensional Mutual Information", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 7, pp. 1342-1343, July 2010, doi:10.1109/TPAMI.2010.62
[1] R. Battiti, "Using Mutual Information for Selecting Features in Supervised Neural Net Learning," IEEE Trans. Neural Networks vol. 5, no. 4, pp. 537-550, 1994.
[2] H. Peng, F. Long, and C. Ding, "Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1226-1238, Aug. 2005.
[3] H.H. Yang and J. Moody, "Feature Selection Based on Joint Mutual Information," Proc. Int'l ICSC Symp. Advances in Intelligent Data Analysis, pp. 22-25, 1999.
[4] M.E. Hellman and J. Raviv, "Probability of Error, Equivocation, and the Chernoff Bound," IEEE Trans. Information Theory, vol. 16, pp. 368-372, 1970.
[5] T.M. Cover and J.A. Thomas, Elements of Information Theory, first ed. Wiley, 2004.
[6] L.G. J. Beirlant, E.J. Dudewicz, and E. van der Meulen, "Nonparametric Entropy Estimation: An Overview," Int'l J. Math. and Statistical Sciences, vol. 6, no. 1, pp. 17-39, 1997.
[7] N. Kwak and C.-H. Choi, "Input Feature Selection by Mutual Information Based on Parzen Window," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 12, pp. 1067-1671, Dec. 2002.
[8] P.M. LewisIII, "The Characteristic Selection Problem in Recognition Systems," IRE Trans. Information Theory, vol. 8, pp. 171-178, 1962.
[9] N. Kwak and C.-H. Choi, "Input Feature Selection for Classification Problems," IEEE Trans. Neural Networks, vol. 13, no. 1, pp. 143-159, 2002.
[10] H. Neemuchwala, A. Hero, and P. Carson, "Image Matching Using Alpha-Entropy Measures and Entropic Graphs," Signal Processing, vol. 85, no. 2, pp. 277-296, 2002.
[11] A.O. Hero, B. Ma, O.J.J. Michel, and J. Gorman, "Applications of Entropic Spanning Graphs," IEEE Signal Process Magazine, vol. 19, no. 5, pp. 85-95, 2002.
[12] B. Bonev, F. Escolano, and M. Cazorla, "Feature Selection, Mutual Information, and the Classification of High-Dimensional Patterns," Pattern Analysis and Applications, vol. 11, pp. 309-319, 2008.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool