This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
On the Feature Selection Criterion Based on an Approximation of Multidimensional Mutual Information
July 2010 (vol. 32 no. 7)
pp. 1342-1343
Kiran S. Balagani, Louisiana Tech University, Ruston
Vir V. Phoha, Louisiana Tech University, Ruston
We derive the feature selection criterion presented in [CHECK END OF SENTENCE] and [CHECK END OF SENTENCE] from the multidimensional mutual information between features and the class. Our derivation: 1) specifies and validates the lower-order dependency assumptions of the criterion and 2) mathematically justifies the utility of the criterion by relating it to Bayes classification error.

[1] R. Battiti, "Using Mutual Information for Selecting Features in Supervised Neural Net Learning," IEEE Trans. Neural Networks vol. 5, no. 4, pp. 537-550, 1994.
[2] H. Peng, F. Long, and C. Ding, "Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1226-1238, Aug. 2005.
[3] H.H. Yang and J. Moody, "Feature Selection Based on Joint Mutual Information," Proc. Int'l ICSC Symp. Advances in Intelligent Data Analysis, pp. 22-25, 1999.
[4] M.E. Hellman and J. Raviv, "Probability of Error, Equivocation, and the Chernoff Bound," IEEE Trans. Information Theory, vol. 16, pp. 368-372, 1970.
[5] T.M. Cover and J.A. Thomas, Elements of Information Theory, first ed. Wiley, 2004.
[6] L.G. J. Beirlant, E.J. Dudewicz, and E. van der Meulen, "Nonparametric Entropy Estimation: An Overview," Int'l J. Math. and Statistical Sciences, vol. 6, no. 1, pp. 17-39, 1997.
[7] N. Kwak and C.-H. Choi, "Input Feature Selection by Mutual Information Based on Parzen Window," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 12, pp. 1067-1671, Dec. 2002.
[8] P.M. LewisIII, "The Characteristic Selection Problem in Recognition Systems," IRE Trans. Information Theory, vol. 8, pp. 171-178, 1962.
[9] N. Kwak and C.-H. Choi, "Input Feature Selection for Classification Problems," IEEE Trans. Neural Networks, vol. 13, no. 1, pp. 143-159, 2002.
[10] H. Neemuchwala, A. Hero, and P. Carson, "Image Matching Using Alpha-Entropy Measures and Entropic Graphs," Signal Processing, vol. 85, no. 2, pp. 277-296, 2002.
[11] A.O. Hero, B. Ma, O.J.J. Michel, and J. Gorman, "Applications of Entropic Spanning Graphs," IEEE Signal Process Magazine, vol. 19, no. 5, pp. 85-95, 2002.
[12] B. Bonev, F. Escolano, and M. Cazorla, "Feature Selection, Mutual Information, and the Classification of High-Dimensional Patterns," Pattern Analysis and Applications, vol. 11, pp. 309-319, 2008.

Index Terms:
Feature selection, entropy, mutual information, Bayes classification error, entropy estimation.
Citation:
Kiran S. Balagani, Vir V. Phoha, "On the Feature Selection Criterion Based on an Approximation of Multidimensional Mutual Information," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 7, pp. 1342-1343, July 2010, doi:10.1109/TPAMI.2010.62
Usage of this product signifies your acceptance of the Terms of Use.