This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Input Feature Selection by Mutual Information Based on Parzen Window
December 2002 (vol. 24 no. 12)
pp. 1667-1671

Abstract—Mutual information is a good indicator of relevance between variables, and have been used as a measure in several feature selection algorithms. However, calculating the mutual information is difficult, and the performance of a feature selection algorithm depends on the accuracy of the mutual information. In this paper, we propose a new method of calculating mutual information between input and class variables based on the Parzen window, and we apply this to a feature selection algorithm for classification problems.

[1] T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley&Sons, 1991.
[2] R. Battiti, “Using Mutual Information for Selecting Features in Supervised Neural Net Learning,” IEEE Trans. Neural Networks, vol. 5, pp. 537-550, July 1994.
[3] N. Kwak and C.-H. Choi, Improved Mutual Information Feature Selector for Neural Networks in Supervised Learning Proc. Int'l Joint Conf. Neural Networks, July 1999.
[4] N. Kwak and C.-H. Choi, Input Feature Selection for Classification Problems IEEE Trans. Neural Networks, vol. 13, no. 1, Jan. 2002.
[5] D. Xu and J. Principe, “Learning from Examples with Quadratic Mutual Information,” Neural Networks for Signal Processing—Proc. IEEE Workshop, pp. 155-164, 1998.
[6] K. Torkkola and W.M. Campbell, Mutual Information in Learning Feature Transformations Proc. Int'l Conf. Machine Learning, 2000.
[7] K. Torkkola, “Nonlinear Feature Transforms Using Maximum Mutual Information,” Proc. Int'l Joint Conf. Neural Networks, pp. 2756-2761, July 2001.
[8] E. Parzen, “On Estimation of a Probability Density Function and Mode,” Annals of Math. Statistics, vol. 33, pp. 1065-1076, Sept. 1962.
[9] G.A. Babich and O. Camps, Weighted Parzen Windows for Pattern Classification IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 5, pp. 567-570, May 1996.
[10] K. Fukunaga and R.R. Hayes, “The Reduced Parzen Classifier,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 4, pp. 423-425, Apr. 1989.
[11] C. Meilhac and C. Nastar, “Relevance Feedback and Category Search in Image Databases,” Proc. Int'l Conf. Multimedia Computing and Systems, pp. 512-517, 1999.
[12] Y. Muto, H. Nagase, and Y. Hamamoto, “Evaluation of a Modified Parzen Classifier in High-Dimensional Spaces,” Proc. 15th Int'l Conf. Pattern Recognition, vol. 2, pp. 67-70, 2000.
[13] Y. Hamamoto, S. Uchimura, and S. Tomita, “On the Behavior of Artificial Neural Network Classifiers in High-Dimensional Spaces,” IEEE Trans. Pattern Analysis and Machine Inteligence, vol. 18, no. 5, pp. 571-574, May 1996.
[14] P.M. Murphy and D.W. Aha, “UCI Repository of Machine Learning Databases,” For more information contacthttp://www.inria.fr/RRRTRR-3440.htmlml-repository@ics.uci.edu orhttp://www.cs.toronto.edu~delve/, 1994.
[15] N.R. Draper and H. Smith, Applied Regression Analysis, second ed. New York: John Wiley&Sons, 1981.
[16] J.R. Quinlan, C4.5: Programs for Machine Learning,San Mateo, Calif.: Morgan Kaufman, 1992.
[17] J.P. Siebert, “Vehicle Recognition Using Rule Based Methods,” technical report, Research Memorandum Turing Institute, TIRM-87-018, Mar. 1987.

Index Terms:
Feature selection, mutual information, Parzen window.
Citation:
Nojun Kwak, Chong-Ho Choi, "Input Feature Selection by Mutual Information Based on Parzen Window," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 12, pp. 1667-1671, Dec. 2002, doi:10.1109/TPAMI.2002.1114861
Usage of this product signifies your acceptance of the Terms of Use.