This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A New Criterion in Selection and Discretization of Attributes for the Generation of Decision Trees
December 1997 (vol. 19 no. 12)
pp. 1371-1375

Abstract—It is important to use a better criterion in selection and discretization of attributes for the generation of decision trees to construct a better classifier in the area of pattern recognition in order to intelligently access huge amount of data efficiently. Two well-known criteria are gain and gain ratio, both based on the entropy of partitions. We propose in this paper a new criterion based also on entropy, and use both theoretical analysis and computer simulation to demonstrate that it works better than gain or gain ratio in a wide variety of situations. We use the usual entropy calculation where the base of the logarithm is not two but the number of successors to the node. Our theoretical analysis leads some specific situations in which the new criterion works always better than gain or gain ratio, and the simulation result may implicitly cover all the other situations not covered by the analysis.

[1] J.R. Quinlan, "Induction of Decision Trees," Machine Learning, vol. 1, pp. 81-106, 1986.
[2] J. Cheng, U.M. Fayyad, K.B. Irani, and Z. Quian, "Improved Decision Trees: A Generalized Version of ID3," Proc. Fifth Int'l Conf. Machine Learning.San Mateo, Calif.: Morgan Kaufmann, pp. 100-108, 1988.
[3] U.M. Fayyad, "Branching on Attribute Values in Decision Tree Generation," AAAI-94; Proc. 12th Nat'l Conf. Artificial Intelligence, pp. 601-606,Seattle, Wash., July 31- Aug.4 1994.
[4] J.R. Quinlan, P.J. Compton, K.A. Horn, and L. Lazarus, "Inductive Knowledge Acquisition: A Case Study," J.R. Quinlan, Applications of Expert Systems.Reading, Mass.: Addison-Wesley, pp. 157-173, 1987.
[5] J.R. Quinlan, C4.5: Programs for Machine Learning.San Mateo, Calif.: Morgan Kaufmann, 1993.
[6] R.B. Ash, Information Theory.New York: Interscience, 1965.
[7] J.S. Lim, Two-Dimensional Signal and Image Processing.Englewood Cliffs, N.J.: Prentice Hall, 1990.
[8] U.M. Fayyad, On the Induction of Decision Trees for Multiple Concept Learning, PhD dissertation, Univ. of Michigan, 1991.
[9] J.R. Quinlan, "Decision Trees and Multi-Valued Attributes," J. Richard, ed., Machine Intelligence, vol. 11. Oxford, England: Oxford Univ. Press, pp. 305-318, 1988.
[10] J. Mingers, "An Empirical Comparison of Selection Measures for Decision-Tree Induction," Machine Learning, vol. 3, no. 4, pp. 319-342, 1989.
[11] L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone, Classification and Regression Trees.Belmont, Calif.: Wadsworth, 1984.
[12] U.M. Fayyad and K.B. Irani, "What Should Be Minimized in a Decision Tree?" AAAI-90; Proc. Eighth Nat'l Conf. Artificial Intelligence, pp. 749-754, 1990.

Index Terms:
Decision-tree generators, attribute selection, discretization, grouping, gain, gain ratio, normalized gain, entropy.
Citation:
Byung Hwan Jun, Chang Soo Kim, Hong-Yeop Song, Jaihie Kim, "A New Criterion in Selection and Discretization of Attributes for the Generation of Decision Trees," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 12, pp. 1371-1375, Dec. 1997, doi:10.1109/34.643896
Usage of this product signifies your acceptance of the Terms of Use.