This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
NeC4.5: Neural Ensemble Based C4.5
June 2004 (vol. 16 no. 6)
pp. 770-773

Abstract—Decision tree is with good comprehensibility while neural network ensemble is with strong generalization ability. In this paper, these merits are integrated into a novel decision tree algorithm NeC4.5. This algorithm trains a neural network ensemble at first. Then, the trained ensemble is employed to generate a new training set through replacing the desired class labels of the original training examples with those output from the trained ensemble. Some extra training examples are also generated from the trained ensemble and added to the new training set. Finally, a C4.5 decision tree is grown from the new training set. Since its learning results are decision trees, the comprehensibility of NeC4.5 is better than that of neural network ensemble. Moreover, experiments show that the generalization ability of NeC4.5 decision trees can be better than that of C4.5 decision trees.

[1] C. Blake, E. Keogh, and C.J. Merz, UCI Repository of Machine Learning Databases http://www.ics.uci.edu/~mlearnMLRepository.html , Dept. of Information and Computer Science, Univ. of California, Irvine, 1998.
[2] L. Breiman, Bagging Predictors Machine Learning, vol. 24, no. 2, pp. 123-140, 1996.
[3] B. Efron and R. Tibshirani, An Introduction to the Bootstrap. New York: Chapman&Hall, 1993.
[4] L.K. Hansen and P. Salamon, “Neural Network Ensembles,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 10, pp. 993-1001, Oct. 1990.
[5] Working Notes of the IJCAI-95 Workshop on Machine Learning and Comprehensibility, Y. Kodratoff and C. Nédellec, eds., 1995.
[6] J.R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, Calif: Morgan Kaufmann, 1993.
[7] N.S.V. Rao, On Fusers that Perform Better than Best Sensor IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 8, pp. 904-909, Aug. 2001.
[8] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, Learning Internal Representations by Error Propagation Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 318-362, 1986.
[9] Z.-H. Zhou and Z.-Q. Chen, Hybrid Decision Tree Knowledge-Based Systems, vol. 15, no. 8, pp. 515-528, 2002.
[10] Z.-H. Zhou, J. Wu, and W. Tang, Ensembling Neural Networks: Many Could be Better than All Artificial Intelligence, vol. 137, nos. 1-2, pp. 239-263, 2002.

Index Terms:
Machine learning, decision tree, neural networks, ensemble learning, neural network ensemble, generalization, comprehensibility.
Citation:
Zhi-Hua Zhou, Yuan Jiang, "NeC4.5: Neural Ensemble Based C4.5," IEEE Transactions on Knowledge and Data Engineering, vol. 16, no. 6, pp. 770-773, June 2004, doi:10.1109/TKDE.2004.11
Usage of this product signifies your acceptance of the Terms of Use.