This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
An Iterative Growing and Pruning Algorithm for Classification Tree Design
February 1991 (vol. 13 no. 2)
pp. 163-174

A critical issue in classification tree design-obtaining right-sized trees, i.e. trees which neither underfit nor overfit the data-is addressed. Instead of stopping rules to halt partitioning, the approach of growing a large tree with pure terminal nodes and selectively pruning it back is used. A new efficient iterative method is proposed to grow and prune classification trees. This method divides the data sample into two subsets and iteratively grows a tree with one subset and prunes it with the other subset, successively interchanging the roles of the two subsets. The convergence and other properties of the algorithm are established. Theoretical and practical considerations suggest that the iterative free growing and pruning algorithm should perform better and require less computation than other widely used tree growing and pruning algorithms. Numerical results on a waveform recognition problem are presented to support this view.

[1] A. C. Anderson and K. S. Fu, "Design and development of a linear binary tree classifier for leukocytes," Purdue Univ., West Lafayette, IN, Tech. Rep. TR-EE-79-31, 1979.
[2] P. Argentiero, R. Chin, and P. Beaudet, "An automated approach to the design of decision tree classifiers,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-4, pp. 51-57, 1982.
[3] L. Atlas, J. Connor, D. Park, M. El-Sharkawi, R. Marks II, A. Lippman, R. Cole, and Y. Muthusamy, "A performance comparison of trained multilayer perceptrons and trained classifier trees," inProc. IEEE Conf. Systems, Man, Cybernetics, 1989, pp. 915-920.
[4] L.A. Bartolucci, P.H. Swain, and C. Wu, "Selective radiant temperature mapping using a layered classifier,"IEEE Trans. Geosci. Electron., vol. GE-14, pp. 101-106, 1976.
[5] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone,Classification and Regression Trees. Belmont, CA: Wadsworth, 1984.
[6] R. G. Casey and G. Nagy, "Decision tree design using a probabilistic model,"IEEE Trans. Inform. Theory, vol. IT-30, pp. 93-99, 1984.
[7] P. A. Chou, T. Lookabaugh, and R. M. Gray, "Optimal pruning with applications to tree structured source coding and modeling,"IEEE Trans. Inform. Theory, vol. 35, pp. 299-315, Mar. 1989.
[8] G. R. Dattatreya and V. V. S. Sarma, "Decision tree design for pattern recognition including feature measurement cost," inProc. 5th Int. Conf. Pattern Recognition, Dec. 1980, pp. 1212-1214.
[9] J.H. Friedman, "A recursive partitioning decision rule for nonparametric classification,"IEEE Trans. Comput., vol. C-26, pp. 404-408, 1973.
[10] R. Goodman and P. Smyth, "Decision tree design from a communication theory standpoint,"IEEE Trans. Inform. Theory, vol. IT-34, pp. 979-994, 1988.
[11] Y. X. Gu, Q. R. Wang, and C. Y. Suen, "Application of multilayer decision tree in computer recognition of Chinese characters,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-5, pp. 83-89, Jan. 1983.
[12] D. E. Gustafson, S. B. Gelfand, and S. K. Mitter, "A nonparametric multiclass partitioning method for classification," inProc. 5th Int. Conf. Pattern Recognition, 1980, pp. 654-659.
[13] E. G. Henrichon and K. S. Fu, "A nonparametric partitioning procedure for pattern classification,"IEEE Trans. Comput., vol. C-18, pp. 614-624, 1969.
[14] A. V. Kulkarni, "On the mean accuracy of hierarchical classifiers,"IEEE Trans. Comput., vol. C-27, pp. 771-776, 1978.
[15] A. V. Kulkarni and L. N. Kanal, "Admissible search strategies for parametric and nonparametric hierarchical classifiers," inProc. 4th Int. Joint Conf. Pattern Recognition, 1978, pp. 238-248.
[16] M. W. Kurzynski, "The optimal strategy of a tree classifier,"Pattern Recognition, vol. 16, pp. 81-87, 1983.
[17] M. W. Kurzynski, "Decision rules for a hierarchical classifier,"Pattern Recognition Lett., vol. 1, pp. 305-310, July 1983.
[18] G. H. Landeweerd, T. Timmers, and E. S. Gelsema, "Binary tree versus single level tree classification of white blood cells,"Pattern Recognition, vol. 16, no. 6, pp. 571-577, Dec. 1983.
[19] Y. K. Lin and K. S. Fu, "Automatic classification of cervical cells using a binary tree classifier,"Pattern Recognition, vol. 16, pp. 69-80, 1983.
[20] W. S. Meisel and D.A. Michalopoulos, "A partitioning algorithm with application in pattern classification and the optimization of decision tree,"IEEE Trans. Comput., vol. C-22, pp. 93-103, 1973.
[21] J. K. Mui and K. S. Fu, "Automated classification of nucleated blood cells using a binary tree classifier,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-2, pp. 429-443, 1980.
[22] H. J. Payne and W. S. Meisel, "An algorithm for constructing optimal binary decision trees,"IEEE Trans. Comput., vol. C-26 pp. 905-916, 1977.
[23] S. Qing-Yun and K. S. Fu, "A method for the design of binary tree classifiers,"Pattern Recognition, vol. 16, pp. 593-603, 1983.
[24] E. M. Rounds, "A combined nonparametric approach to feature selection and binary decision tree design,"Pattern Recognition, vol. 12, pp. 313-317, 1980.
[25] I. K. Sethi and B. Chatterjee, "Efficient decision tree design for discrete variable pattern recognition problems,"Pattern Recognition, vol. 9, pp. 197-206, 1977.
[26] I. K. Sethi and G. P. R. Sarvarayudu, "Hierarchical classifier design using mutual information,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-4, pp. 441-445, 1981.
[27] Q. Y. Shi, "A method for the design of binary tree classifiers," inProc. IEEE Conf. Image Processing and Pattern Recognition, Aug. 1981, pp. 21-26.
[28] J. Sklansky, "Locally trained piecewise linear classifiers,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-2, pp. 101-111, 1980.
[29] C. Y. Suen and Q. R. Wang, "Large tree classifier with heuristic search and global training,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-9, pp. 91-101, 1987.
[30] P.H. Swain and H. Hauska, "The decision tree classifier: design and potential,"IEEE Trans. Geosci. Electron., vol. GE-15, pp. 142-147, 1977.
[31] J. Taylor, P. H. Bartels, M. Bibbo, and G. L. Wied, "Automated hierarchic decision structures for multiple category cell classification by TICAS,"Acta Cytol., vol. 22, p. 4, 1978.
[32] Q. R. Wang and C. Y. Suen, "Analysis and design of decision tree based on entropy reduction and its application to large character set recognition,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-6, pp. 406-417, July 1984.
[33] K.C. You and K.S. Fu, "An approach to the design of a linear binary tree classifier," inProc. Symp. Machine Processing of Remotely Sensed Data, Purdue Univ., West Lafayette, IN, 1976, pp. 3A-10.

Index Terms:
pattern recognition; estimation theory; Bayes methods; pruning algorithm; classification tree design; right-sized trees; stopping rules; terminal nodes; iterative method; waveform recognition problem; Bayes methods; estimation theory; iterative methods; pattern recognition; trees (mathematics)
Citation:
S.B. Gelfand, C.S. Ravishankar, E.J. Delp, "An Iterative Growing and Pruning Algorithm for Classification Tree Design," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 2, pp. 163-174, Feb. 1991, doi:10.1109/34.67645
Usage of this product signifies your acceptance of the Terms of Use.