The Community for Technology Leaders
RSS Icon
Issue No.11 - November (2009 vol.21)
pp: 1505-1514
Hsiao-Wei Hu , National Central University, Chung-Li
Yen-Liang Chen , National Central University, Chung-Li
Kwei Tang , Purdue University, West Lafayette
In traditional decision (classification) tree algorithms, the label is assumed to be a categorical (class) variable. When the label is a continuous variable in the data, two possible approaches based on existing decision tree algorithms can be used to handle the situations. The first uses a data discretization method in the preprocessing stage to convert the continuous label into a class label defined by a finite set of nonoverlapping intervals and then applies a decision tree algorithm. The second simply applies a regression tree algorithm, using the continuous label directly. These approaches have their own drawbacks. We propose an algorithm that dynamically discretizes the continuous label at each node during the tree induction process. Extensive experiments show that the proposed method outperforms the preprocessing approach, the regression tree approach, and several nontree-based algorithms.
Decision trees, data mining, classification.
Hsiao-Wei Hu, Yen-Liang Chen, Kwei Tang, "A Dynamic Discretization Approach for Constructing Decision Trees with a Continuous Label", IEEE Transactions on Knowledge & Data Engineering, vol.21, no. 11, pp. 1505-1514, November 2009, doi:10.1109/TKDE.2009.24
[1] J.R. Cano, F. Herrera, and M. Lozano, “Evolutionary Stratified Training Set Selection for Extracting Classification Rules with Trade Off Precision-Interpretability,” Data & Knowledge Eng., vol. 60, no. 1, pp. 90-108, 2007.
[2] G. Jagannathan and R.N. Wright, “Privacy-Preserving Imputation of Missing Data,” Data & Knowledge Eng., vol. 65, no. 1, pp. 40-56, 2008.
[3] X.B. Li, J. Sweigart, J. Teng, J. Donohue, and L. Thombs, “A Dynamic Programming Based Pruning Method for Decision Trees,” INFORMS J. Computing, vol. 13, pp. 332-344, 2001.
[4] S. Piramuthu, “Feature Selection for Financial Credit-Risk Evaluation Decisions,” INFORMS J. Computing, vol. 11, pp. 258-266, 1999.
[5] F. Bonchi, F. Giannotti, G. Mainetto, and D. Pedreschi, “A Classification-Based Methodology for Planning Audit Strategies in Fraud Detection,” Proc. Fifth ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 175-184, 1999.
[6] M.J.A. Berry and G.S. Linoff, Mastering Data Mining: The Art and Science of Customer Relationship Management. Wiley, 1999.
[7] J.B. Ayers, Handbook of Supply Chain Management, second ed., pp.426-427. Auerbach Publication, 2006.
[8] M.R. Chmielewski and J.W. Grzymala-Busse, “Global Discretization of Continuous Attributes as Preprocessing for Machine Learning,” Proc. Third Int'l Workshop Rough Sets and Soft Computing, pp. 294-301, 1994.
[9] J. Cerquides and R.L. Mantaras, “Proposal and Empirical Comparison of a Parallelizable Distance-Based Discretization Method,” Proc. Third Int'l Conf. Knowledge Discovery and Data Mining, pp. 139-142, 1997.
[10] M.R. Chmielewski and J.W. Grzymala-Busse, “Global Discretization of Continuous Attributes as Preprocessing for Machine Learning,” Int'l J. Approximate Reasoning, vol. 15, pp. 319-331, 1996.
[11] T. Van de Merckt, “Decision Tree in Numerical Attribute Space,” Proc. 13th Int'l Joint Conf. Artificial Intelligence, pp. 1016-1021, 1993.
[12] J. Dougherty, R. Kohavi, and M. Sahami, “Supervised and Unsupervised Discretization of Continuous Features,” Proc. Int'l Conf. Machine Learning, pp. 194-202, 1995.
[13] J. Bogaert, R. Ceulemans, and E.D. Salvador-Van, “Decision Tree Algorithm for Detection of Spatial Processes in Landscape Transformation,” Environmental Management, vol. 33, no. 1, pp.62-73, 2004.
[14] L. Borzemski, “The Use of Data Mining to Predict Web Performance,” Cybernetics and Systems, vol. 37, no. 6, pp. 587-608, 2006.
[15] W. Desheng, “Detecting Information Technology Impact on Firm Performance Using DEA and Decision Tree,” Int'l J. Information Technology and Management, vol. 5, nos. 2/3, pp. 162-174, 2006.
[16] S.A. Gaddam, V.V. Phoha, and K.S. Balagani, “K-Means+ID3: A Novel Method for Supervised Anomaly Detection by Cascading K-Means Clustering and ID3 Decision Tree Learning Methods,” IEEE Trans. Knowledge and Data Eng., vol. 19, no. 3, pp. 345-354, Mar. 2007.
[17] R. Jin and G. Agrawal, “Efficient Decision Tree Construction on Streaming Data,” Proc. Ninth ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 571-576, 2003.
[18] M. Last, “Online Classification of Nonstationary Data Streams,” Intelligent Data Analysis, vol. 6, no. 2, pp. 129-147, 2002.
[19] M. Last, M. Friedman, and A. Kandel, “Using Data Mining for Automated Software Testing,” Int'l J. Software Eng. and Knowledge Eng., vol. 14, no. 4, pp. 369-393, 2004.
[20] L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone, Classification and Regression Trees. Chapman & Hall, 1993.
[21] S. Kramer, “Structural Regression Trees,” Proc. 13th Nat'l Conf. Artificial Intelligence, pp. 812-819, 1996.
[22] J. Yang and J. Stenzel, “Short-Term Load Forecasting with Increment Regression Tree,” Electric Power Systems Research, vol. 76, nos. 9/10, pp. 880-888, June 2006.
[23] C.-W. Hsu, C.-C. Chang, and C.-J. Lin, A Practical Guide to Support Vector Classification, guideguide.pdf, 2003.
[24] S.K. Shevade, S.S. Keerthi, C. Bhattacharyya, and K.R.K. Murthy, “Improvements to the SMO Algorithm for SVM Regression,” IEEE Trans. Neural Networks, vol. 11, no. 5, pp. 1188-1193, Sept. 2000.
[25] J.R. Quinlan, “Introduction of Decision Trees,” Machine Learning, vol. 1, pp. 81-106, 1986.
[26] J.R. Quinlan, C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
[27] P.M. Murphy and D.W. Aha, “UCI Repository of Machine Learning Database,” http://archive.ics.uci.eduml/, 1994.
[28] Y.L. Chen, C.L. Hsu, and S.C. Chou, “Constructing a Multi-Valued and Multi-Labeled Decision Tree,” Expert Systems with Applications, vol. 25, pp. 199-209, 2003.
[29] J. Han and M. Kamber, Data Mining: Concepts and Techniques. Morgan Kaufmann, 2001.
[30] G.V. Kass, “An Exploratory Technique for Investigating Large Quantities of Categorical Data,” Applied Statistics, vol. 29, pp. 119-127, 1980.
[31] W.Y. Loh and Y.S. Shih, “Split Selection Methods for Classification Trees,” Statistica Sinica, vol. 7, pp. 815-840, 1997.
[32] J.R. Quinlan, “Improved Use of Continuous Attributes in C4.5,” Artificial Intelligence, vol. 4, pp. 77-90, 1996.
[33] J. Catlett, “Megainduction: Machine Learning on Very Large Databases,” PhD thesis, Univ. of Sydney, 1991.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool