The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - Nov. (2012 vol.24)
pp: 2052-2064
Qinghua Hu , Harbin Institute of Technology, Harbin
Xunjian Che , Harbin Institute of Technology, Harbin
Lei Zhang , The Hong Kong Polytechnic University, Hong Kong
David Zhang , The Hong Kong Polytechnic University, Hong Kong
Maozu Guo , Harbin Institute of Technology, Harbin
Daren Yu , Harbin Institute of Technology, Harbin
ABSTRACT
In many decision making tasks, values of features and decision are ordinal. Moreover, there is a monotonic constraint that the objects with better feature values should not be assigned to a worse decision class. Such problems are called ordinal classification with monotonicity constraint. Some learning algorithms have been developed to handle this kind of tasks in recent years. However, experiments show that these algorithms are sensitive to noisy samples and do not work well in real-world applications. In this work, we introduce a new measure of feature quality, called rank mutual information (RMI), which combines the advantage of robustness of Shannon's entropy with the ability of dominance rough sets in extracting ordinal structures from monotonic data sets. Then, we design a decision tree algorithm (REMT) based on rank mutual information. The theoretic and experimental analysis shows that the proposed algorithm can get monotonically consistent decision trees, if training samples are monotonically consistent. Its performance is still good when data are contaminated with noise.
INDEX TERMS
Entropy, Decision trees, Mutual information, Rough sets, Algorithm design and analysis, Robustness, Noise measurement, decision tree, Monotonic classification, rank entropy, rank mutual information
CITATION
Qinghua Hu, Xunjian Che, Lei Zhang, David Zhang, Maozu Guo, Daren Yu, "Rank Entropy-Based Decision Trees for Monotonic Classification", IEEE Transactions on Knowledge & Data Engineering, vol.24, no. 11, pp. 2052-2064, Nov. 2012, doi:10.1109/TKDE.2011.149
REFERENCES
[1] J. Wallenius, J.S. Dyer, P.C. Sishburn, R.E. Steuer, S. Zionts, and K. Deb, "Multiple Criteria Decision Making, Multiattribute Utility Theory: Recent Accomplishments and What Lies Ahead," Management Science, vol. 54, no. 7, pp. 1336-1349, 2008.
[2] B. Zhao, F. Wang, and C.S. Zhang, "Block-Quantized Support Vector Ordinal Regression," IEEE Trans. Neural Networks, vol. 20, no. 5, pp. 882-890, May 2009.
[3] B.Y. Sun et al., "Kernel Discriminant Learning for Ordinal Regression," IEEE Trans. Knowledge and Data Engineering, vol. 22, no. 6, pp. 906-910, June 2010.
[4] C. Zopounidis and M. Doumpos, "Multicriteria Classification and Sorting Methods: A Literature Review," European J. Operational Research, vol. 138, pp. 229-246, 2002.
[5] R. Potharst and A.J. Feelders, "Classification Trees for Problems with Monotonicity Constraints," ACM SIGKDD Explorations Newsletter, vol. 4, no. 1, pp. 1-10, 2002.
[6] A. Ben-David, L. Sterling, and Y.H. Pao, "Learning and Classification of Monotonic Ordinal Concepts," Computational Intelligence, vol. 5, pp. 45-49, 1989.
[7] A. Ben-David, "Automatic Generation of Symbolic Multiattribute Ordinal Knowledge-Based DSSs: Methodology and Applications," Decision Sciences, vol. 23, pp. 1357-1372, 1992.
[8] E. Frank and M. Hall, "A Simple Approach to Ordinal Classification," Proc. 12th European Conf. Machine Learning, pp. 145-156, 2001.
[9] J.P. Costa and J.S. Cardoso, "Classification of Ordinal Data Using Neural Networks," Proc. 16th European Conf. Machine Learning, pp. 690-697, 2005.
[10] J.R. Quinlan, "Induction of Decision Trees," Machine Learning, vol. 1, no. 1, pp. 81-106, 1986.
[11] J.R. Quinlan, C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
[12] L. Breiman et al., Classification and Regression Trees. Chapman and Hall, 1993.
[13] J. Mingers, "An Empirical Comparison of Selection Measures for Decision-Tree Induction," Machine Learning, vol. 3, no. 4, pp. 319-342, 1989.
[14] A. Ben-David, "Monotonicity Maintenance in Information-Theoretic Machine Learning Algorithms," Machine Learning, vol. 19, pp. 29-43, 1995.
[15] R. Potharst and J.C. Bioch, "Decision Trees for Ordinal Classification," Intelligent Data Analysis, vol. 4, pp. 97-111, 2000.
[16] K. Cao-Van and B.D. Baets, "Growing Decision Trees in an Ordinal Setting," Int'l J. Intelligent Systems, vol. 18, pp. 733-750, 2003.
[17] F. Xia, W.S. Zhang, F.X. Li, and Y.W. Yang, "Ranking with Decision Tree," Knowledge and Information Systems, vol. 17, pp. 381-395, 2008.
[18] W. Kotlowski and R. Slowinski, "Rule Learning with Monotonicity Constraints," Proc. 26th Ann. Int'l Conf. Machine Learning, pp. 537-544, 2009.
[19] A. Jimnez, F. Berzal, and J.-C. Cubero, "POTMiner: Mining Ordered, Unordered, and Partially-Ordered Trees," Knowledge and Information Systems, vol. 23, no. 5, pp. 199-224, 2010.
[20] S. Greco, B. Matarazzo, and R. Slowinski, "Rough Approximation of a Preference Relation by Dominance Relations," European J. Operational Research, ICS Research Report 16/96, vol. 117, pp. 63-83, 1999.
[21] S. Greco, B. Matarazzo, and R. Slowinski, "Rough Sets Methodology for Sorting Problems in Presence of Multiple Attributes and Criteria," European J. Operational Research, vol. 138, pp. 247-259, 2002.
[22] S. Greco, B. Matarazzo, and R. Slowinski, "Rough Approximation by Dominance Relations," Int'l J. Intelligent Systems, vol. 17, pp. 153-171, 2002.
[23] J.W.T. Lee and E.C.C. Tsang, "Rough Sets and Ordinal Reducts," Soft Computing, vol. 10, pp. 27-33, 2006.
[24] Q.H. Hu, D.R. Yu, and M.Z. Guo, "Fuzzy Preference-Based Rough Sets," Information Sciences, vol. 180, no. 10, pp. 2003-2022, 2010.
[25] A. Ben-David, L. Sterling, and T. Tran, "Adding Monotonicity to Learning Algorithms May Impair Their Accuracy," Expert Systems with Applications, vol. 36, pp. 6627-6634, 2009.
[26] J.S. Dyer, P.C. Fishburn, R.E. Steuer, J. Wallenius, and S. Zionts, "Multiple Criteria Decision Making, Multiattribute Utility Theory: The Next Ten Years," Management Science, vol. 38, pp. 645-654, 1992.
[27] Q.H. Hu, M.Z. Guo, D.R. Yu, and J.F. Liu, "Information Entropy for Ordinal Classification," Science in China Series F: Information Sciences, vol. 53, no. 6, pp. 1188-1200, 2010.
[28] S. Greco, B. Matarazzo, R. Slowinski, and J. Stefanowski, "Variable Consistency Model of Dominance-Based Rough Sets Approach," Proc. Second Int'l Conf. Rough Sets and Current Trends in Computing (RSCTC '00), pp. 170-181, 2001.
[29] B. Chandra and P.P. Varghese, "Fuzzy SLIQ Decision Tree Algorithm," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 38, no. 5, pp. 1294-1301, Oct. 2008.
[30] H.W. Hu, Y.L. Chen, and K. Tang, "Dynamic Discretization Approach for Constructing Decision Trees with a Continuous Label," IEEE Trans. Knowledge and Data Eng., vol. 21, no. 11, pp. 1505-1514, Nov. 2009.
[31] D. Hush and R. Porter, "Algorithms for Optimal Dyadic Decision Trees," Machine Learning, vol. 80, no. 1, pp. 85-107, 2010.
[32] R.V. Kamp, A.J. Feelders, and N. Barile, "Isotonic Classification Trees," Proc. Eight Int'l Symp. Intelligent Data Analysis, pp. 405-416, 2009.
[33] A.J. Feelders and M. Pardoel, "Pruning for Monotone Classification Trees," Proc. Fifth Int'l Symp. Intelligent Data Analysis, pp. 1-12, 2003.
[34] D. Cai, "An Information-Theoretic Foundation for the Measurement of Discrimination Information," IEEE Trans. Knowledge and Data Eng., vol. 22, no. 9, pp. 1262-1273, Sept. 2010.
[35] Y.H. Qian, C.Y. Dang, J.Y. Liang, and D.W. Tang, "Set-Valued Ordered Information Systems," Information Sciences, vol. 179, no. 16, pp. 2809-2832, 2009.
[36] J.C. Bioch and V. Popova, "Rough Sets and Ordinal Classification," Proc. 12th Belgian-Dutch Artificial Intelligence Conf. (BNAIC '00), pp. 85-92, 2000.
[37] J.C. Bioch and V. Popova, "Labelling and Splitting Criteria for Monotone Decision Trees," Proc. 12th Belgian-Dutch Conf. Machine Learning (BENELEARN '02), pp. 3-10, 2002.
[38] J.C. Bioch and V. Popova, "Monotone Decision Trees and Noisy Data," Proc. 14th Belgian-Dutch Conf. Artificial Intelligence (BNAIC '02), pp. 19-26, 2002.
[39] V. Popova, "Knowledge Discovery and Monotonicity," PhD thesis, Erasmus Univ., 2004.
[40] A. Feelders, "Monotone Relabeling in Ordinal Classification," Proc. IEEE Int'l Conf. Data Mining (ICDM '10), pp. 803-808, 2010.
[41] C.-V. Kim, "Supervised Ranking from Semantics to Algorithms," PhD thesis, Ghent Univ., 2003.
628 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool