This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
CAIM Discretization Algorithm
February 2004 (vol. 16 no. 2)
pp. 145-153

Abstract—The task of extracting knowledge from databases is quite often performed by machine learning algorithms. The majority of these algorithms can be applied only to data described by discrete numerical or nominal attributes (features). In the case of continuous attributes, there is a need for a discretization algorithm that transforms continuous attributes into discrete ones. This paper describes such an algorithm, called CAIM (class-attribute interdependence maximization), which is designed to work with supervised data. The goal of the CAIM algorithm is to maximize the class-attribute interdependence and to generate a (possibly) minimal number of discrete intervals. The algorithm does not require the user to predefine the number of intervals, as opposed to some other discretization algorithms. The tests performed using CAIM and six other state-of-the-art discretization algorithms show that discrete attributes generated by the CAIM algorithm almost always have the lowest number of intervals and the highest class-attribute interdependency. Two machine learning algorithms, the CLIP4 rule algorithm and the decision tree algorithm, are used to generate classification rules from data discretized by CAIM. For both the CLIP4 and decision tree algorithms, the accuracy of the generated rules is higher and the number of the rules is lower for data discretized using the CAIM algorithm when compared to data discretized using six other discretization algorithms. The highest classification accuracy was achieved for data sets discretized with the CAIM algorithm, as compared with the other six algorithms.

[1] J. Catlett, On Changing Continuous Attributes into Ordered Discrete Attributes Proc. European Working Session on Learning, pp. 164-178, 1991.
[2] C.C. Chan, C. Bartur, and A. Srinivasasn, Determination of Quantization Intervals in Rule Based Model for Dynamic Systems Proc. IEEE Conf. System, Man, and Cybernetics, pp. 1719-1723, 1991.
[3] J.Y. Ching, A.K.C. Wong, and K.C.C. Chan, Class-Dependent Discretization for Inductive Learning from Continuous and Mixed Mode Data IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 7, pp. 641-651, July 1995.
[4] D. Chiu, A. Wong, and B. Cheung, Information Discovery through Hierarchical Maximum Entropy Discretization and Synthesis Knowledge Discovery in Databases, G. Piatesky-Shapiro and W.J. Frowley, ed., MIT Press, 1991.
[5] K.J. Cios and L. Kurgan, Hybrid Inductive Machine Learning Algorithm That Generates Inequality Rules Information Sciences, special issue on soft computing data mining, accepted, 2002.
[6] K.J. Cios and L. Kurgan, Hybrid Inductive Machine Learning: An Overview of CLIP Algorithms New Learning Paradigms in Soft Computing, L.C. Jain and J. Kacprzyk, ed., pp. 276-322, Physica-Verlag (Springer), 2001.
[7] K.J. Cios, W. Pedrycz, and R. Swiniarski, Data Mining Methods for Knowledge Discovery. Kluwer, 1998, http://www.wkap.nl/book.htm0-7923-8252-8 .
[8] P. Clark and T. Niblett, The CN2 Algorithm Machine Learning, vol. 3, pp. 261-283, 1989.
[9] P. Clark and R. Boswell, Rule Induction with CN2: Some Recent Improvements Proc. European Working Session on Learning, 1991.
[10] Data Mining Tools,http://www.rulequest.comsee5-info.html, 2003.
[11] J. Dougherty, R. Kohavi, and M. Sahami, Supervised and Unsupervised Discretization of Continuous Features Proc. 12th Int'l Conf. Machine Learning, pp. 194-202, 1995.
[12] U.M. Fayyad and K.B. Irani, On the Handling of Continuous-Valued Attributes in Decision Tree Generation Machine Learning, vol. 8, pp. 87-102, 1992.
[13] U.M. Fayyad and K.B. Irani, Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning Proc. 13th Int'l Joint Conf. Artificial Intelligence, pp. 1022-1027, 1993.
[14] W. Huang, Discretization of Continuous Attributes for Inductive Machine Learning master's thesis, Dept. Computer Science, Univ. of Toledo, Ohio, 1996.
[15] K.A. Kaufman and R.S. Michalski, Learning from Inconsistent and Noisy Data: The AQ18 Approach Proc. 11th Int'l Symp. Methodologies for Intelligent Systems, 1999.
[16] L. Kurgan and K.J. Cios, Discretization Algorithm that Uses Class-Attribute Interdependence Maximization Proc. 2001 Int'l Conf. Artificial Intelligence (IC-AI-2001), pp. 980-987, 2001.
[17] R. Kerber, ChiMerge: Discretization of Numeric Attributes Proc. Ninth Int'l Conf. Artificial Intelligence (AAAI-91), pp. 123-128, 1992.
[18] Y. Linde, A. Buzo, R.M. Gray, An Algorithm for Vector Quantizer Design IEEE Trans. Comm., vol. 28, no. 1, pp. 84-95, 1980.
[19] H. Liu and R. Setiono, "Feature Selection via Discretization of Numeric Attributes," IEEE Trans. Knowledge and Data Eng., vol. 9, no. 4, July/Aug. 1997.
[20] R.S. Michalski, I. Mozetic, J. Hong, and N. Lavrac, The Multipurpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains Proc. Fifth Nat'l Conf. Artificial Intelligence, pp. 1041-1045, 1986.
[21] A. Paterson and T.B. Niblett, ACLS Manual. Edinburgh: Intelligent Terminals, Ltd, 1987.
[22] B. Pfahringer, Compression-Based Discretization of Continuous Attributes Proc. 12th Int'l Conf. Machine Learning, pp. 456-463, 1995.
[23] J.R. Quinlan, C4.5 Programs for Machine Learning. Morgan-Kaufmann, 1993.
[24] P. Vlachos, StatLib Project Repository,http://lib.stat.cmu.edu/datasetscsb/, 2000.
[25] C.L. Blake and C.J. Merz, UCI Repository of Machine Learning Databases,http://www.ics.uci.edu/~mlearnMLRepository.html , , UC Irvine, Dept. Information and Computer Science, 1998.
[26] J.T. Tou and R.C. Gonzalez, Pattern Recognition Principles. Addison-Wesley, 1874.
[27] A.K.C. Wong and D.K.Y. Chiu, Synthesizing Statistical Knowledge from Incomplete Mixed-Mode Data IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, pp. 796-805, 1987.
[28] A.K.C. Wong and T.S. Liu, Typicality, Diversity and Feature Pattern of an Ensemble IEEE Trans. Computers, vol. 24, pp. 158-181, 1975.
[29] X. Wu, A Bayesian Discretizer for Real-Valued Attributes The Computer J., vol. 39, 1996.

Index Terms:
Supervised discretization, class-attribute interdependency maximization, classification, CLIP4 machine learning algorithm.
Citation:
Lukasz A. Kurgan, Krzysztof J. Cios, "CAIM Discretization Algorithm," IEEE Transactions on Knowledge and Data Engineering, vol. 16, no. 2, pp. 145-153, Feb. 2004, doi:10.1109/TKDE.2004.1269594
Usage of this product signifies your acceptance of the Terms of Use.