This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
R-MINI: An Iterative Approach for Generating Minimal Rules from Examples
September-October 1997 (vol. 9 no. 5)
pp. 709-717

Abstract—Generating classification rules or decision trees from examples has been a subject of intense study in the pattern recognition community, the statistics community, and the machine-learning community of the artificial intelligence area. We pursue a point of view that minimality of rules is important, perhaps above all other considerations (biases) that come into play in generating rules. We present a new minimal rule-generation algorithm called R-MINI (Rule-MINI) that is an adaptation of a well-established heuristic-switching-function-minimization technique, MINI. The main mechanism that reduces the number of rules is repeated application of generalization and specialization operations to the rule set while maintaining completeness and consistency. R-MINI results on some benchmark cases are also presented.

[1] R. Michalski, I. Mozetic, J. Hong, and N. Lavrac, "The Multi-Purpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains," Proc. AAAI, pp. 1,041-1,045, 1986.
[2] P. Clark and T. Niblett, "The CN2 Induction Algorithm," Machine Learning, vol. 3, pp. 261-283, 1989.
[3] S. Clearwater and F. Provost, "RL4: A Tool for Knowledge-Based Induction," Proc. IEEE Conf. Tools Artificial Intelligence, pp. 24-30, 1990.
[4] S. Weiss and N. Indurkhya, "Reduced Complexity Rule Induction," Proc. 12th IJCAI, pp. 678-684, 1991.
[5] J. Quinlan, "Generating Production Rules From Decision Trees," Proc. 10th IJCAI, pp. 304-307, 1987.
[6] G. Pagallo, "Learning DNF by Decision Trees," Proc. 11th IJCAI, pp. 639-644, 1989.
[7] A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, "Occam's Razor," Information Processing Letters, vol. 24, NorthHolland, pp. 377-380, 1987.
[8] A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, "Learnability and the Vapnik-Chervonenkis Dimension," J. ACM, vol. 36, pp. 929-965, 1989.
[9] J. Rissanen, Stochastic Complexity in Statistical Inquiry. World Scientific Series in Computer Science, vol. 15, 1989.
[10] U. Fayyad and K. Irani, "What Should be Minimized in a Decision Tree," Proc. AAAI, pp. 749-754, 1990.
[11] J. Quinlan and R. Rivest, "Inferring Decision Trees Using Minimum Description Length Principle," Information and Computation, vol. 80, pp. 227-248, 1989.
[12] L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees. Wadsworth, 1984.
[13] S. Weiss and C. Kulikowski, Computer Systems That Learn. Morgan Kaufmann, 1991.
[14] J. Dougherty, R. Kohavi, and M. Sahami, "Supervised and Unsupervised Discretization of Continuous Features," Proc. ML, 1995.
[15] S. Hong, "Use of Contextual Information for Feature Ranking and Discretization," IEEE Trans. Knowledge and Data Eng., vol. 9, no. 5, pp. 718-730, Sept./Oct. 1997.
[16] S. Hong, R. Cain, and D. Ostapko, "MINI: A Heuristic Algorithm for Two-Level Logic Minimization," IBM J. Research and Development, vol. 18, no. 5, pp. 443-458, Sept. 1974; also appears in Selected Papers on Logic Synthesis for Integrated Circuit Design, R. Newton, ed., IEEE CS/Press, 1987.
[17] R.K. Brayton, G.D. Hachtel, C.T. McMullen, and A.L. Sangiovanni-Vincintelli, Logic Minimization Algorithms for VLSI Synthesis.Boston: Kluwer Academic, 1984.
[18] J. Holland, K. Holyoak, R. Nisbett, and P. Thorgard, Induction: Process of Inference, Learning, and Discovery. MIT Press, pp. 84-89, 1986.
[19] P. Smyth and R. Goodman, "An Information Theoretic Approach to Rule Induction from Databases," IEEE Trans Knowledge and Data Eng., vol. 4, no. 4, pp. 301-316, Aug. 1992.
[20] C. Apte, S. Hong, S. Prasad, and B. Rosen, "RAMP: Rules Abstraction for Modeling and Prediction," IBM Technical Report RC-20271, 1995.
[21] S. Hong and D. Ostapko, "A Practical Approach to Two Level Minimization of Multi-Valued Logic," Proc. Int'l Symp. Multiple Valued Logic, May 1974
[22] C. Apte and S. Hong, "Predicting Equity Returns from Securities Data with Minimal Rule Generation," Knowledge Discovery in Databases, U. Fayyad, ed. AAAI/MIT Press, 1995.
[23] Machine Learning: An Artificial Intelligence Approach, chapters 3-5, R. Michalski, J. Carbonell, and T. Mitchell, eds. Tioga Press, 1983.
[24] Machine Learning, Neural, and Statistical Classification, D. Michie, D. Spiegelhalter, and C. Taylor, eds. Ellis Horwood, 1994.
[25] M. Mehta, J. Rissanen, and R. Agrawal, "MDL-based Decision Tree Pruning," Proc. KDDM, Aug. 1995.

Index Terms:
Classification, minimal rules, DNF, Occam's razor, generalization, specialization.
Citation:
Se June Hong, "R-MINI: An Iterative Approach for Generating Minimal Rules from Examples," IEEE Transactions on Knowledge and Data Engineering, vol. 9, no. 5, pp. 709-717, Sept.-Oct. 1997, doi:10.1109/69.634750
Usage of this product signifies your acceptance of the Terms of Use.