This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Rough-Fuzzy MLP: Modular Evolution, Rule Generation, and Evaluation
January/February 2003 (vol. 15 no. 1)
pp. 14-25

Abstract—A methodology is described for evolving a Rough-fuzzy multi layer perceptron with modular concept using a genetic algorithm to obtain a structured network suitable for both classification and rule extraction. The modular concept, based on "divide and conquer" strategy, provides accelerated training and a compact network suitable for generating a minimum number of rules with high certainty values. The concept of variable mutation operator is introduced for preserving the localized structure of the constituting knowledge-based subnetworks, while they are integrated and evolved. Rough set dependency rules are generated directly from the real valued attribute table containing fuzzy membership values. Two new indices viz., "certainty" and "confusion" in a decision are defined for evaluating quantitatively the quality of rules. The effectiveness of the model and the rule extraction algorithm is extensively demonstrated through experiments alongwith comparisons.

[1] L.A. Zadeh, “Fuzzy Logic, Neural Networks, and Soft Computing,” Comm. ACM, vol. 37, pp. 77-84, 1994.
[2] Z. Pawlak, Rough Sets—Theoretical Aspects of Reasoning about Data. Kluwer Academic, 1991.
[3] Z. Pawlak, “Rough Sets,” Int'l J. Computer and Information Sciences, vol. 11, pp. 341-356, 1982.
[4] Rough Fuzzy Hybridization: New Trends in Decision Making. S.K. Pal and A. Skowron, eds., Singapore: Springer Verlag, 1999.
[5] M. Banerjee, S. Mitra, and S.K. Pal, Rough Fuzzy MLP: Knowledge Encoding and Classification IEEE Trans. Neural Networks, vol. 9, no. 6, pp. 1203-1216, 1998.
[6] “Rough-Neuro Computing,” Neurocomputing, S.K. Pal, W. Pedrycz, A. Skowron, and R. Swiniarski, eds., vol. 36, pp. 1-4, special issue, 2001.
[7] H.S. Nguyen, M. Szczuka, and D. Slezak, “Neural Network Design: Rough Set Approach to Real-Valued Data,” Proc. First European Symp. Principles of Data Mining and Knowledge Discovery (PKDD '97), pp. 359-366, 1997.
[8] M. Szczuka, “Refining Classifiers with Neural Networks,” Int'l J. Computer and Information Sciences, vol. 16, pp. 39-56, 2001.
[9] L. Han, J.F. Peters, S. Ramanna, and R. Zhai, “Classifying Faults in High Voltage Power Systems: A Rough Fuzzy Neural Computational Approach,” New Directions in Rough Sets, Data Mining, and Granular Soft Computing, pp. 47-54, 1999.
[10] J.F. Peters, A. Skowron, L. Han, and S. Ramanna, “Towards Rough Neural Computing Based on Rough Neural Networks,” Proc. Int'l Conf. Rough Sets and Current Trends in Computing (RSTC '00), pp. 572-579, 2000.
[11] S.K. Pal and D. Bhandari, “Selection of Optimum set of Weights in a Layered Network Using Genetic Algorithms,” Information Sciences, vol. 80, pp. 213-234, 1994.
[12] Genetic Algorithms for Pattern Recognition. S.K. Pal and P.P. Wang, eds. Boca Raton, Fla.: CRC Press, 1996.
[13] A.B. Tickle, R. Andrews, M. Golea, and J. Diederich, "The Truth Will Come to Light: Directions and Challenges in Extracting the Knowledge Embedded within Trained Artificial Neural Networks," IEEE Trans. Neural Networks, vol. 9, no. 6, pp. 1,057-1,068, 1998.
[14] S. Mitra and Y. Hayashi, "Neuro-Fuzzy Rule Generation: Survey in Soft Computing Framework," IEEE Trans. Neural Networks, vol. 11, no. 3, May 2000, pp. 748-768.
[15] B.M. Happel and J.J. Murre, “Design and Evolution of Modular Neural Network Architectures,” Neural Networks, vol. 7, pp. 985-1004, 1994.
[16] L.M. Fu, "Knowledge-Based Connectionism for Revising Domain Theories," IEEE Trans. Systems, Man, and Cybernetics, vol. 23, no. 1, pp. 173-182, 1993.
[17] G.G. Towell and J.W. Shavlik, "The Extraction of Refined Rules from Knowledge-Based Neural Networks," Machine Learning, vol. 13, no. 1, pp. 71-101, 1993.
[18] H. Liu and S.T. Tan, “X2R: A Fast Rule Generator,” Proc. IEEE Int'l Conf. System Man Cybernetics, pp. 215-220, 1995.
[19] S.K. Pal and S. Mitra, Multi-Layer Perceptron, Fuzzy Sets and Classification IEEE Trans. Neural Networks, vol. 3, pp. 683-697, 1992.
[20] S.K. Pal and D.K.D. Majumder, Fuzzy Mathematical Approach to Pattern Recognition. New Delhi, India: Wiley Eastern Ltd., 1986.
[21] A. Skowron and C. Rauszer, “The Discernibility Matrices and Functions in Information Systems,” Intelligent Decision Support, Handbook of Applications and Advances of the Rough Sets Theory, R. Slowinski, ed., Dordrecht: Kluwer Academic, pp. 331-362, 1992.
[22] A. Skowron and J. Stepaniuk, “Decision Rules Based on Discernibility Matrices and Decision Matrices,” Proc. Int'l Workshop Rough Sets and Soft Computing, pp. 602-609, 1994.
[23] Q. Zhao, “A Co-Evolutionary Algorithm for Neural NetworkLearning,” Proc. IEEE Int'l Conf. Neural Networks, pp. 432-437, 1997.
[24] D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, Mass.: Addison-Wesley, 1989.
[25] I.A. Taha and J. Ghosh, “Symbolic Interpretation of Artificial Neural Networks,” IEEE Trans. Knowledge and Data Eng., vol. 11, pp. 448-463, 1999.
[26] R. Andrews, J. Diederich, and A. B. Tickle, “A Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks,” Knowledge-Based Systems, vol. 8, pp. 373-389, 1995.
[27] G.H. Rosenfeld and K. Fitzpatrick-Lins, “Coefficient of Agreement as a Measure of Thematic Classification Accuracy,” Photogrammetric Engineering and Remote SensingIntroduction, vol. 52, pp. 223-227, 1986.
[28] E.L. Lehmann, Testing of Statistical Hypotheses. New York: John Wiley, 1976.
[29] A. Aspin, “Tables for Use in Comparisons Whose Accuracy Involves Two Variances,” Biometrika, vol. 36, pp. 245-271, 1949.
[30] J.R. Quinlan, C4.5: Programs for Machine Learning,San Mateo, Calif.: Morgan Kaufman, 1992.
[31] S. Mitra, S.K. Pal, and P. Mitra, “Data Mining in Soft Computing Framework: A Survey,” IEEE Trans. Neural Networks, vol. 13, pp. 3-14, 2001.
[32] Soft Computing in Case Based Reasoning. S.K. Pal, T.S. Dillon, and D.S. Yeung, eds., London: Springer Verlag, 2001.

Index Terms:
Soft computing, knowledge-based fuzzy networks, rough sets, genetic algorithms, pattern recognition, rule extraction/evaluation, knowledge discovery, data mining.
Citation:
Sankar K. Pal, Sushmita Mitra, Pabitra Mitra, "Rough-Fuzzy MLP: Modular Evolution, Rule Generation, and Evaluation," IEEE Transactions on Knowledge and Data Engineering, vol. 15, no. 1, pp. 14-25, Jan.-Feb. 2003, doi:10.1109/TKDE.2003.1161579
Usage of this product signifies your acceptance of the Terms of Use.