This Article 
 Bibliographic References 
 Add to: 
Extracting Takagi-Sugeno Fuzzy Rules with Interpretable Submodels via Regularization of Linguistic Modifiers
August 2009 (vol. 21 no. 8)
pp. 1191-1204
Shang-Ming Zhou, De Montfort University, Leicester
John Q. Gan, University of Essex, Colchester
In this paper, a method for constructing Takagi-Sugeno (TS) fuzzy system from data is proposed with the objective of preserving TS submodel comprehensibility, in which linguistic modifiers are suggested to characterize the fuzzy sets. A good property held by the proposed linguistic modifiers is that they can broaden the cores of fuzzy sets while contracting the overlaps of adjoining membership functions (MFs) during identification of fuzzy systems from data. As a result, the TS submodels identified tend to dominate the system behaviors by automatically matching the global model (GM) in corresponding subareas, which leads to good TS model interpretability while producing distinguishable input space partitioning. However, the GM accuracy and model interpretability are two conflicting modeling objectives, improving interpretability of fuzzy models generally degrades the GM performance of fuzzy models, and vice versa. Hence, one challenging problem is how to construct a TS fuzzy model with not only good global performance but also good submodel interpretability. In order to achieve a good tradeoff between GM performance and submodel interpretability, a regularization learning algorithm is presented in which the GM objective function is combined with a local model objective function defined in terms of an extended index of fuzziness of identified MFs. Moreover, a parsimonious rule base is obtained by adopting a QR decomposition method to select the important fuzzy rules and reduce the redundant ones. Experimental studies have shown that the TS models identified by the suggested method possess good submodel interpretability and satisfactory GM performance with parsimonious rule bases.

[1] T. Takagi and M. Sugeno, “Fuzzy Identification of Systems and Its Applications to Modeling and Control,” IEEE Trans. Systems, Man, and Cybernetics, vol. 15, no. 1, pp. 116-132, 1985.
[2] J.-S.R. Jang, “ANFIS: Adaptive-Network-Based Fuzzy Inference System,” IEEE Trans. Systems, Man, and Cybernetics, vol. 23, no. 3, pp. 665-685, 1993.
[3] L.-X. Wang, Adaptive Fuzzy Systems and Control. Prentice Hall, 1994.
[4] C.J. Harris, X. Hong, and J.Q. Gan, Adaptive Modeling, Estimation and Fusion from Data—A Neurofuzzy Approach. Springer, 2002.
[5] A. Gegov, Complexity Management in Fuzzy Systems—A Rule Base Compression Approach. Springer, 2007.
[6] S. Guillaume, “Designing Fuzzy Inference Systems from Data: AnInterpretability-Oriented Review,” IEEE Trans. Fuzzy Systems, vol. 9, no. 3, pp. 426-443, 2001.
[7] R. Alcalá, J. Alcalá-Fdez, M.J. Gacto, and F. Herrera, “On the Use of Multiobjective Genetic Algorithms to Improve the Accuracy-Interpretability Trade-Off of Fuzzy Rule-Based Systems,” Multi-objective Evolutionary Algorithms for Knowledge Discovery from Data Base, A. Ghosh, S. Dehuri, and S. Ghosh, eds., vol. 98, Springer Verlag, 2008.
[8] J.R. Cano, F. Herrera, and M. Lozano, “Evolutionary Stratified Training Set Selection for Extracting Classification Rules with Trade-Off Precision-Interpretability,” Data and Knowledge Eng., vol. 60, pp. 90-108, 2007.
[9] J. Yen, L. Wang, and C.W. Gillespie, “Improving the Interpretability of TSK Fuzzy Models by Combining Global Learning and Local Learning,” IEEE Trans. Fuzzy Systems, vol. 6, no. 4, pp. 530-537, 1998.
[10] S.-M. Zhou and J.Q. Gan, “Low-Level Interpretability and High-Level Interpretability: A Unified View of Data-Driven Interpretable Fuzzy System Modeling,” Fuzzy Sets and Systems, vol. 159, no. 23, pp. 3091-3131, 2008.
[11] R. Jensen and Q. Shen, “Semantics-Preserving Dimensionality Reduction: Rough and Fuzzy-Rough-Based Approaches,” IEEE Trans. Knowledge and Data Eng., vol. 16, no. 12, pp. 1457-1471, Dec. 2004.
[12] Y. Jin, “Fuzzy Modeling of High-Dimensional Systems: Complexity Reduction and Interpretability Improvement,” IEEE Trans. Fuzzy Systems, vol. 8, no. 2, pp. 212-221, 2000.
[13] A. Hamilton-Wright and D.W. Stashuk, “Transparent Decision Support Using Statistical Reasoning and Fuzzy Inference,” IEEE Trans. Knowledge and Data Eng., vol. 18, no. 8, pp. 1125-1137, Aug. 2006.
[14] H. Roubos and M. Setnes, “Compact and Transparent Fuzzy Models and Classifiers through Iterative Complexity Reduction,” IEEE Trans. Fuzzy Systems, vol. 9, no. 4, pp. 516-524, 2001.
[15] W. Pedrycz, J.C. Bezdek, R.J. Hathaway, and G.W. Rogers, “Two Nonparametric Models for Fusing Heterogeneous Fuzzy Data,” IEEE Trans. Fuzzy Systems, vol. 6, no. 3, pp. 411-425, 1998.
[16] J.V. de Oliveira, “Semantic Constraints for Membership Function Optimization,” IEEE Trans. Systems, Man, and Cybernetics—Part A, vol. 29, no. 1, pp. 128-138, 1999.
[17] J. Espinosa and J. Vandewalle, “Constructing Fuzzy Models with Linguistic Integrity from Numerical Data-AFRELI Algorithm,” IEEE Trans. Fuzzy Systems, vol. 8, no. 5, pp. 591-600, 2000.
[18] J. Victor and A. Dourado, “On-Line Interpretability by Fuzzyrule-Base Simplification and Reduction,” Proc. European Symp. Intelligent Technologies (EUNITE), 2004.
[19] H. Ishibuchi and T. Yamamoto, “Fuzzy Rule Selection by Multi-Objective Genetic Local Search Algorithms and Rule Evaluation Measures in Data Mining,” Fuzzy Sets and Systems, vol. 141, no. 1, pp. 59-88, 2004.
[20] F. Hoppner and F. Klawonn, “Obtaining Interpretable Fuzzy Models from Fuzzy Clustering and Fuzzy Regression,” Proc. Fourth Int'l Conf. Knowledge-Based Intelligent Eng. Systems and Allied Technologies (KES '00), pp. 162-165, 2000.
[21] C.A. Penna-Reyes and M. Sipper, “Fuzzy CoCo: Balancing Accuracy and Interpretability of Fuzzy Models by Means of Coevolution,” Accuracy Improvements in Linguistic Fuzzy Modeling, J. Casillas, O. Cordon, F. Herrera, and L Magdalena, eds., vol. 129 of Studies in Fuzziness and Soft Computing, pp. 119-146, Springer, 2003.
[22] J.C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum, 1981.
[23] R.L. de Mantaras and L. Valverde, “New Results in Fuzzy Clustering Based on the Concept of Indistinguishability Relation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 5, pp. 754-757, May 1988.
[24] I. Gath and A.B. Geva, “Unsupervised Optimal Fuzzy Clustering,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 7, pp. 773-781, July 1989.
[25] F. Hoppner and F. Klawonn, “A New Approach to Fuzzy Partitioning,” Proc. Joint Ninth IFSA World Congress and 20thNAFIPS Int'l Conf. (IFSA '01), pp. 1419-1424, 2001.
[26] J. Abonyi and R. Babuska, “Local and Global Identification and Interpretation of Parameters in Takagi-Sugeno Fuzzy Models,” Proc. IEEE Int'l Conf. Fuzzy Systems (FUZZ-IEEE '00), pp. 835-840, 2000.
[27] T.A. Johansen and R. Babuska, “Multi-Objective Identification of Takagi-Sugeno Fuzzy Models,” IEEE Trans. Fuzzy Systems, vol. 11, no. 6, pp. 847-860, 2003.
[28] J.Q. Gan and C.J. Harris, “Fuzzy Local Linearization and Local Basis Function Expansion in Nonlinear System Modeling,” IEEE Trans. Systems, Man, and Cybernetics—Part B, vol. 29, no. 4, pp.559-565, 1999.
[29] A. Fiordaliso, “A Constrained Takagi-Sugeno Fuzzy System That Allows for Better Interpretation and Analysis,” Fuzzy Sets and Systems, vol. 118, pp. 307-318, 2001.
[30] T.G. Amaral, V.F. Pires, and M.M. Crisóstomo, “An Approach to Improve the Interpretability of Neuro-Fuzzy Systems,” Proc. IEEE Int'l Conf. Fuzzy Systems (FUZZ-IEEE '06), pp. 8502-8509, 2006.
[31] M. Setnes, R. Babuska, U. Kaymak, and H.R. van Nauta Lemke, “Similarity Measures in Fuzzy Rule Base Simplification,” IEEE Trans. Systems, Man and Cybernetics—Part B, vol. 28, no. 3, pp. 376-386, 1998.
[32] H. Wang, S. Kwong, Y. Jin, W. Wei, and K. Man, “Agent-Based Evolutionary Approach to Interpretable Rule-Based Knowledge Extraction,” IEEE Trans. Systems, Man, and Cybernetics—Part C, vol. 29, no. 2, pp. 143-155, 2005.
[33] C. Mencar, G. Castellano, and A.M. Fanelli, “Distinguishability Quantification of Fuzzy Sets,” Information Sciences, vol. 177, pp.130-149, 2007.
[34] H.A. Hefny, “Comments on ‘Distinguishability Quantification of Fuzzy Sets’,” Information Sciences, 2007.
[35] A. Kaufmann, Introduction to the Theory of Fuzzy Subsets. Academic Press, 1975.
[36] G.C. Mouzouris and J.M. Mendel, “Designing Fuzzy Logic Systems for Uncertain Environments Using a Singular-Value-QR Decomposition Method,” Proc. Fifth IEEE Int'l Conf. Fuzzy Systems (FUZZ-IEEE '96), pp. 295-301, 1996.
[37] M. Setnes and R. Babuska, “Rule Base Reduction: Some Comments on the Use of Orthogonal Transforms,” IEEE Trans. Systems, Man, and Cybernetics—Part C, vol. 31, no. 2, pp. 199-206, 2001.
[38] S.-M. Zhou and J.Q. Gan, “Constructing Accurate and Parsimonious Fuzzy Models with Distinguishable Fuzzy Sets Based on an Entropy Measure,” Fuzzy Sets and Systems, vol. 157, no. 8, pp. 1057-1074, 2006.
[39] T. Furuhashi and T. Suzuki, “On Interpretability of Fuzzy Models Based on Conciseness Measure,” Proc. 10th IEEE Int'l Conf. Fuzzy Sets (FUZZ-IEEE), 2001.
[40] A. De Luca and S. Termini, “A Definition of Non-Probabilistic Entropy in the Setting of Fuzzy Set Theory,” Information Control, vol. 20, pp. 301-312, 1972.
[41] J. Yen and L. Wang, “Application of Statistical Information Criteria for Optimal Fuzzy Model Construction,” IEEE Trans. Fuzzy Systems, vol. 6, no. 3, pp. 362-372, 1998.
[42] S.-M. Zhou and J.Q. Gan, “An Unsupervised Kernel Based Fuzzy c-Means Clustering Algorithm with Kernel Normalization,” Int'l J. Computational Intelligence and Applications, vol. 4, no. 4, pp. 355-373, 2004.
[43] M.J. Moody and C.J. Darken, “Fast Learning in Networks of Locally-Tuned Processing Units,” Neural Computation, vol. 1, no. 2, pp. 281-294, 1989.
[44] S. Bittanti and L. Piroddi, “Nonlinear Identification and Control of a Heat Exchanger: A Neural Network Approach,” J. Franklin Inst., vol. 334, no. 1, pp. 135-153, 1997.
[45] N.F. Rulkov, L.S. Tsimring, and H.D.I. Abarbanel, “Tracking Unstable Orbits in Chaos Using Dissipative Feedback Control,” Physical Rev. E, vol. 50, no. 1, pp. 314-324, 1994.

Index Terms:
Interpretability, distinguishability, knowledge extraction, local models, submodels, Takagi-Sugeno fuzzy models, regularization, fuzziness.
Shang-Ming Zhou, John Q. Gan, "Extracting Takagi-Sugeno Fuzzy Rules with Interpretable Submodels via Regularization of Linguistic Modifiers," IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 8, pp. 1191-1204, Aug. 2009, doi:10.1109/TKDE.2008.208
Usage of this product signifies your acceptance of the Terms of Use.