Subscribe

Issue No.05 - May (2010 vol.22)

pp: 624-638

Suyun Zhao , Hong Kong Polytechnic University, Hong Kong and Hebei University, Baoding

Eric C.C. Tsang , Hong Kong Polytechnic University, Hong Kong

Degang Chen , North China Electric Power University, Beijing

XiZhao Wang , Hebei University, Baoding

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2009.118

ABSTRACT

The fuzzy-rough set (FRS) methodology, as a useful tool to handle discernibility and fuzziness, has been widely studied. Some researchers studied on the rough approximation of fuzzy sets, while some others focused on studying one application of FRS: attribute reduction (i.e., feature selection). However, constructing classifier by using FRS, as another application of FRS, has been less studied. In this paper, we build a rule-based classifier by using one generalized FRS model after proposing a new concept named as “consistence degree” which is used as the critical value to keep the discernibility information invariant in the processing of rule induction. First, we generalized the existing FRS to a robust model with respect to misclassification and perturbation by incorporating one controlled threshold into knowledge representation of FRS. Second, we propose a concept named as “consistence degree” and by the strict mathematical reasoning, we show that this concept is reasonable as a critical value to reduce redundant attribute values in database. By employing this concept, we then design a discernibility vector to develop the algorithms of rule induction. The induced rule set can function as a classifier. Finally, the experimental results show that the proposed rule-based classifier is feasible and effective on noisy data.

INDEX TERMS

Knowledge-based systems, fuzzy-rough hybrids, rule-based classifier, IF-THEN rule.

CITATION

Suyun Zhao, Eric C.C. Tsang, Degang Chen, XiZhao Wang, "Building a Rule-Based Classifier—A Fuzzy-Rough Set Approach",

*IEEE Transactions on Knowledge & Data Engineering*, vol.22, no. 5, pp. 624-638, May 2010, doi:10.1109/TKDE.2009.118REFERENCES

- [1] Z. Pawlak and A. Skowron, "Rough Sets: Some Extensions,"
Information Sciences, vol. 177, pp. 28-40, 2007.- [2] L.A. Zadeh, "Fuzzy Sets,"
Information and Control, vol. 8, pp. 338-353, 1965.- [3] D. Dubois and H. Prade, "Rough Fuzzy Sets and Fuzzy Rough Sets,"
Int'l J. General Systems, vol. 17, pp. 191-208, 1990.- [4] D. Dubois and H. Prade, "Putting Rough Sets and Fuzzy Sets Together,"
Intelligent Decision Support, Handbook of Applications and Advances of the Rough Sets Theory, R. Slowinski, ed., pp. 203-232, Kluwer Academic Publishers, 1992.- [5] J.S. Mi and W.X. Zhang, "An Axiomatic Characterization of a Fuzzy Generalization of Rough Sets,"
Information Sciences, vol. 160, pp. 235-249, 2004.- [6] W.Z. Wu, J.S. Mi, and W.X. Zhang, "Generalized Fuzzy Rough Sets,"
Information Sciences, vol. 151, pp. 263-282, 2003.- [7] W.Z. Wu and W.X. Zhang, "Constructive and Axiomatic Approaches of Fuzzy Approximation Operators,"
Information Sciences, vol. 159, pp. 233-254, 2004.- [8] D.G. Chen, X.Z. Wang, D.S. Yeung, and E.C.C. Tsang, "Rough Approximations on a Complete Completely Distributive Lattice with Applications to Generalized Rough Sets,"
Information Sciences, vol. 176, pp. 1829-1848, 2006.- [9] D.S. Yeung, D.G. Chen, E.C.C. Tsang, J.W.T. Lee, and X.Z. Wang, "On the Generalization of Fuzzy Rough Sets,"
IEEE Trans. Fuzzy Systems, vol. 13, no. 3, pp. 343-361, June 2005.- [10] N.N. Morsi and M.M Yakout, "Axiomatics for Fuzzy Rough Sets,"
Fuzzy Sets and Systems, vol. 100, pp. 327-342, 1998.- [11] A.M. Radzikowska and E.E. Kerre, "A Comparative Study of Fuzzy Rough Sets,"
Fuzzy Sets and Systems, vol. 126, pp. 137-155, 2002.- [12] M.D. Cock, C. Cornelis, and E.E. Kerre, "Fuzzy Rough Sets: The Forgotten Step,"
IEEE Trans. Fuzzy Systems, vol. 15, no. 1, pp. 121-130, Feb. 2007.- [13] C. Cornelis, M.D. Cock, and A.M. Radzikowska, "Fuzzy Rough Sets: From Theory into Practice,"
Handbook of Granular Computing, W. Pedrycz, A. Skowron, and V. Kreinovich, eds., Springer-Verlag, 2008.- [14] R. Slowinski and D. Vanderpooten, "A Generalized Definition of Rough Approximations Based on Similarity,"
IEEE Trans. Knowledge and Data Eng., vol. 12, no. 2, pp. 331-336, Mar./Apr. 2000.- [15] J.M.F. Salido and S. Murakami, "Rough Set Analysis of a General Type of Fuzzy Data Using Transitive Aggregations of Fuzzy Similarity Relations,"
Fuzzy Sets and Systems, vol. 139, pp. 635-660, 2003.- [16] Y.Y. Yao, "Combination of Rough and Fuzzy Sets Based on Alpha-Level Sets,"
Rough Sets and Data Mining: Analysis for Imprecise Data, T.Y. LIN and N. Cercone, eds., pp. 301-321, Kluwer Academic Publishers, 1997.- [17] Q. Hu, Y. Daren, and X. Zongxia, "Fuzzy Probabilistic Approximation Spaces and Their Information Measures,"
IEEE Trans. Fuzzy Systems, vol. 14, no. 2, pp. 191-201, Apr. 2006.- [18] R. Jensen and Q. Shen, "Fuzzy-Rough Attributes Reduction with Application to Web Categorization,"
Fuzzy Sets and Systems, vol. 141, pp. 469-485, 2004.- [19] Q. Shen and R. Jensen, "Selecting Informative Features with Fuzzy-Rough Sets and Its Application for Complex Systems Monitoring,"
Pattern Recognition, vol. 37, pp. 1351-1363, 2004.- [20] R. Jensen and Q. Shen, "Semantics-Preserving Dimensionality Reduction: Rough and Fuzzy-Rough-Based Approaches,"
IEEE Trans. Knowledge and Data Eng., vol. 16, no. 12, pp. 1457-1471, Dec. 2004.- [21] R. Jensen and Q. Shen, "Fuzzy-Rough Sets Assisted Attribute Selection,"
IEEE Trans. Fuzzy Systems, vol. 15, no. 1, pp. 73-89, Feb. 2007.- [22] R. Jensen and Q. Shen, "New Approaches to Fuzzy-Rough Feature Selection,"
IEEE Trans. Fuzzy Systems, vol. 17, no. 4, pp. 824-838, Aug. 2009.- [23] E.C.C. Tsang, D.G. Chen, D.S. Yeung, X.Z. Wang, and J.W.T. Lee, "Attribute Reduction Using Fuzzy Rough Sets,"
IEEE Trans. Fuzzy Systems, vol. 16, no. 5, pp. 1130-1141, 2008.- [24] D.G. Chen, E.C.C. Tsang, and S.Y. Zhao, "Attributes Reduction with Fuzzy Rough Sets,"
Proc. 2007 IEEE Int'l Conf. Systems, Man, and Cybernetics, vol. 1, pp. 486-491, 2007.- [25] Q.H. Hu, D.R. Yu, and Z.X. Xie, "Information-Preserving Hybrid Data Reduction Based on Fuzzy-Rough Techniques,"
Pattern Recognition Letters, vol. 27, pp. 414-423, 2006.- [26] Q.H. Hu, D.R. Yu, Z.X. Xie, and X.D. Li, "EROS: Ensemble Rough Subspaces,"
Pattern Recognition, vol. 40, pp. 3728-3739, 2007.- [27] Q.H. Hu, Z.X. Xie, and D.R. Yu, "Hybrid Attribute Reduction Based on a Novel Fuzzy-Rough Model and Information Granulation,"
Pattern Recognition, vol. 40, pp. 3509-3521, 2007.- [28] D. Slezak, "Approximate Reducts in Decision Tables,"
Proc. Sixth Int'l Conf. Information Processing and Management of Uncertainty in Knowledge-Based Systems, pp. 1159-1164, 1996.- [29] D. Slezak, "Approximate Entropy Reducts,"
Fundamenta Informaticae, vol. 53, pp. 365-390, 2002.- [30] H.S. Nguyen and D. Slezak, "Approximate Reducts and Association Rules—Corresponding and Complexity Results,"
Proc. Int'l Workshop New Directions in Rough Sets, Data Mining, and Granular-Soft Computing (RSFDGrC '99), N. Zhong, A. Skowron, and S. Ohsuga, eds., pp. 137-145, 1999.- [31] C. Cornelis, G.H. Martín, R. Jensen, and D. Slezak, "Feature Selection with Fuzzy Decision Reducts,"
Proc. Third Int'l Conf. Rough Sets and Knowledge Technology (RSKT '08), 2008.- [32] E.C.C. Tsang, S.Y. Zhao, and J.W.T. Lee, "Rule Induction Based on Fuzzy Rough Sets,"
Proc. 2007 Int'l Conf. Machine Learning and Cybernetics, vol. 5, pp. 3028-3033, Aug. 2007.- [33] C. Cornelis and R. Jensen, "A Noise-Tolerant Approach to Fuzzy-Rough Feature Selection,"
Proc. 17th Int'l Conf. Fuzzy System (FUZZ-IEEE '08), 2008.- [34] R.B. Bhatt and M. Gopal, "On Fuzzy Rough Sets Approach to Feature Selection,"
Pattern Recognition Letters, vol. 26, pp. 1632-1640, 2005.- [35] Q. Shen and A. Chouchoulas, "A Rough-Fuzzy Approach for Generating Classification Rules Pattern Recognition,"
Pattern Recognition, vol. 35, no. 11, pp. 2425-2438, 2002.- [36] S.K. Pal and P. Mitra, "Case Generation Using Rough Sets with Fuzzy Representation,"
IEEE Trans. Knowledge and Data Eng., vol. 16, no. 3, pp. 292-300, Mar. 2004.- [37] Y. Li, S.C.K. Shiu, and S.K. Pal, "Combining Feature Reduction and Case Selection in Building CBR Classifiers,"
IEEE Trans. Knowledge and Data Eng., vol. 18, no. 2, pp. 415-429, Mar. 2006.- [38] Q.H. Hu, D.R. Yu, and Z.X. Xie, "Neighborhood Classifiers,"
Expert Systems with Applications, vol. 34, pp. 866-876, 2008.- [39] G.Q. Cao, S.C.K. Shiu, and X.Z. Wang, "A Fuzzy-Rough Approach for the Maintenance of Distributed Case-Based-Reasoning Systems,"
Soft Computing, vol. 7, no. 8, pp. 491-499, 2003.- [40] M. Sarkar, "Fuzzy-Rough Nearest Neighbor Algorithms in Classification,"
Fuzzy Sets and Systems, vol. 158, no. 19, pp. 2134-2152, 2007.- [41] R.B. Bhatt and M. Gopal, "FRCT: Fuzzy-Rough Classification Trees,"
Pattern Analysis and Applications, vol. 11, no. 1, pp. 73-88, 2008.- [42] R. Yasdi, "Learning Classification Rules from Database in the Context of Knowledge Acquisition and Representation,"
IEEE Trans. Knowledge and Data Eng., vol. 3, no. 3, pp. 293-306, Sept. 1991.- [43] Y.C. Tsai, C.H. Cheng, and J.R. Chang, "Entropy-Based Fuzzy Rough Classification Approach for Extracting Classification Rules,"
Expert Systems with Applications, vol. 31, no. 2, pp. 436-443, 2006.- [44] E.C.C. Tsang and S.Y. Zhao, "Decision Table Reduction in KDD: Fuzzy Rough Based Approach, Transaction on Rough Sets,"
Lecture Notes in Computer Sciences, vol. 5946, pp. 177-188, 2010.- [45] X.Z. Wang, E.C.C. Tsang, S.Y. Zhao, D.G. Chen, and D.S. Yeung, "Learning Fuzzy Rules from Fuzzy Samples Based on Rough Set Technique,"
Information Sciences, vol. 177, no. 20, pp. 4493-4514, 2007.- [46] S.Y. Zhao, E.C.C. Tsang, and D.G. Chen, "The Model of Fuzzy Variable Precision Rough Sets,"
IEEE Trans. Fuzzy Systems, vol. 17, no. 2, pp. 451-467, Apr. 2009.- [47] A. Skowron and C. Rauszer, "The Discernibility Matrices and Functions in Information Systems,"
Intelligent Decision Support—Handbook of Applications and Advances of the Rough Sets Theory, R. Slowinski, ed., pp. 331-362, Kluwer Academic Publishers, 1992.- [48] Y.Y. Yao, Y. Zhao, and J. Wang, "On Reduct Construction Algorithms,"
Proc. First Int'l Conf. Rough Sets and Knowledge Technology (RSKT '06), pp. 297-304, July 2006.- [49] http://www.ics.uci.edu/∼mlearnMLRepository.html , 2008.
- [50] S.R. Safavian and D. Landgrebe, "A Survey of Decision Tree Classifier Methodology,"
IEEE Trans. Systems, Man Cybernetics, vol. 21, no. 3, pp. 660-674, May/June 1991.- [51] Y.F. Yuan and M.J. Shaw, "Introduction of Fuzzy Decision Tree,"
Fuzzy Sets and Systems, vol. 69, pp. 125-139, 1995.- [52] J.R. Quinlan, See5, Release 1.13, http:/www.rulequest.com/, 2009.
- [53] J.C. Bezdek and J.O. Harris, "Fuzzy Partitions and Relations: An Axiomatic Basis of Clustering,"
Fuzzy Sets and Systems, vol. 84, pp. 143-153, 1996.- [54] T. Sudkamp, "Similarity, Interpolation, and Fuzzy Rule Construction,"
Fuzzy Sets and Systems, vol. 58, pp. 73-86, 1993.- [55] G.J. Klir and B. Yuan,
Fuzzy Logic: Theory and Applications. Prentice-Hall, 1995.- [56] W. Zhu and F.Y. Wang, "On Three Types of Covering-Based Rough Sets,"
IEEE Trans. Knowledge and Data Eng., vol. 19, no. 8, pp. 1131-1144, Aug. 2007.- [57] S.H. Nguyen and H.S. Nguyen, "Some Effective Algorithms for Rough Set Methods,"
Proc. Conf. Int'l Processing and Management of Uncertainty in Knowledge Based Systems, pp. 1451-1456, 1996. |