The Community for Technology Leaders
RSS Icon
Issue No.11 - November (2011 vol.23)
pp: 1649-1667
Qinghua Hu , Harbin Institute of Technology, Harbin and The Hong Kong Polytechnic University, Hong Kong
Daren Yu , Harbin Institute of Technology, Harbin
Witold Pedrycz , University of Alberta, Edmonton and Polish Academy of Sciences, Warsaw
Degang Chen , North China Electric Power University, Beijing
Kernel machines and rough sets are two classes of commonly exploited learning techniques. Kernel machines enhance traditional learning algorithms by bringing opportunities to deal with nonlinear classification problems, rough sets introduce a human-focused way to deal with uncertainty in learning problems. Granulation and approximation play a pivotal role in rough sets-based learning and reasoning. However, a way how to effectively generate fuzzy granules from data has not been fully studied so far. In this study, we integrate kernel functions with fuzzy rough set models and propose two types of kernelized fuzzy rough sets. Kernel functions are employed to compute the fuzzy T-equivalence relations between samples, thus generating fuzzy information granules in the approximation space. Subsequently fuzzy granules are used to approximate the classification based on the concepts of fuzzy lower and upper approximations. Based on the models of kernelized fuzzy rough sets, we extend the measures existing in classical rough sets to evaluate the approximation quality and approximation abilities of the attributes. We discuss the relationship between these measures and feature evaluation function ReliefF, and augment the ReliefF algorithm to enhance the robustness of these proposed measures. Finally, we apply these measures to evaluate and select features for classification problems. The experimental results help quantify the performance of the KFRS.
Rough set, fuzzy rough set, kernel, feature evaluation, feature selection.
Qinghua Hu, Daren Yu, Witold Pedrycz, Degang Chen, "Kernelized Fuzzy Rough Sets and Their Applications", IEEE Transactions on Knowledge & Data Engineering, vol.23, no. 11, pp. 1649-1667, November 2011, doi:10.1109/TKDE.2010.260
[1] R.B. Bhatt and M. Gopal, "FRCT: Fuzzy-Rough Classification Trees," Pattern Analysis and Applications, vol. 11, no. 1, pp. 73-88, 2008.
[2] M. Bohanec and I. Bratko, "Trading Accuracy For Simplicity in Decision Trees," Machine Learning, vol. 15, pp. 223-250, 1994.
[3] L. Breiman, Classification and Regression Trees. Chapman & Hall, 1993.
[4] C. Cortes and V. Vapnik, "Support-Vector Networks," Machine Learning, vol. 20, pp. 273-297, 1995.
[5] M. Dash and H. Liu, "Consistency-Based Search in Feature Selection," Artificial Intelligence, vol. 151, nos. 1/2, pp. 155-176, 2003.
[6] D. Dubois and H. Prade, "Rough Fuzzy Sets and Fuzzy Rough Sets," Int'l J. General Systems, vol. 17, nos. 2/3, pp. 191-209, 1990.
[7] R. Duda, P. Hart, and D.G. Stork, Pattern Classification, second ed. John Wiley and Sons, Inc., 2001.
[8] F. Esposito, D. Malerba, and G. Semeraro, "A Comparative Analysis of Methods for Pruning Decision Trees," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 5, pp. 476-491, May 1997.
[9] F. Fernandez-Riverola, F. Diaz, and J.M. Corchado, "Reducing the Memory Size of a Fuzzy Case-Based Reasoning System Applying Rough Set Techniques," IEEE Trans. Systems Man and Cybernetics Part C-Applications and Rev., vol. 37, no. 1, pp. 138-146, Jan. 2007.
[10] M. Genton, "Classes of Kernels for Machine Learning: A Statistics Perspective," J. Machine Learning Research, vol. 2, pp. 299-312, 2001.
[11] A. Gretton, R. Herbrich, A. Smola, O. Bousquet, and B. Schölkopf, "Kernel Methods for Measuring Independence," J. Machine Learning Research, vol. 6, pp. 2075-2129, 2005.
[12] I. Guyon and A. Elisseeff, "An Introduction to Variable and Feature Selection," J. Machine Learning Research, vol. 3, pp. 1157-1182, 2003.
[13] M.A. Hall, "Correlation-Based Feature Subset Selection for Machine Learning," Hamilton, New Zealand, 1998.
[14] A. Hassanien, "Fuzzy Rough Sets Hybrid Scheme for Breast Cancer Detection," Image and Vision Computing, vol. 25, no. 2, pp. 172-183, 2007.
[15] T.P. Hong et al., "Learning a Coverage Set of Maximally General Fuzzy Rules by Rough Sets," Expert Systems with Applications, vol. 19, no. 2, pp. 97-103, 2000.
[16] S.J. Hong, "Use of Contextual Information for Feature Ranking and Discretization," IEEE Trans. Knowledge and Data Eng., vol. 9, no. 5, pp. 718-730, Sep./Oct. 1997.
[17] Q.H. Hu, D.R. Yu, and Z.X. Xie, "Information-Preserving Hybrid Data Reduction Based on Fuzzy-Rough Techniques," Pattern Recognition Letters, vol. 27, no. 5, pp. 414-423, 2006.
[18] Q.H. Hu, Z.X. Xie, and D.R. Yu, "Hybrid Attribute Reduction Based on a Novel Fuzzy-Rough Model and Information Granulation," Pattern Recognition, vol. 40, no. 12, pp. 3509-3521, 2007.
[19] Q.H. Hu, D.R. Yu, and Z.X. Xie, "Neighborhood Classifiers," Expert Systems with Applications, vol. 34, pp. 866-876, 2008.
[20] R. Jensen and Q. Shen, "Fuzzy-Rough Sets Assisted Attribute Selection," IEEE Trans. Fuzzy Systems, vol. 15, no. 1, pp. 73-89, Feb. 2007.
[21] R. Jensen and Q. Shen, "Semantics-Preserving Dimensionality Reduction: Rough and Fuzzy-Rough-Based Approaches," IEEE Trans. Knowledge and Data Eng., vol. 16, no. 12, pp. 1457-1471, Dec. 2004.
[22] K. Kira and L.A. Rendell, "A Practical Approach to Feature Selection," Proc. Ninth Int'l Conf. Machine Learning (ICML '92), D. Sleeman and P. dwards, eds., pp. 249-256. 1992.
[23] E.P. klement, R. Mesiar, and E. Pap, Triangular Norms. Kluwer Academic Publishers, 2001.
[24] I. Kononenko, "Estimating Attributes: Analysis and Extensions of Relief," Machine Learning: ECML-94, L. De Raedt and F. Bergadano, eds., pp. 171-182, Springer Verlag, 1994.
[25] C.F. Lin and S.D. Wang, "Fuzzy Support Vector Machines," IEEE Trans. Neural Networks, vol. 13, no. 2, pp. 464-471, Mar. 2002.
[26] H. Liu and L. Yu, "Toward Integrating Feature Selection Algorithms for Classification and Clustering," IEEE Trans. Knowledge and Data Eng., vol. 17, no. 4, pp. 491-502, Apr. 2005.
[27] J. Mi and W. Zhang, "An Axiomatic Characterization of a Fuzzy Generalization of Rough Sets," Information Sciences, vol. 160, pp. 235-249, 2004.
[28] N.N. Morsi and M.M. Yakout, "Axiomatics for Fuzzy Rough Set," Fuzzy Sets System, vol. 100, pp. 327-342, 1998.
[29] B. Moser, "On the T-Transitivity of Kernels," Fuzzy Sets and Systems, vol. 157, pp. 1787-1796, 2006.
[30] B. Moser, "On Representing and Generating Kernels by Fuzzy Equivalence Relations," J. Machine Learning Research, vol. 7, pp. 2603-2620, 2006.
[31] R. Paredes and E. Vidal, "Learning Weighted Metrics to Minimize Nearest-Neighbor Classification Error," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1100-1110, July 2006.
[32] Z. Pawlak, Rough Sets: Theoretical Aspects of Reasoning About Data. Kluwer Academic Publishers, 1991.
[33] A.M. Radzikowska and E.E. Kerre, "A Comparative Study of Fuzzy Rough Sets," Fuzzy Sets and Systems, vol. 126, pp. 137-155, 2002.
[34] M. Robnik-sikonja and I. Kononenko, "Theoretical and Empirical Analysis of ReliefF and RReliefF," Machine Learning, vol. 53, pp. 23-69, 2003.
[35] B. Scholkopf, A. Smola, and K.-R. Muller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem," Neural Computation, vol. 10, pp. 1299-1319, 1998.
[36] J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge Univ. Press, 2004.
[37] P. Somol, P. Pudil, and J. Kittler, "Fast Branch & Bound Algorithms for Optimal Feature Selection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 900-912, July 2004.
[38] Y.J. Sun, "Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Application," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1035-1051, June 2007.
[39] Y.C. Tsai, C.H. Cheng, and J.R. Chang, "Entropy-Based Fuzzy Rough Classification Approach for Extracting Classification Rules," Expert Systems with Applications, vol. 31, no. 2, pp. 436-443, 2006.
[40] Y.F. Wang, "Mining Stock Price Using Fuzzy Rough Set System," Expert Systems with Applications, vol. 24, no. 1, pp. 13-23, 2003.
[41] X.Z. Wang et al., "Learning Fuzzy Rules from Fuzzy Samples Based on Rough Set Technique," Information Sciences, vol. 177, no. 20, pp. 4493-4514, 2007.
[42] Q. Wu, Y. Ying, and D.-X. Zhou, "Multi-Kernel Regularized Classifiers," J. Complexity, vol. 23, pp. 108 -134, 2007.
[43] W. Wu and W. Zhang, "Constructive and Axiomatic Approaches of Fuzzy Approximation Operators," Information Sciences, vol. 159, pp. 233-254, 2004.
[44] R.R. Yager, "Using Fuzzy Methods to Model Nearest Neighbor Rules," IEEE Trans. Systems, Man, and Cybernetics—Part B: Cybernetics, vol. 32, no. 4, pp. 512-525, Aug. 2002.
[45] D.S. Yeung, D.-G. Chen, E.C.C. Tsang, J.W.T. Lee, and X.-Z Wang, "On the Generalization of Fuzzy Rough Sets," IEEE Trans. Fuzzy Systems, vol. 13, no. 3, pp. 343-361, June 2005.
[46] L. Yu and H. Liu, "Efficient Feature Selection via Analysis of Relevance and Redundancy," J. Machine Learning Research, vol. 5, pp. 1205-1224, 2004.
[47] L.A. Zadeh, "Fuzzy ${\rm Logic} = {\rm Computing}$ with Words," IEEE Trans. Fuzzy Systems, vol. 4, no. 2, pp. 103-111, May 1996.
[48] L.A. Zadeh, "Toward a Theory of Fuzzy Information Granulation and Its Centrality in Human Reasoning and Fuzzy Logic," Fuzzy Sets and Systems, vol. 90, no. 2, pp. 111-127, 1997.
[49] W. Zhu and F.Y. Wang, "On Three Types of Covering-Based Rough Sets," IEEE Trans. Knowledge and Data Eng., vol. 19, no. 8, pp. 1131-1144, Aug. 2007.
[50] W.-Z. Wu., "Attribute Reduction Based on Evidence Theory in Incomplete Decision Systems," Information Sciences, vol. 178, no. 5, pp. 1355-1371, 2008.
[51] Q.H. Hu, D.R. Yu, J. Liu, and C. Wu., "Neighborhood Rough Set Based Heterogeneous Feature Subset Selection," Information Sciences, vol. 178, no. 18, pp. 3577-3594, 2008.
[52] X. Liu, W. Pedrycz, and M. Song, "The Development of Fuzzy Rough Sets with the Use of Structures and Algebras of Axiomatic Fuzzy Sets," IEEE Trans. Knowledge and Data Eng., vol. 23, no. 3, pp. 443-462, Mar. 2009.
[53] P. Maji and S.K. Pal, "Rough-Fuzzy C-Medoids Algorithm and Selection of Bio-Basis for Amino Acid Sequence Analysis," IEEE Trans. Knowledge and Data Eng., vol. 19, no. 6, pp. 859-872, June 2007.
[54] Q. Hu, J. Liu, and D. Yu, "Stability Analysis on Rough Set Based Feature Evaluation," Proc. Third Int'l Conf. Rough Sets and Knowledge Technology (RSKT '08), pp. 88-96, 2008.
[55] Q. Hu, L. Zhang, D. Chen, W. Pedrycz, and D. Yu, "Gaussian Kernel Based Fuzzy Rough Sets: Model, Uncertainty Measures and Applications," Int'l J. Approximate Reasoning, vol. 51, no. 4, pp. 453-471, 2010.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool