The Community for Technology Leaders
RSS Icon
Issue No.04 - April (2012 vol.24)
pp: 605-618
Brijesh Verma , Central Queensland University, Rockhampton
This paper presents a novel cluster-oriented ensemble classifier. The proposed ensemble classifier is based on original concepts such as learning of cluster boundaries by the base classifiers and mapping of cluster confidences to class decision using a fusion classifier. The categorized data set is characterized into multiple clusters and fed to a number of distinctive base classifiers. The base classifiers learn cluster boundaries and produce cluster confidence vectors. A second level fusion classifier combines the cluster confidences and maps to class decisions. The proposed ensemble classifier modifies the learning domain for the base classifiers and facilitates efficient learning. The proposed approach is evaluated on benchmark data sets from UCI machine learning repository to identify the impact of multicluster boundaries on classifier learning and classification accuracy. The experimental results and two-tailed sign test demonstrate the superiority of the proposed cluster-oriented ensemble classifier over existing ensemble classifiers published in the literature.
Ensemble classifier, clustering, classification, fusion of classifiers.
Brijesh Verma, "Cluster-Oriented Ensemble Classifier: Impact of Multicluster Characterization on Ensemble Classifier Learning", IEEE Transactions on Knowledge & Data Engineering, vol.24, no. 4, pp. 605-618, April 2012, doi:10.1109/TKDE.2011.28
[1] R. Polikar, "Ensemble Based Systems in Decision Making," IEEE Circuits and Systems Magazine, vol. 6, no. 3, pp. 21-45, July-Sept. 2006.
[2] R. Caruana and A.N. Mizil, "An Empirical Comparison of Supervised Learning Algorithms," Proc. Int'l Conf. Machine Learning (ICML), pp. 161-168, 2006.
[3] T. Windeatt, "Accuracy/Diversity and Ensemble MLP Classifier Design," IEEE Trans. Neural Networks, vol. 17, no. 5, pp. 1194-1211, Sept. 2006.
[4] L. Breiman, "Bagging Predictors," Machine Learning, vol. 24, no. 2, pp. 123-140, 1996.
[5] L. Breiman, "Random Forests," Machine Learning, vol. 45, no. 1, pp. 5-32, Oct. 2001.
[6] G. Fumera, F. Roli, and A. Serrau, "A Theoretical Analysis of Bagging as a Linear Combination of Classifiers," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 7, pp. 1293-1299, July 2008.
[7] R.E. Schapire, "The Strength of Weak Learnability," Machine Learning, vol. 5, no. 2, pp. 197-227, 1990.
[8] Y. Freund and R.E. Schapire, "Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting," J. Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1997.
[9] R.E. Banfield, L.o. Hall, K.W. Bowyer, W.P. Kegelmeyer, "A New Ensemble Diversity Measure Applied to Thinning Ensembles," Proc. Fourth Int'l Workshop Multiple Classifier Systems (MCS '03), pp. 306-316, 2003.
[10] N.G. Pedrajas, "Constructing Ensembles of Classifiers by Means of Weighted Instance Selection," IEEE Trans. Neural Networks, vol. 20, no. 2, pp. 258-277, Feb. 2009.
[11] G.M. Munoz, D.H. Lobato, and A. Suarez, "An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 245-259, Feb. 2009.
[12] J.J. Rodriguez and J. Maudes, "Boosting Recombined Weak Classifiers," Pattern Recognition Letters, vol. 29, pp. 1049-1059, 2008.
[13] L. Nanni and A. Lumini, "Fuzzy Bagging: A Novel Ensemble of Classifiers," Pattern Recognition, vol. 39, pp. 488-490, 2006.
[14] L. Chen and M.S. Kamel, "A Generalized Adaptive Ensemble Generation and Aggregation Approach for Multiple Classifiers Systems," Pattern Recognition, vol. 42, pp. 629-644, 2009.
[15] J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas, "On Combining Classifiers," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 226-239, Mar. 1998.
[16] L.I. Kuncheva, J.C. Bezdek, and R. Duin, "Decision Templates for Multiple Classifier Fusion: An Experimental Comparison," Pattern Recognition, vol. 34, no. 2, pp. 299-314, 2001.
[17] A.H.R. Ko, R. Sabourin, S. Britto, and L. Oliveira, "Pairwise Fusion Matrix for Combining Classifiers," Pattern Recognition, vol. 40, pp. 2198-2210, 2007.
[18] N.M. Wanas, R.A. Dara, and M.S. Kamel, "Adaptive Fusion and Co-Operative Training for Classifier Ensembles," Pattern Recogntion, vol. 39, pp. 1781-1794, 2006.
[19] O.R. Terrades, E. Valveny, and S. Tabbone, "Optimal Classifier Fusion in a Non-Bayesian Probabilistic Framework," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 9, pp. 1630-1644, Sept. 2009.
[20] D. Parikh and R. Polikar, "Ensemble Based Incrimental Learning Approach to Data Fusion," IEEE Trans. Systeams, Man, and Cybernetics, vol. 37, no. 2, pp. 437-450, Apr. 2007.
[21] M.D. Muhlbaier, A. Topalis, and R. Polikar, "Learn++.NC: Combining Ensemble of Classifiers with Dynamically Weighted Consult-and-Vote for Efficient Incremental Learning of New Classes," IEEE Trans. Neural Networks, vol. 20, no. 1, pp. 152-168, Jan. 2009.
[22] R. Maclin and J.W. Shavlik, "Combining the Predictions of Multiple Classifiers: Using Competitive Learning to Initialize Neural Networks," Proc. 14th Int'l Joint Conf. Artificial Intelligence, pp. 524-531, 1995.
[23] O. Pujol and D. Masip, "Geometry-Based Ensembles: Toward a Structural Characterization of the Classification Boundary," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 6, pp. 1140-1146, June 2009.
[24] P. Chaudhuri, A.K. Ghosh, and H. Oja, "Classification Based on Hybridization of Parametric and Non-Parametric Classifiers," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 7, pp. 1153-1164, July 2009.
[25] A. Strehl and J. Ghosh, "Cluster Ensembles—A Knowledge Reuse Framework for Combining Multiple Partitions," The J. Machine Learning Research, vol. 3, pp. 583-617, 2003.
[26] E. Forgy, "Cluster Analysis of Multivariate Data: Efficiency vs. Interpretability of Classifications," Biometrics, vol. 21, pp. 768-780, 1965.
[27] UCI Machine Learning Database, http://archive.ics.uci.eduml/, Feb. 2010.
[28] LIBSVM, "A Library for Support Vector Machines,"∼cjlin libsvm/, Feb. 2010.
[29] J. Demsar, "Statistical Comparisons of Classifiers Over Multiple Data Sets," J. Machine Learning Research, vol. 7, pp. 1-30, 2006.
[30] D.J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures. Chapman & Hall/CRC, 2000.
[31] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I.H. Witten, "The WEKA Data Mining Software: An Update," ACM SIGKDD Explorations Newsletter, vol. 11, no. 1, pp. 10-18, 2009.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool