CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2010 vol.32 Issue No.01 - January

Subscribe

Issue No.01 - January (2010 vol.32)

pp: 120-134

Sergio Escalera , Universitat de Barcelona and Universitate Autonoma de Barcelona, Barcelona

Oriol Pujol , Universitat de Barcelona and Universitate Autonoma de Barcelona, Barcelona

Petia Radeva , Universitat de Barcelona and Universitate Autonoma de Barcelona, Barcelona

ABSTRACT

A common way to model multiclass classification problems is to design a set of binary classifiers and to combine them. Error-Correcting Output Codes (ECOC) represent a successful framework to deal with these type of problems. Recent works in the ECOC framework showed significant performance improvements by means of new problem-dependent designs based on the ternary ECOC framework. The ternary framework contains a larger set of binary problems because of the use of a “do not care” symbol that allows us to ignore some classes by a given classifier. However, there are no proper studies that analyze the effect of the new symbol at the decoding step. In this paper, we present a taxonomy that embeds all binary and ternary ECOC decoding strategies into four groups. We show that the zero symbol introduces two kinds of biases that require redefinition of the decoding design. A new type of decoding measure is proposed, and two novel decoding strategies are defined. We evaluate the state-of-the-art coding and decoding strategies over a set of UCI Machine Learning Repository data sets and into a real traffic sign categorization problem. The experimental results show that, following the new decoding strategies, the performance of the ECOC design is significantly improved.

INDEX TERMS

Error-correcting output codes, decoding, multiclass classification, embedding of dichotomizers.

CITATION

Sergio Escalera, Oriol Pujol, Petia Radeva, "On the Decoding Process in Ternary Error-Correcting Output Codes",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.32, no. 1, pp. 120-134, January 2010, doi:10.1109/TPAMI.2008.266REFERENCES

- [1] N.J. Nilsson,
Learning Machines. McGraw-Hill, 1965.- [2] V. Vapnik,
The Nature of Statistical Learning Theory. Springer, 1995.- [3] OSU-SVM-TOOLBOX, http:/svm.sourceforge.net, 2009.
- [4] J. Friedman, T. Hastie, and R. Tibshirani, “Additive Logistic Regression: A Statistical View of Boosting,”
The Annals of Statistics, vol. 38, pp. 337-374, 1998.- [5] E. Allwein, R. Schapire, and Y. Singer, “Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers,”
J. Machine Learning Research, vol. 1, pp. 113-141, 2002.- [6] T. Dietterich and G. Bakiri, “Solving Multiclass Learning Problems via Error-Correcting Output Codes,”
J. Artificial Intelligence Research, vol. 2, pp. 263-286, 1995.- [7] T. Windeatt and R. Ghaderi, “Coding and Decoding for Multi-Class Learning Problems,”
Information Fusion, vol. 4, pp. 11-21, 2003.- [8] T. Hastie and R. Tibshirani, “Classification by Pairwise Grouping,”
Proc. Conf. Neural Information Processing Systems, vol. 26, pp.451-471, 1998.- [9] K. Crammer and Y. Singer, “On the Learnability and Design of Output Codes for Multi-Class Problems,”
Machine Learning, vol. 47, pp. 201-233, 2002.- [10] O. Pujol, P. Radeva, and J. Vitrià, “Discriminant ECOC: A Heuristic Method for Application Dependent Design of Error Correcting Output Codes,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 1001-1007, June 2006.- [11] R. Rifkin and A. Klautau, “In Defense of One-vs-All Classification,”
J. Machine Learning Research, vol. 5, pp. 101-141, 2004.- [12] O. Pujol, S. Escalera, and P. Radeva, “An Incremental Node Embedding Technique for Error Correcting Output Codes,”
Pattern Recognition, to appear. - [13] T. Windeatt and G. Ardeshir, “Boosted ECOC Ensembles for Face Recognition,”
Proc. Int'l Conf. Visual Information Eng., pp. 165-168, 2003.- [14] J. Kittler, R. Ghaderi, T. Windeatt, and J. Matas, “Face Verification Using Error Correcting Output Codes,”
Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp.755-760, 2001.- [15] R. Ghani, “Combining Labeled and Unlabeled Data for Text Classification with a Large Number of Categories,”
Proc. Int'l Conf. Data Mining, pp. 597-598, 2001.- [16] J. Zhou and C. Suen, “Unconstrained Numeral Pair Recognition Using Enhanced Error Correcting Output Coding: A Holistic Approach,”
Proc. Int'l Conf. Document Analysis and Recognition, vol. 1, pp. 484-488, 2005.- [17] A. Passerini, M. Pontil, and P. Frasconi, “New Results on Error Correcting Output Codes of Kernel Machines,”
IEEE Trans. Neural Networks, vol. 15, no. 1, pp. 45-54, Jan. 2004.- [18] O. Dekel and Y. Singer, “Multiclass Learning by Probabilistic Embeddings,”
Proc. Conf. Neural Information Processing Systems, vol. 15, 2002.- [19] N. Ishii, E. Tsuchiya, Y. Bao, and N. Yamaguchi, “Combining Classification Improvements by Ensemble Processing,”
Proc. ACIS Int. Conf. Software Eng. Research, Management, and Applications, pp.240-246, 2005.- [20] W. Utschick and W. Weichselberger, “Stochastic Organization of Output Codes in Multiclass Learning Problems,”
Neural Computation, vol. 13, no. 5, pp. 1065-1102, 2004.- [21] T. Dietterich and E. Kong, “Error-Correcting Output Codes Corrects Bias and Variance,”
Proc. 21th Int'l Conf. Machine Learning, S. Prieditis and S. Russell, eds., pp. 313-321, 1995.- [22] J. Casacuberta, J. Miranda, M. Pla, S. Sanchez, A. Serra, and J. Talaya, “On the Accuracy and Performance of the Geomobil System,”
Proc. Congress of Int'l Soc. for Photogrammetry and Remote Sensing, 2004.- [23] R. Schapire and Y. Singer, “Improved Boosting Algorithms Using Confidence-Rated Prediction,”
Machine Learning, vol. 37, no. 3, pp.297-336, 1999.- [24] J. Zhu, S. Rosset, H. Zou, and T. Hastie, “Multi-Class Adaboost. A Multiclass Generalization of the Adaboost Algorithm, Based on a Generalization of the Exponential Loss,” 2005.
- [25] T. Kikuchi and S. Abe, “Error Correcting Output Codes vs. Fuzzy Support Vector Machines,”
Proc. Conf. Artificial Neural Networks in Pattern Recognition, 2003.- [26] F. Ricci and D. Aha, “Error-Correcting Output Codes for Local Learners,”
Proc. European Conf. Machine Learning, pp. 280-291, 1998.- [27] S. Escalera, O. Pujol, and P. Radeva, “Boosted Landmarks of Contextual Descriptors and Forest-ECOC: A Novel Framework to Detect and Classify Objects in Clutter Scenes,”
Pattern Recognition Letters, vol. 28, no. 13, pp. 1759-1768, 2007.- [28] R.S. Smith and T. Windeatt, “Decoding Rules for Error Correcting Output Code Ensembles,”
Multiple Classifier Systems, pp. 53-63, Springer, 2005.- [29] A. Asuncion and D.J. Newman,, UCI Machine Learning Repository, Dept. of Information and Computer Science, Univ. of California, Irvine, http://www.ics.uci.edu/mlearnMLRepository.html , 2007.
- [30] J. Demsar, “Statistical Comparisons of Classifiers over Multiple Data Sets,”
J. Machine Learning Research, vol. 7, pp. 1-30, 2006. |