This Article 
 Bibliographic References 
 Add to: 
Improving Multiclass Pattern Recognition by the Combination of Two Strategies
June 2006 (vol. 28 no. 6)
pp. 1001-1006
We present a new method of multiclass classification based on the combination of one-vs-all method and a modification of one-vs-one method. This combination of one-vs-all and one-vs-one methods proposed enforces the strength of both methods. A study of the behavior of the two methods identifies some of the sources of their failure. The performance of a classifier can be improved if the two methods are combined in one, in such a way that the main sources of their failure are partially avoided.

[1] P. Clark and R. Boswell, “Rule Induction with CN2: Some Recent Improvements,” Proc. Fifth European Working Session on Learning (EWSL-91), pp. 151-163, 1991.
[2] R. Anand, K.G. Mehrotra, C.K. Mohan, and S. Ranka, “Efficient Classification for Multiclass Problems Using Modular Neural Networks,” IEEE Trans. Neural Networks, vol. 6, pp. 117-124, 1995.
[3] S. Knerr, L. Personnaz, and G. Dreyfus, “Single-Layer Learning Revisited: A Stepwise Procedure for Building and Training a Neural Network,” Neurocomputing: Algorithms, Architectures, and Applications, J. Fogelman, ed., New York: Springer-Verlag, 1990.
[4] T. Hastie and R. Tibshirani, “Classification by Pairwise Coupling,” The Annals of Statistics, vol. 26, no. 2, pp. 451-471, 1998.
[5] J. Friedman, “Another Approach to Polychotomous Classification,” technical report, Dept. of Statistics, Stanford Univ., 1996.
[6] J.C. Platt, N. Christiani, and J. Shawe-Taylor, “Large Margin DAGs for Multiclass Classification,” Proc. Neural Information Processing Systems (NIPS '99), S.A. Solla, T.K. Leen, and K.-R. Müller, eds., pp. 547-553, 1999.
[7] T.G. Dietterich and G. Bakiri, “Solving Multiclass Learning Problems via Error-Correcting Output Codes,” J. Artificial Intelligence Research, vol. 2, pp. 263-286, 1995.
[8] T. Windeatt and R. Ghaderi, “Coding and Decoding Strategies for Multi-Class Learning Problems,” Information Fusion, vol. 4, pp. 11-21, 2003.
[9] E.B. Kong and T.G. Dietterich, “Why Error-Correcting Output Coding Works with Decision Trees,” technical report, Dept. of Computer Science, Oregon State Univ., Corvallis, 1995.
[10] R. Rifkin and A. Klautau, “In Defense of One-vs-All Classification,” J. Machine Learning Research, vol. 5, pp. 101-141, 2004.
[11] E.L. Allwein, R.E. Schapire, and Y. Singer, “Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers,” J. Machine Learning Research, vol. 1, pp. 113-141, 2000.
[12] M. Moreira and E. Mayoraz, “Improved Pairwise Coupling Classification with Correcting Classifiers,” Proc. 10th European Conf. Machine Learning (ECML '98), Apr. 1997.
[13] J. Fürnkranz, “Round Robin Classification,” J. Machine Learning Research, vol. 2, pp. 721-747, 2002.
[14] S. Hettich, C.L. Blake, and C.J. Merz, “UCI Repository of Machine Learning Databases,” , 1998.
[15] R.E. Schapire, “Using Output Codes to Boost Multiclass Learning Problems,” Proc. 14th Int'l Conf. Machine Learning, pp. 313-321, 1997.
[16] T.G. Dietterich, “Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms,” Neural Computation, vol. 10, no. 7, pp. 1895-1923, 1998.
[17] G.I. Webb, “Multiboosting: A Technique for Combining Boosting and Wagging,” Machine Learning, vol. 40, no. 2, pp. 159-196, Aug. 2000.

Index Terms:
Multiclass, classification, one-vs-one, one-vs-all, neural networks, support vector machines.
Nicol? Garc?a-Pedrajas, Domingo Ortiz-Boyer, "Improving Multiclass Pattern Recognition by the Combination of Two Strategies," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 1001-1006, June 2006, doi:10.1109/TPAMI.2006.123
Usage of this product signifies your acceptance of the Terms of Use.