This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Morphologically Optimal Strategy for Classifier Combination: Multiple Expert Fusion as a Tomographic Process
March 2003 (vol. 25 no. 3)
pp. 343-353
Josef Kittler, IEEE Computer Society

Abstract—We specify an analogy in which the various classifier combination methodologies are interpreted as the implicit reconstruction, by tomographic means, of the composite probability density function spanning the entirety of the pattern space, the process of feature selection in this scenario amounting to an extremely bandwidth-limited Radon transformation of the training data. This metaphor, once elaborated, immediately suggests techniques for improving the process, ultimately defining, in reconstructive terms, an optimal performance criterion for such combinatorial approaches.

[1] R.A. Jacobs, “Methods for Combining Experts' Probability Assessments,” Neural Computation, vol. 3, pp. 79-87, 1991.
[2] J. Kittler, M. Hatef, R. Duin, and J. Matas, “On Combining Classifiers,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 226-239, Mar. 1998.
[3] L. Lam, C. Suen, “Optimal Combination of Pattern Classifiers,” Pattern Recognition Letters, vol. 16, pp. 945-954, 1995.
[4] A.F.R. Rahman and M.C. Fairhurst, “An Evaluation of Multi-Expert Configurations for the Recognition of Handwritten Numerals,” Pattern Recognition Letters, vol. 31, pp, 1255-1273, 1998.
[5] A.F.R. Rahman and M.C. Fairhurst, “A New Hybrid Approach in Combining Multiple Experts to Recognize Handwritten Numerals,” Pattern Recognition Letters, vol. 18, pp. 781-790, 1997.
[6] K. Woods, W.P. Kegelmeyer, and K.W. Bowyer, "Combination of Multiple Classifiers Using Local Accuracy Estimates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 4, pp. 405-410, Apr. 1997.
[7] R. Neal, “Probabilistic Inference Using Markov Chain Monte Carlo Methods,” Technical Report CRG-TR-93-1 1993, Dept. of Computer Science, Univ. of Toronto, Toronto, Canada, 1993.
[8] L. Breiman, “Bagging Predictors,” Machine Learning, vol. 24, pp. 123-140, 1996.
[9] H. Drucker, C. Cortes, L.D. Jackel, Y. Lecun, and V. Vapnik, Boosting and Other Ensemble Methods, Neural Computation. vol. 6, no. 6, pp. 1289-1301, 1994.
[10] D. Windridge and J. Kittler, “Combined Classifier Optimisation via Feature Selection” Proc. Advances in Pattern Recognition, vol. 1876, 2000.
[11] T.G. Dietterich and G. Bakiri, “Solving Multiclass Learning Problems via Error-Correcting Output Codes,” J. Artificial Intelligence Research, vol. 2, pp. 263-286, 1995.
[12] J. Kittler, Y. Yusoff, W. Christmas, T. Windeatt, and D. Windridge, “Multiple Experts by Joint Optimisation of Decision Thresholds,” Pattern Recognition and Image Analysis, vol. 11, no. 3, pp. 529-541, 2001.
[13] T.J. Cornwell and K.F. Evans, “A Simple Maximum Entropy Deconvolution Algorithm,” J. Astronomy and Astrophysics, vol. 143, pp. 77-83, 1985.
[14] J. Högbom, “Aperture Synthesis with a Non-Regular Distribution of Interferometer Baselines,” J. Astronomy and Astrophysics Supplement Series, vol. 15, pp. 417-426, 1974.
[15] D. Windridge and J. Kittler, “Classifier Combination as a Tomographic Process,” Multiple Classifier Systems, vol. 2096, 2001.
[16] K.M. Ali and M.J. Pazzani, “On the Link Between Error Correlation and Error Reduction in Decision Tree Ensembles,” Technical Report 95-38, ICS-UCI, 1995.
[17] T.K. Ho, J.J. Hull, and S.N. Srihari, “Decision Combination in Multiple Classifiers Systems,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 1, pp. 66-75, Jan. 1994.
[18] L. Xu, A. Krzyzak, and C.Y. Suen, “Methods of Combining Multiple Classifiers and Their Application to Handwriting Recognition,” IEEE Trans. Systems, Man, and Cybernetics, vol. 22, no. 11, pp. 1539-1549, 1994.
[19] J. Kittler, “Improving Recognition Rates by Classifier Combination: A Review,” Proc. First IAPR TC1 Workshop Statistical Techniques in Pattern Recognition, pp. 205-210, June 1997.
[20] F. Natterer, Proc. State of the Art in Numerical Analysis. Apr. 1996.
[21] S. Webb, The Physics of Medical Imaging. Inst. of Physics Publishing, 1993.
[22] G.T. Herman, “Image Reconstruction from Projections, the Fundamentals of Computerized Tomography,” Computer Science and Applied Math., 1980.
[23] D. Windridge and J. Kittler, “A Generalized Solution to the Problem of Multiple Expert Fusion,” Univ. of Surrey Technical Report: VSSP-TR-5/, 2000.
[24] D. Windridge and J. Kittler, “An Efficient Algorithmic Implementation of Tomographic Classifier Combination,” Pattern Recognition J., (submitted).

Index Terms:
Classifier combination, tomography, probability theory, feature selection.
Citation:
David Windridge, Josef Kittler, "A Morphologically Optimal Strategy for Classifier Combination: Multiple Expert Fusion as a Tomographic Process," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 3, pp. 343-353, March 2003, doi:10.1109/TPAMI.2003.1182097
Usage of this product signifies your acceptance of the Terms of Use.