This Article 
 Bibliographic References 
 Add to: 
Theoretical Analysis and Improved Decision Criteria for the n-Tuple Classifier
April 1999 (vol. 21 no. 4)
pp. 336-347

Abstract—The anticipated behavior of the n-tuple classification system is that it gives the highest output score for the class to which the input example actually belongs. By performing a theoretical analysis of how the output scores are related to the underlying probability distributions of the data, this paper shows that this in general is not to be expected. The theoretical results are able to explain the behavior that is observed in experimental studies. The theoretical analysis also give valuable insight into how the n-tuple classifier can be improved to deal with skewed training priors, which until now have been a hard problem for the architecture to tackle. It is shown that by relating an output score to the probability that a given class generates the data makes it possible to design the n-tuple net to operate as a close approximation to the Bayes estimator. It is specifically illustrated that this approximation can be obtained by modifying the decision criteria. In real cases, the underlying example distributions are unknown and accordingly the optimum way to treat the output scores cannot be calculated theoretically. However, it is shown that the feasibility of performing leave-one-out cross-validation tests in n-tuple networks makes it possible to obtain proper processing of the scores in such cases.

[1] I. Aleksander, "Microcircuit Learning Nets: Hamming-Distance Behaviour," Electronic Letters, vol. 6, pp. 134-136, 1970.
[2] I. Aleksander and H. Morton, An Introduction to Neural Computing.London: Chapman and Hall, 1990.
[3] I. Aleksander, W.V. Thomas, and P.A. Bowden, "Wisard: A Radical Step Forward in Image Recognition," Sensor Review, pp. 120-124, 1984.
[4] A.W. Andersen, S.S. Christensen, T.M. Jorgensen, and C. Liisberg, "An Active Vision System for Robot Guidance Using a Low Cost Neural Network Board," Proc. Machine Vision Applications, Architectures, and Systems Integration,Boston, Mass., pp. 163-170, 1994.
[5] J. Austin, RAM-Based Neural Networks.London: World Scientific, 1998.
[6] W.W. Bledsoe and I. Browning, "Pattern Recognition and Reading by Machine," Proc. Eastern Joint Computer Conf., pp. 225-232, 1959.
[7] W.W Bledsoe, "Further Results on the n-tuple Pattern Recognition Method," IRE Trans. Electronic Computers (Correspondence), vol. EC-10, p. 96, 1961.
[8] D.-M. Jung, G. Nagy, and A. Shapira, “N-tuple Features for OCR Revisited,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 7, pp. 734-745, July 1996.
[9] T.M. Jørgensen, "Classification of Handwritten Digits Using a RAM Neural Net Architecture," Int'l J. Neural Systems, vol. 8, no. 1, pp. 17-25, 1997.
[10] T.M. Jørgensen, S.S. Christensen, A.W. Andersen, and C. Liisberg, "Optimization and Application of a RAM Based Neural Network for Fast Image Processing Tasks," Intelligent Robots and Computer Vision,Boston, Mass., pp. 328-338, Oct 31- Nov2 1994.
[11] T.M. Jørgensen, S.S. Christensen, and C. Liisberg, "Cross-Validation and Information Measures for RAM Based Neural Networks," RAM-Based Neural Networks, J. Austin, ed, pp. 78-88.London: World Scientific, 1998.
[12] J.V. Kennedy, J. Austin, R. Pack, and B. Cass, "C-NNAP—A Parallel Processing Architecture for Binary Neural Networks," Proc. IEEE Int'l Conf. Neural Networks, vol. 2, 1995, pp. 1,037-1,041.
[13] D. Michie, D.J. Spiegelhalter, and C.C. Taylor, Machine Learning, Neural and Statistical Classification. Ellis Horwood, 1994.
[14] M. Morciniec and R. Rohwer, "Good-Turing Estimation for the Frequentist n-tuple Classifier," Proc. Weightless Neural Network Workshop 1995, Computing with Logical Neurons, Univ. of Kent, pp. 93-98, 1995.
[15] Nat'l Inst. Standards and Tech nology, NIST Special Database 19, Handprinted Forms and Characters Database, HFCD Rel. 21-1.1, 1995.
[16] R. Rohwer and M. Morciniec, "The Theoretical and Experimental Status of the n-tuple Classifier", Neural Networks, vol. 11, no. 1, pp. 1-14, 1998.
[17] T.J. Stonham, "Improved Hamming-Distance Analysis for Digital Learning Networks," Electronics Letters, vol. 13, no. 6, pp. 155-156, 1977.
[18] D.D. Wackerly, I. William, R.L. Mendenhall, and L. Scheaffer, Mathematical Statistics with Applications.Belmont: Duxbury Press, June 1996.
[19] L. Wehenkel, M. Pavella, E. Euxibie, and B. Heilbronn, "Decision Tree Based Transient Stability Assessment—A Case Study," Proc. IEEE/PES 1993 Winter Meeting,Columbus, Ohio, 1993.

Index Terms:
n-tuple classifier, maximum likelihood, Bayes, cross-validation, RAM-net.
Thomas Martini Jørgensen, Christian Linneberg, "Theoretical Analysis and Improved Decision Criteria for the n-Tuple Classifier," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 4, pp. 336-347, April 1999, doi:10.1109/34.761264
Usage of this product signifies your acceptance of the Terms of Use.