This Article 
 Bibliographic References 
 Add to: 
Are Multilayer Perceptrons Adequate for Pattern Recognition and Verification?
November 1998 (vol. 20 no. 11)
pp. 1121-1132

Abstract—This paper discusses the ability of multilayer perceptrons (MLPs) to model the probability distribution of data in typical pattern recognition and verification problems. It is proven that multilayer perceptrons with sigmoidal units and a number of hidden units less or equal than the number of inputs are unable to model patterns distributed in typical clusters, since these networks draw open separation surfaces in the pattern space. When using more hidden units than inputs, the separation surfaces can be closed but, unfortunately, it is proven that determining whether or not an MLP draws closed separation surfaces in the pattern space is ${\cal NP}$-hard. The major conclusion of this paper is somewhat opposite to what is believed and reported in many application papers: MLPs are definitely not adequate for applications of pattern recognition requiring a reliable rejection and, especially, they are not adequate for pattern verification tasks.

[1] A. Waibel,T. Hanazawa,G. Hinton,K. Shikano,, and K. Lang,“Phoneme recognition using time-delay neural networks,” IEEE Trans. ASSP, vol. 37, no. 3, Mar. 1989.
[2] Y. le Cun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel, "Backpropagation Applied to Handwritten Zip Code Recognition," Neural Computation, vol. 1, pp. 541-551, 1989.
[3] H. Gish and M. Schmidt, "Text-Independent Speaker-Identification," IEEE Signal Processing, vol. 11, pp. 18-32, Oct. 1994.
[4] M. Gori and A. Tesi,“On the problem of local minima in backpropagation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 1, pp. 76-86, 1992.
[5] R. Lippmann, "Review of Neural Networks for Speech Recognition," Neural Computation, vol. 1, pp. 1-38, 1989.
[6] F. Takeda and S. Omatu, "High Speed Paper Currency Recognition by Neural Networks," IEEE Trans. Neural Networks, vol. 6, pp. 73-77, Jan. 1995.
[7] R. Bajaj and S. Chaudhury, "Signature Verification Using Multiple Neural Classfier," Pattern Recognition, vol. 30, no. 1, pp. 1-7, 1997.
[8] J.-P. Drouhard, R. Sabourin, and M. Godbout, "A Neural Network Approach to Off-Line Signature Verification Using Directional PDF," Pattern Recognition, vol. 29, no. 3, pp. 415-424, 1996.
[9] A. Hakulinen and J. Hakkarainen, "A Neural Network Approach to Quality Control of Padlock Manufacturing," Pattern Recognition Letters, vol. 17, pp. 357-362, 1996.
[10] N. Intrator, D. Reisfeld, and Y. Yeshurun, "Face Recognition Using a Hybrid Supervised/Unsupervised Neural Network," Pattern Recognition Letters, vol. 17, pp. 67-76, Jan. 1996.
[11] Y. Zhou and R. Hecht-Nielsen, "Target Recognition Using Multiple Sensors," Neural Networks for Signal Processing, vol. 3, pp. 411-420,New York, 1993.
[12] S.P.C. Spence, S. Hsu, and J. Pearson, "Integrating Neural Networks With Image Pyramids to Learn Target Context," Neural Networks, vol. 8, no. 7-8, pp. 1,143-1,152, 1995.
[13] R.P. Lippmann, "An Introduction to Computing with Neural Nets," IEEE Acoustics, Speech, and Signal Processing Magazine, vol. 4, pp. 4-22, Apr. 1987.
[14] A. Frosini, M. Gori, and P. Priami, "A Neural Network-Based Model for Paper Currency Recognition and Verification," IEEE Trans. Neural Networks, vol. 7, pp. 1,482-1,490, Nov. 1996.
[15] M. Gori, L. Lastrucci, and G. Soda, "Autoassociator-Based Models for Speaker Verification," Pattern Recognition Letters, vol. 17, pp. 241-250, 1996.
[16] J. Oglesby and J. Mason, "Radial Basis Functions for Speaker Recognition," Proc. ICASSP'91, pp. 393-396, 1991.
[17] D.S. Johnson and F.P. Preparata, "The Densest Hemisphere Problem," Theor. Comput. Sci., vol. 6, pp. 93-107, 1978.

Index Terms:
Multilayer perceptrons, pattern recognition, pattern verification, function approximation, closed hemisphere problem.
Marco Gori, Franco Scarselli, "Are Multilayer Perceptrons Adequate for Pattern Recognition and Verification?," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1121-1132, Nov. 1998, doi:10.1109/34.730549
Usage of this product signifies your acceptance of the Terms of Use.