• Publication
  • 2002
  • Issue No. 3 - March
  • Abstract - A Selective Attention-Based Method for Visual Pattern Recognition with Application to Handwritten Digit Recognition and Face Recognition
 This Article 
 Bibliographic References 
 Add to: 
A Selective Attention-Based Method for Visual Pattern Recognition with Application to Handwritten Digit Recognition and Face Recognition
March 2002 (vol. 24 no. 3)
pp. 420-425

Parallel pattern recognition requires great computational resources; it is NP-complete. From an engineering point of view it is desirable to achieve good performance with limited resources. For this purpose, we develop a serial model for visual pattern recognition based on the primate selective attention mechanism. The idea in selective attention is that not all parts of an image give us information. If we can attend only to the relevant parts, we can recognize the image more quickly and using less resources. We simulate the primitive, bottom-up attentive level of the human visual system with a saliency scheme and the more complex, top-down, temporally sequential associative level with observable Markov models. In between, there is a neural network that analyses image parts and generates posterior probabilities as observations to the Markov model. We test our model first on a handwritten numeral recognition problem and then apply it to a more complex face recognition problem. Our results indicate the promise of this approach in complicated vision applications.

[1] D. Noton and L. Stark, “Eye Movements and Visual Perception,” Scientific Am., vol. 224, pp. 34-43, 1971.
[2] J.K. Tsotsos, S.M. Culhane, W.Y.K. Wai, Y. Lai, N. Davis, and F. Nuflo, “Modeling Visual Attention via Selective Tuning,” Artificial Intelligence, vol. 78, pp. 507-545, 1995.
[3] C. Koch and S. Ullman, “Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry,” Human Neurobiology, vol. 4, pp. 219-227, 1985.
[4] F. Crick and C. Koch, “Towards a Neurobiological Theory of Consciousness,” Seminars in the Neurosciences, vol. 2, pp. 263-275, 1990.
[5] L. Itti and C. Koch, “Computational Modeling of Visual Attention,” Nature Reviews Neuroscience, vol. 2, no. 3, pp. 194-203, Mar. 2001.
[6] S. Grossberg, “The Complementary Brain: Unifying Brain Dynamics and Modularity,” Trends in Cognitive Sciences, vol. 4, pp. 233-246, 2000.
[7] L.G. Ungerleider and M. Mishkin, “Two Cortical Visual Systems,” Analysis of Visual Behaviour, D.J. Ingle, M.A. Goodale and R.J.W. Mansfield eds., 1982.
[8] A.M. Treisman and G. Gelade, “A Feature Integration Theory of Attention,” Cognitive Pscychology, vol. 12, no. 1, pp. 97-136, Jan. 1980.
[9] L. Itti and C. Koch, “Feature Combination Strategies for Saliency-Based Visual Attention Systems,” J. Electronic Imaging, vol. 10, pp. 161-169, 2001.
[10] S.M. Culhane and J.K. Tsotsos, “A Prototype for Data-Driven Visual Attention,” Proc. 11th Int'l Conf. Pattern Recognition, vol. 1, pp. 36-40, 1992.
[11] L. Itti, C. Koch, and E. Niebur, “A Model for Saliency-Based Visual Attention for Rapid Scene Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1,254-1,259, Nov. 1998.
[12] R. Desimone and J. Duncan, “Neural Mechanisms of Selective Visual Attention,” Ann. Rev. of Neuroscience, vol. 18, pp. 193-222, 1995.
[13] M. Jägersand, “Saliency Maps and Attention Selection in Scale and Spatial Coordinates: An Information Thoretic Approach,” Proc. Fifth Int'l Conf. Computer Vision, pp. 195-202, 1995.
[14] G.E. Legge, T.S. Klitz, and B.S. Tjan, “Mr. Chips: An Ideal-Observer Model of Reading,” Psychological Rev., vol. 104, no. 3, pp. 524-553, 1997.
[15] K. Schill, E. Umkehrer, S. Beinlich, G. Krieger, and C. Zetzsche, “Scene Analysis with Saccadic Eye Movements: Top-Down and Bottom-Up Modeling,” J. Electronic Imaging, vol. 10, no. 1, pp. 152-160, Jan. 2001.
[16] R.L. Didday and M.A. Arbib, “Eye Movements and Visual Perception: A `Two Visual System' Model,” Int'l J. Man-Machine Studies, vol. 7, pp. 547-569, 1975.
[17] S.S. Hacisalihzade, L.W. Stark, and J.S. Allen, “Visual Perception and Sequences of Eye Movement Fixations: A Stochastic Modeling Approach,” IEEE Trans. System, Man, and Cybernetics, vol. 22, no. 3, pp. 474-481, 1992.
[18] N. Oliver, B. Rosario, and A. Pentland, "A Bayesian Computer Vision System for Modeling Human Interactions," IEEE Trans. Pattern Analysis and Machine Intelligence, Aug. 2000, pp. 831-843.
[19] R.D. Rimey and C.M. Brown, “Selective Attention as Sequential Behavior: Modeling Eye Movements with an Augmented Hidden Markov Model,” TR-327, Computer Science, Univ. of Rochester, Feb. 1990.
[20] R.P. Rao, G.J. Zelinsky, M.M. Hayhoe, and D.H. Ballard, “Eye Movements in Visual Cognition: A Computational Study,” Technical Report, 97.1, Univ. of Rochester, Computer Science Dept., 1997.
[21] E. Alpaydin, “Selective Attention for Handwritten Digit Recognition,” Advances in Neural Information Processing Systems 8, D.S. Touretzky, M.C. Mozer, and M.E. Hasselmo eds., pp. 771-777, 1996.
[22] K. Fukushima, “Neural Network Model for Selective Attention in Visual Pattern Recognition and Associative Recall,” Applied Optics, vol. 26, no. 23, pp. 4985-4992, Dec. 1987.
[23] I.A. Rybak, V.I. Gusakova, A.V. Golovan, L.N. Podladchikova, and N.A. Shevtsova, “A Model of Attention-Guided Visual Perception and Recognition,” Vision Research, vol. 38, pp. 2387-2400, 1998.
[24] C.L. Blake and C.J. Mertz, UCI Repository of Machine Learning Databases, Univ. of California, Irine,http://www.ics.uci.edu/~mlearnMLRepository.html , 1998.
[25] Y. Le Cun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel, “Backpropagation Applied to Handwritten Zip Code Recognition,” Neural Computation, vol. 1, no. 4, pp. 541-551, 1989.
[26] The Olivetti Research Laboratory Database of Faces, http://www.cam-orl.co.ukfacedatabase.html . 1994.
[27] D.H. Foster and S. Westland, “Multiple Groups of Orientation-Selective Visual Mechanisms Underlying Rapid Oriented-line Detection,” Proc. Royal Soc. London, vol. 265, pp. 1605-1613, 1998.
[28] R.M. Klein, “Inhibition of Return,” Trends in Cognitive Sciences, vol. 4, no. 4, pp. 138-147, Apr. 2000.
[29] L.R. Rabiner, “Tutorial on Hidden Markov Model and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257-285, 1989.
[30] L. Itti and C. Koch, “A Comparison of Feature Combination Staregies for Saliency-Based Visual Attention Systems,” SPIE Human Vision and Electronic Imaging IV, vol. 3644, pp. 373-382, Jan. 1999.
[31] D. Reisfeld, H. Wolfsow, and Y. Yeshurun, “Context-Free Attentional Operators: the Generalized Symmetry Transform,” Int'l J. Computer Vision, vol. 14, pp. 119-130, 1995.
[32] C.M. Privitera and L.W. Stark, “Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 9, pp. 970-982, Sept. 2000.
[33] J.G. Daugman, “Complete Discrete 2D Gabor Transforms by Neural Networks for Image Analysis and Compression,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 36, no. 7, 1988.
[34] F. Smeraldi and J. Bigün, “Facial Feature Detection by Saccadic Exploration of the Gabor Decomposition,” Proc. Int'l Conf. Image Processing, vol. 3, pp. 163-167, 1998.
[35] J.G. Keller, S.K. Rogers, M. Kabrisky, and M.E. Oxley, “Object Recognition Based on Human Saccadic Behaviour,” Pattern Analysis&Applications, vol. 2, pp. 251-263, 1999.
[36] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J. Cognitive Science, vol. 3, no. 1, pp. 71-96, 1991.
[37] S. Lawrence, C.L. Giles, A.C. Tsoi, and A.D. Back, “Face Recognition: A Convolutional Neural-Network Approach,” IEEE Trans. Neural Networks, vol. 8, pp. 98-113, 1997.
[38] A.A. Salah, E. Alpaydin, and L. Akarun, “Selective Attention Based Visual Pattern Recognition,” Technical Report, FBE/CMPE-03/2001-12, Bogaziçi Univ., Dept. of Computer Eng.,http://www.cmpe.boun.edu.tr/~salahtr_salah.zip , 2001

Index Terms:
selective attention, Markov models, feature integration, face recognition, handwritten digit recognition
A.A. Salah, E. Alpaydin, L. Akarun, "A Selective Attention-Based Method for Visual Pattern Recognition with Application to Handwritten Digit Recognition and Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 420-425, March 2002, doi:10.1109/34.990146
Usage of this product signifies your acceptance of the Terms of Use.