This Article 
 Bibliographic References 
 Add to: 
Evolutionary Pursuit and Its Application to Face Recognition
June 2000 (vol. 22 no. 6)
pp. 570-582

Abstract—This paper introduces Evolutionary Pursuit (EP) as a novel and adaptive representation method for image encoding and classification. In analogy to projection pursuit methods, EP seeks to learn an optimal basis for the dual purpose of data compression and pattern classification. The challenge for EP is to increase the generalization ability of the learning machine as a result of seeking the trade-off between minimizing the empirical risk encountered during training and narrowing the confidence interval for reducing the guaranteed risk during future testing on unseen images. Toward that end, EP implements strategies characteristic of genetic algorithms (GAs) for searching the space of possible solutions to determine the optimal basis. EP starts by projecting the original data into a lower dimensional whitened Principal Component Analysis (PCA) space. Directed but random rotations of the basis vectors in this space are then searched by GAs where evolution is driven by a fitness function defined in terms of performance accuracy (“empirical risk”) and class separation (“confidence interval”). Accuracy indicates the extent to which learning has been successful so far, while separation gives an indication of the expected fitness on future trials. The feasibility of the new method has been successfully tested on face recognition where the large number of possible bases requires some type of greedy search algorithm. The particular face recognition task involves 1,107 FERET frontal face images corresponding to 369 subjects. To assess both accuracy and generalization capability, the data includes for each subject images acquired at different times or under different illumination conditions. The results reported show that EP improves on face recognition performance when compared against PCA (“Eigenfaces”) and displays better generalization abilities than the Fisher linear discriminant (“Fisherfaces”).

[1] H.B. Barlow, “Unsupervised Learning,” Neural Computation, vol.71, pp. 295-311, 1989.
[2] P.N. Belhumeur, J. Hespanda, and D. Kriegeman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[3] R. Brunelli and T. Poggio, "Face Recognition: Features vs. Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1,042-1,053, Oct. 1993.
[4] T.F. Chan, “An Improved Algorithm for Computing the Singular Value Decomposition,” ACM Trans. Math. Software, vol. 8, pp. 72-83, 1982.
[5] R. Chellappa, C. Wilson, and S. Sirohey, "Human and Machine Recognition of Faces: A Survey," Proc. IEEE, vol. 83, no. 5, pp. 705-740, 1995.
[6] S. Chen and D. Donoho, “Basis Pursuit,” technical report, Dept. Statistics, Stanford Univ., 1996.
[7] P. Comon, “Independent Component Analysis, a New Concept?” Signal Processing, vol. 36, no. 3, 1994.
[8] T.F. Cootes, G.J. Edwards, and C.J. Taylor, “Active Appearance Models,” Proc. European Conf. Computer Vision, pp. 484-498, June 1998.
[9] J.G. Daugman, “An Information-Theoretic View of Analog Representation in Striate Cortex,” Computational Neuroscience, E.L. Schwartz, ed., pp. 403-424, MIT Press, 1990.
[10] K.I. Diamantaras and S.Y. Kung, Principal Component Neural Networks: Theory and Applications. John Wiley and Sons, 1996.
[11] G.M. Edelman, Neural Darwinism. Basic Books, 1987.
[12] K. Etemad and R. Chellappa, “Discriminant Analysis for Recognition of Human Face Images,” J. Optical Soc. Am. A, vol. 14, pp. 1,724-1,733, 1997.
[13] R.A. Fisher, “The Use of Multiple Measures in Taxonomic Problems,” Ann. Eugenics, vol. 7, pp. 179-188, 1936.
[14] J.H. Friedman and W. Stuetzle, “Projection Pursuit Regression,” J. Am. Statistical Assoc., vol. 76, pp. 817-823, 1981.
[15] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[16] F. Girosi, “An Equivalence between Sparse Approximation and Support Vector Machines,” Neural Computation, vol. 10, pp. 1,455-1,480, 1998.
[17] D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, Mass.: Addison-Wesley, 1989.
[18] S. Haykin, Neural Networks: A Comprehensive Foundation, Macmillan College Press, New York, 1994.
[19] P.J. Huber, “Projection Pursuit,” Annals Statistics, vol. 13, pp. 435-475, 1985.
[20] A. Hyvarinen and E. Oja, “A Fast Fixed-Point Algorithm for Independent Component Analysis,” Neural Computation, vol. 9, pp. 1,483-1,492, 1997.
[21] I.T. Jolliffe, Principal Component Analysis, New York: Springe, 1986.
[22] J. Karhunen, E. Oja, L. Wang, R. Vigario, and J. Joutsensalo, “A Class of Neural Networks for Independent Component Analysis,” IEEE Trans. Neural Networks, vol. 8, no. 3, pp. 486-504, 1997.
[23] M. Kirby and L. Sirovich,“Application of Karhunen-Loève procedure for the characterization of human faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 103-108, Jan. 1990.
[24] R. Linsker, “Self-Organization in a Perceptual Network,” Computer, vol. 21, pp. 105-117, 1988.
[25] C. Liu and H. Wechsler, “Enhanced Fisher Linear Discriminant Models for Face Recognition,” Proc. 14th Int'l Conf. Pattern Recognition, 1998.
[26] C. Liu and H. Wechsler, “Face Recognition Using Evolutionary Pursuit,” Proc. Fifth European Conf. Computer Vision, 1998.
[27] C. Liu and H. Wechsler, “Probabilistic Reasoning Models for Face Recognition,” Proc. Computer Vision and Pattern Recognition, 1998.
[28] C. Liu and H. Wechsler, “Comparative Assessment of Independent Component Analysis(IC A) for Face Recognition,” Proc. Second Int'l Conf. Audio- and Video-Based Biometric Person Authentication, 1999.
[29] C. Liu and H. Wechsler, “Face Recognition Using Shape and Texture,” Proc. Computer Vision and Pattern Recognition, 1999.
[30] C. Liu and H. Wechsler, “Robust Coding Schemes for Indexing and Retrieval from Large Face Databases,” IEEE Trans. Image Processing, vol. 9, no. 1, pp. 132-137, 2000.
[31] B.A. Olshausen and D.J. Field, “Emergence of Simple-Cell Receptive Field Properties by Learning a Sparse Code for Natural Images,” Nature, vol. 381, no. 13, pp. 607-609, 1996.
[32] P.S. Penev and J.J. Atick, “Local Feature Analysis: A General Statistical Theory for Object Representation,” Network: Computation in Neural Systems, vol. 7, pp. 477-500, 1996.
[33] P.J. Phillips, “Matching Pursuit Filters Applied to Face Identification,” IEEE Trans. Image Processing, vol. 7, no. 8, pp. 1,150-1,164, 1998.
[34] P.J. Phillips, H. Wechsler, J. Huang, and P. Rauss, “The FERET Database and Evaluation Procedure for Face-Recognition Algorithms,” Image and Vision Computing, vol. 16, pp. 295-306, 1998.
[35] T. Poggio and F. Girosi, “A Sparse Representation for Function Approximation,” Neural Computation, vol. 10, pp. 1,445-1,454, 1998.
[36] T. Poggio, V. Torre, and C. Koch, “Computational Vision and Regularization Theory,” Nature, vol. 317, pp. 314-319, 1985.
[37] D. Ruderman, “The Statistics of Natural Images,” Network: Computation in Neural Systems, vol. 5, pp. 598-605, 1994.
[38] A. Samal and P.A. Iyengar,“Automatic recognition and analysis of human faces and facial expressions: A survey,” Pattern Recognition, vol. 25, no. 1, pp. 65-77, 1992.
[39] P. Sinha and T. Poggio, “I Think I Know that Face...,” Nature, vol. 384, pp. 404, 1996.
[40] D.L. Swets and J. Weng, Using Discriminant Eigenfeatures for Image Retrieval IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 8, pp. 831-836, Aug. 1996.
[41] A.N. Tikhonov and V.Y. Arsenin, Solutions of Ill-Posed Problems, Washington: W.H. Winston, 1977.
[42] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J. Cognitive Neuroscience, vol. 13, no. 1, pp. 71-86, 1991.
[43] V.N. Vapnik, Statistical Learning Theory, John Wiley&Sons, 1998.

Index Terms:
Evolutionary pursuit, face recognition, genetic algorithms, optimal basis, Principal Component Analysis (PCA), Fisher Linear Discriminant (FLD).
Chengjun Liu, Harry Wechsler, "Evolutionary Pursuit and Its Application to Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 570-582, June 2000, doi:10.1109/34.862196
Usage of this product signifies your acceptance of the Terms of Use.