In this paper evolutionary classifier fusion method is used to optimize the performance of face recognition system. Initially different illumination environments are modeled as multiple contexts using unsupervised learning and then optimized classifier ensemble are searched for each context using Genetic Algorithm (GA). For each context multiple optimized classifiers are searched each of which are referred as context based classifier. Then evolutionary framework of combination of such classifiers is applied to optimize the face recognition as a whole. Evolutionary classifier fusion is compared with the single classifier system.Experiment is done using real time Inha database and FERET database. Experimental results show that the proposed multiple context based fusion method gives superior performance than the method without using fusion and optimize face recognition performance.