We propose criteria for a feature space for face image processing and a method for generating such a space. Beginning with many input dimensions, including deformation vectors (obtained through optical flow analysis between an input image and a neutral template) and deformation residues, we apply principal component analysis and Fisher's classification criterion to derive a feature space. We demonstrate classification in two important tasks - face detection and expression analysis - in each case using only one linear discriminant, thereby demonstrating that the feature space fulfils a restricted version of the criteria.