A new method called Two-Dimensional Fisher Discriminant Analysis (2D-FDA) is proposed to deal with the Small Sample Size (SSS) problem in LDA based face recognition. Then appearance and depth information are combined to improve face recognition rate. Different from the conventional 1D-FDA (PCA plus LDA) approaches, 2D-FDA is based on 2D image matrices rather than column vectors so the image matrix does not need to be transformed into a long vector before feature extraction. The advantage arising in this way is that the SSS problem does not exist any more because the between-class and withinclass scatter matrices constructed in 2D-FDA are both of full-rank. It was verified that 2D-FDA outperforms 1D FDA.