This paper presents a new approach to represent face by using a non-tensor product bivariate wavelet filters. A new non-tensor product bivariate wavelet filter banks with linear phase are constructed from the centrally symmetric matrices. Our investigations demonstrate that these filter banks have a matrix factorization and they are capable of representing facial features for recognition. The implementations of our algorithm are made of three parts: First, face images are represented by the lowest resolution subbands after 2-level new non-tensor product wavelet decomposition. Second, the Principal Component Analysis (PCA) feature selection scheme is adopted to reduce the computational complexity of feature representation. Finally, Support Vector Machines (SVM) is applied for classification.The experimental results show that our method is superior to other methods in terms of recognition accuracy and efficiency.