In this paper we propose a new general framework for real-time multi-view face recognition in real-world conditions, based on a novel nonlinear dimensionality reduction method IsoScale and Generalized Linear Models (GLMs). Multi-view face sequences of freely moving people are obtained from several stereo cameras installed in an ordinary room, and IsoScale is used to map the faces into a low-dimensional space where the manifold structure of the view-varied faces is preserved, but the face classes are forced to be linearly separable. Then a GLM-based linear map is learnt between the low-dimensional face representation and the classes, providing posterior probabilities of class membership for the test faces. The benefits of the proposed method are illustrated in a typical HCI application.