This paper presents motion field histograms as a new way of extracting facial features and modeling expressions. Features are based on local receptive field histograms, which are robust against errors in rotation, translation and scale changes during image alignment. Motion information is incorporated into the histograms by using difference images instead of raw images. We take the principal components of these histograms of selected facial regions and use the top 20 eigenvectors for compact representation. The eigen-coefficients are then used to model the temporal structure of different facial expressions from real-life data in the presence of translational and rotational errors that arise from head tracking. The results demonstrate a 44% average performance increase over traditional optic flow methods for expressions extracted from unconstrained interactions.