Human identification from gait is a challenging task in realistic surveillance scenarios in which people walking along arbitrary directions are imaged by a single camera. In this paper, motivated by the view-invariance issue in the human ID from gait problem, we address the general problem of classifying the "content" of human motions of unknown "style". Given a dataset of sequences in which different people walking normally are seen from several wellseparated views, we propose a three-layer scheme based on bilinear models, in which image sequences are mapped to observation vectors of fixed dimension using Markov modeling. We test our approach on the CMU Mobo database, showing how bilinear separation outperforms other approaches, opening the way to view- and action-invariant identity recognition, as well as subject- and view-invariant action recognition.