In this paper, we present a novel multiscale approach to recovery of non-rigid motion from sequences of registered intensity and range images. The main idea of our approach is that a finite element (FEM) model can naturally handle both registration and deformation modeling using a single model-driving strategy. The method includes a multiscale iterative algorithm based on analysis of the undirected Hausdorff distance to recover correspondences. The method is evaluated with respect to speed, accuracy, and noise sensitivity. Advantages of the proposed approach are demonstrated using man-made elastic materials and human skin motion. Experiments with regular grid features are used for performance comparison with a conventional approach (separate snakes and FEM models). It is shown that the new method does not require a grid and can adapt the model to available object features.