A new algorithm for recovering 3-D camera motion from sequences of images is proposed. The algorithm has four stages. In the first stage, the motion vector field is segmented using a novel EM-based method. The resulting segments are compared and the coherent regions are merged in the second stage. The candidates for the background regions are determined and finally used for 3-D motion estimation in the last two stages. Unlike most of the other methods, this approach tolerates also non-rigid motion in the scene. The experiments performed show that in some cases more information or reasoning is needed for selecting plausible motion parameters from several hypotheses.