ABSTRACT: This paper presents an iterative maximum likelihood framework for motion segmentation. Our representation of the segmentation problem is based on a similarity matrix for the motion vectors for pairs of pixel blocks. By applying eigen decomposition to the similarity matrix, we develop a maximum likelihood method for grouping the pixel blocks into objects which share a common motion vector. We experiment with the resulting clustering method on a number of real world motion sequences. Here ground truth data indicates that the method can result in motion classification errors as low as 3%.