Humans are articulated objects composed of non-rigid parts. We are interested in detecting and tracking human motions over various periods of time. In this paper we describe a method of detecting and tracking human body parts in color video sequences. The dominant motion region is detected using normal flow ; Expectation Maximization, uniform sampling, and a shortest path algorithm are used to find the bounding contour for the moving arm. An affine motion model is fi t to the arm region; residual analysis and outlier rejection are used for robust parameter estimation. The estimated parameters are used for the prediction of the location of the moving limb in the next frame. Detection and tracking results are combined to account for the deviations from the affine flow model and increase the robustness of the method. We demonstrate our method on several long image sequences corresponding to different limb movements.