This paper presents a novel head detection algorithm based on contour analysis on depth images. A sequence of depth-valued images is used as the system input using a 3D time-of-flight depth sensor. The background and foreground layers of the image are segmented using a straightforward depth thresholding technique. Moving regions are further processed in each frame, and contour analysis is performed on the depth maps to extract the curves of moving regions. Finally, ellipse fitting is performed to determine the objective head targets in the image. This information will be passed to the tracker in order to accomplish tracking the targets in the scene. Experimental results demonstrate the efficiency of the proposed method.