The ability to track multiple people and their body parts (i.e., face and hands) in a complex environment is crucial for designing a collaborative natural human computer interaction (HCI). One of the challenging issues in tracking body parts of people is the data association uncertainty while assigning measurements to the proper tracks in the case of occlusion and close interaction of body parts of different people. This paper describes a framework for tracking body parts of people in 2D/3D using multiple hypothesis tracking (MHT) algorithm. A path coherence function has been incorporated along with MHT to reduce the negative effects of closely spaced measurements that produce unconvincing tracks and needless computations. The performance of the framework has been validated using experiments on real sequence of images.