Varying illumination and partial occlusion are two main difficulties in visual tracking. Existing methods based on appearance information cannot solve these problems effectively since appearance is sensitive to lighting and the appearances under occlusions are quite different. In this paper, we propose a descriptor-based dynamic tracking approach that can track objects under partial occlusions and varying illumination. Instead of global appearance, an object is represented by a set of invariant feature descriptors that are generated from local regions around some salient points. By integrating the local descriptor information into the observation model, our method is effective under varying illumination and partial occlusions.