We present a dynamic inference algorithm in a globally parameterized nonlinear manifold and demonstrate it on the problem of visual tracking. An appearance manifold is usually nonlinear, embedded in a high dimensional space, and can be approximated by a mixture of locally linear models. Existing methods for nonlinear dimensionality reduction, which map an appearance manifold to a single low dimensional coordinate system, preserve only spatial relationships among manifold points and render low dimensional embeddings rather than mapping functions. In this paper, we parameterize the mixture of linear appearance subspaces of an object in a global coordinate system, and apply it to visual tracking using a Rao-Blackwellized particle filter. Experimental results demonstrate that the proposed approach performs well on object tracking problem in scenes with significant clutter and temporary occlusions which pose difficulties for other methods.