Vision-based gesture recognition systems require identifying and tracking the boundaries of skin segments. In this article, we present four variations of the Mean-Shift algorithm with the applications of hand and face tracking in user interface applications. The first variation is based on continuous sampling of the boundaries of the kernel and changing the size of the kernel. The second variation addresses tracking of multiple skin blobs in the image sequence. The third algorithm provides a high level of tracking which we cal a "Macro tracker" for tracking activity from multiple trackers. The fourth variation addresses the application of depth information in blob tracking. This additional variable makes the algorithm more robust in handling occlusion, which often occurs in user interface applications.