Particle filtering is an efficient and successful technique for tracking 2D and 3D motion through an image. We present the enhanced tracking of two hands based on a statistical model using only a skin colour feature with particle filtering for gesture recognition. Our framework employs one particle filter per hand individually with the pixel-wise classification of the likelihood of the skin in the window search. The skin classifier decision was trained from a set of skin samples in YCrCb space using an elliptical model. The tracking scheme employs a reliability measurement derived from the particle distribution which is used to adaptively weight the colour classification. The K-means algorithm is used to discriminate the split and merge between left and right hand. Experiments with a set of videos including the movement of two hands in cluttered backgrounds show that adaptive use of our scheme provides improvement compared to use with other techniques such as mean-shift tracking.