Human face detection plays an important role in real life application such as Face recognition, Human computer interface, Video surveillance and Face image database management. Automatic Face detection in such scenario is a challenging task. Here we propose to use skin color cue to detect face in a video sequence by modeling the skin color by an one class classifier named SVDD (Support Vector Data Descriptor). We apply the one class classifier for facial skin model on Various color transformations (YCrCb, HSV, Normalized RGB). We also examine SVDD on all the three Y, Cb and Cr and their combinations for the skin model as it is known that the non linear combination of the three gives better skin model and experiments on large set of data sets proves the robustness of the method.