Detecting regions of interest in video sequences is the most important task in many high level video processing applications. In this paper a robust technique based on recursive learning of video background and foreground models is presented. Our contributions can be described along four directions. First, a recursive learning scheme is developed to build pixel models based on their colors. Second, we generate background and foreground models to enforce the temporal consistency of detected foregrounds. Third, we exploit dependencies between pixel colors to insure that the model is not restricted to using only independent features. Finally, an adaptive pixel-wise criterion is proposed that incorporates different spatial situations in the scene.