Object detection in video surveillance is typically done through background subtraction or temporal differencing. While these techniques perform very well under scenes where there are minimal light changes, they begin to fail when the scene contains rapid illumination changes. The effects of this is most profound in indoor environments. Under these conditions, the background modeling techniques produce large numbers of false positives. This paper proposes a sequential approach to suppressing these false positives. Both frame and region level spatial scales are considered to detect sudden light changes and make use of Gabor filter responses and edge maps to identify and remove false positives.