Many video surveillance and identification applications need to find moving objects in the field of view of a stationary camera. A popular method for obtaining these silhouettes is through the process of background subtraction. We present a novel method for comparing image frames to the model of the stationary background that exploits the spatial and temporal dependencies that objects in motion impose on their images. We achieve this through the development and use of Markov random fields of binary segmentation variates. We show that the MRF approach produces more accurate and visually appealing silhouettes that are less prone to noise and background camouflaging effects than traditional per-pixel based methods. Results include visual examination of silhouettes, comparisons against hand-segmented data, and an analysis of the effects of various silhouette extraction techniques on gait recognition performance.