Consider a color camera mounted at a fixed point and a target object to be tracked. We propose a method to replace the common “blue screen” technique in performing segmentation of the object. The target object image in the video sequence is separated from the background through a segmentation process employing the color mixture model techniques. The background information is recovered through a similarity search with the panorama of the original background. The foreground segment can then be encoded by traditional compression while the scene background is represented as a panorama. Finally, the foreground object combines with the corresponding panoramic segment or any desired image on the fly to reconstruct the video frame. Our system can be used in low bandwidth applications as well as special demonstration in which the demonstrators can appear/disappear at any time he/she wishes.