This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Silhouette Segmentation in Multiple Views
July 2011 (vol. 33 no. 7)
pp. 1429-1441
Wonwoo Lee, Gwnagju Institute of Science and Technology, Gwangju
Woontack Woo, Gwnagju Institute of Science and Technology, Gwangju
Edmond Boyer, INRIA Grenoble Rhône-Alpes, Saint Ismier
In this paper, we present a method for extracting consistent foreground regions when multiple views of a scene are available. We propose a framework that automatically identifies such regions in images under the assumption that, in each image, background and foreground regions present different color properties. To achieve this task, monocular color information is not sufficient and we exploit the spatial consistency constraint that several image projections of the same space region must satisfy. Combining the monocular color consistency constraint with multiview spatial constraints allows us to automatically and simultaneously segment the foreground and background regions in multiview images. In contrast to standard background subtraction methods, the proposed approach does not require a priori knowledge of the background nor user interaction. Experimental results under realistic scenarios demonstrate the effectiveness of the method for multiple camera set ups.

[1] C. Hernández and F. Schmitt, "Silhouette and Stereo Fusion for 3D Object Modeling," Computer Vision and Image Understanding, vol. 96, no. 3, pp. 367-392, Dec. 2004.
[2] Y. Furukawa and J. Ponce, "Carved Visual Hulls for Image-Based Modeling," Proc. European Conf. Computer Vision, pp. 564-577, 2006.
[3] A. Zaharescu, E. Boyer, and R. Horaud, "Transformesh: A Topology-Adaptive Mesh-Based Approach to Surface Evolution," Proc. Asian Conf. Computer Vision, pp. 166-175, 2007.
[4] C. Wren, A. Azarbayejani, T. Darrell, and A. Pentland, "Pfinder: Real-Time Tracking of the Human Body," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780-785, July 1997.
[5] S. Rowe and A. Blake, "Statistical Mosaics for Tracking," Image and Vision Computing, vol. 14, pp. 549-564, 1996.
[6] N. Friedman and S. Russell, "Image Segmentation in Video Sequences: A Probabilistic Approach," Proc. 13th Conf. Uncertainty in Artificial Intelligence, 1997.
[7] C. Stauffer and W. Grimson, "Adaptative Background Mixture Models for Real-Time Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 246-252, 1999.
[8] K. Toyama, J. Krumm, B. Brumitt, and B. Meyers, "Wallflower: Principles and Practice of Background Maintenance," Proc. Int'l Conf. Computer Vision, pp. 255-261, Sept. 1999.
[9] A.M. Elgammal, D. Harwood, and L.S. Davis, "Non-Parametric Model for Background Subtraction," Proc. European Conf. Computer Vision, pp. 751-767, 2000.
[10] Y. Boykov and M.-P. Jolly, "Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images," Proc. Int'l Conf. Computer Vision, vol. 1, pp. 105-112, 2001.
[11] C. Rother, V. Kolmogorov, and A. Blake, "GrabCut-Interactive Goreground Extraction Using Iterated Graph Cuts," Proc. ACM SIGGRAPH, vol. 24, no. 3, pp. 309-314, 2004.
[12] Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, "Lazy Snapping," Proc. ACM SIGGRAPH, vol. 23, no. 3, pp. 303-308, 2004.
[13] D. Freedman and T. Zhang, "Interactive Graph Cut Based Segmentation with Shape Priors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 755-762, June 2005.
[14] N. Vu and B. Manjunath, "Shape Prior Segmentation of Multiple Objects with Graph Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2008.
[15] J. Sun, W. Zhang, X. Tang, and H.-Y. Shum, "Background Cut," Proc. European Conf. Computer Vision, pp. 628-641, 2006.
[16] Y. Li, J. Sun, and H.-Y. Shum, "Video Object Cut and Paste," ACM Trans. Graphics, vol. 24, no. 3, pp. 595-600, 2005.
[17] J. Wang, P. Bhat, A. Colburn, M. Agrawala, and M. Cohen, "Interactive Video Cutout," Proc. ACM SIGGRAPH, pp. 585-594, 2005.
[18] B. Micusik and A. Hanbury, "Automatic Image Segmentation by Positioning a Seed," Proc. European Conf. Computer Vision, pp. 468-480, 2006.
[19] E.N. Mortensen and J. Jia, "Real-Time Semi-Automatic Segmentation Using a Bayesian Network," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 1007-1014, 2006.
[20] I. Kompatsiaris, D. Tzovaras, and M.G. Strintzis, "3D Model-Based Segmentation of Videoconference Image Sequences," IEEE Trans. Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 547-561, Sept. 1998.
[21] G. Gordon, T. Darrell, M. Harville, and J. Woodfill, "Background Estimation and Removal Based on Range and Color," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 459-464, 1999.
[22] V. Kolmogorov, A. Criminisi, A. Blake, G. Cross, and C. Rother, "Probabilistic Fusion of Stereo with Color and Contrast for Bi-Layer Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1480-1492, Sept. 2006.
[23] G. Zeng and L. Quan, "Silhouette Extraction from Multiple Images of an Unknown Background," Proc. Asian Conf. Computer Vision, vol. 2, pp. 628-633, 2004.
[24] M. Sormann, C. Zach, and K. Karner, "Graph Cut Based Multiple View Segmentation for 3d Reconstruction," Proc. Int'l Symp. 3D Data Processing, Visualization and Transmission, 2006.
[25] M. Bray, P. Kohli, and P.H. Torr, "PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts," Proc. European Conf. Computer Vision, pp. 642-655, 2006.
[26] N. Campbell, G. Vogiatzis, C. Hernández, and R. Cipolla, "Automatic 3D Object Segmentation in Multiple Views Using Volumetric Graph-Cuts," Proc. British Machine Vision Conf., vol. 1, pp. 530-539, 2007.
[27] D. Snow, P. Viola, and R. Zabih, "Exact Voxel Occupancy with Graph Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 345-352, 2000.
[28] J.-S. Franco and E. Boyer, "Fusion of Multi-View Silhouette Cues Using a Space Occupancy Grid," Proc. IEEE Int'l Conf. Computer Vision, pp. 1747-1753, 2005.
[29] L. Guan, J.-S. Franco, and M. Pollefeys, "Multi-Object Shape Estimation and Tracking from Silhouette Cues," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[30] E. Boyer, "On Using Silhouettes for Camera Calibration," Proc. Asian Conf. Computer Vision, pp. 1-10, Jan. 2006.
[31] A. Laurentini, "The Visual Hull Concept for Silhouette-Based Image Understanding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 2, pp. 150-162, Feb. 1994.
[32] N. Kim, W. Woo, G. Kim, and C.-M. Park, "3D Virtual Studio for Natural Inter-Acting," IEEE Trans. Systems, Man, and Cybernetics, Part A, vol. 36, no. 4, pp. 758-773, July 2006.
[33] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[34] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[35] D. Nister, "An Efficient Solution to the Five-Point Relative Pose Problem," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 756-777, June 2004.
[36] N. Snavely, S.M. Seitz, and R. Szeliski, "Modeling the World from Internet Photo Collections," Int'l J. Computer Vision, vol. 80, no. 2, pp. 189-210, Nov. 2008.
[37] X. Bai, J. Wang, D. Simons, and G. Sapiro, "Video Snapcut: Robust Video Object Cutout Using Localized Classifiers," ACM Trans. Graphics, vol. 28, no. 3, pp. 1-11, 2009.
[38] C. Xu and J. Prince, "Snakes, Shapes, and Gradient Vector Flow," IEEE Trans. Image Processing, vol. 7, no. 3, pp. 359-369, Mar. 1998.
[39] S.M. Khan and M. Shah, "Reconstructing Non-Stationary Articulated Objects in Monocular Video Using Silhouette Information," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[40] P.-L. Lai and A. Yilmaz, "Efficient Object Shape Recovery via Slicing Planes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-6, 2008.
[41] "Temple, Data Set," Multi-View Stereo Evaluation Web Page, http://vision.middlebury.edumview/, 2010.
[42] "Kung-Fu Girl, Data Set," http://www.mpi-inf.mpg.de/ departments/ irg3kungfu/, 2010.
[43] "Dancer, Data Set," Multiple-Camera/Multiple-Video Database, https://charibdis.inrialpes.fr/htmlindex.php , 2009.
[44] "Bust, Data Set," http://www.cs.ust.hk/quan/WebPamipami. html , 2010.
[45] "Violet, Data Set," Multi-View Stereo Datasets, http://www.cs.toronto.edu/kyros/soft-data/ staticindex.html, 2010.
[46] "GML C++ Camera Calibration Toolbox," http://graphics.cs. msu.ru/en/science/research/ calibrationcpp, 2009.
[47] "Camera Calibration Toolbox for Matlab," http://www.vision. caltech.edu/bouguetjcalib_doc , 2009.
[48] C. Wu, "SiftGPU: A GPU Implementation of Scale Invariant Feature Transform (SIFT)," http://cs.unc.edu/ccwusiftgpu, 2007.

Index Terms:
Background region, foreground region, multiview silhouette consistency, silhouette segmentation.
Citation:
Wonwoo Lee, Woontack Woo, Edmond Boyer, "Silhouette Segmentation in Multiple Views," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 7, pp. 1429-1441, July 2011, doi:10.1109/TPAMI.2010.196
Usage of this product signifies your acceptance of the Terms of Use.