The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2013 vol.19)
pp: 824-837
Tao Chen , Dept. of Comput. Sci., Tsinghua Univ., Beijing, China
Ping Tan , Dept. of Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore
Li-Qian Ma , Dept. of Comput. Sci., Tsinghua Univ., Beijing, China
Ming-Ming Cheng , Dept. of Comput. Sci., Tsinghua Univ., Beijing, China
A. Shamir , Efi Arazi Sch. of Comput. Sci., Interdiscipl. Center, Herzelia, Israel
Shi-Min Hu , Dept. of Comput. Sci., Tsinghua Univ., Beijing, China
ABSTRACT
We present PoseShop - a pipeline to construct segmented human image database with minimal manual intervention. By downloading, analyzing, and filtering massive amounts of human images from the Internet, we achieve a database which contains 400 thousands human figures that are segmented out of their background. The human figures are organized based on action semantic, clothes attributes, and indexed by the shape of their poses. They can be queried using either silhouette sketch or a skeleton to find a given pose. We demonstrate applications for this database for multiframe personalized content synthesis in the form of comic-strips, where the main character is the user or his/her friends. We address the two challenges of such synthesis, namely personalization and consistency over a set of frames, by introducing head swapping and clothes swapping techniques. We also demonstrate an action correlation analysis application to show the usefulness of the database for vision application.
INDEX TERMS
Humans, Skin, Image segmentation, Image databases, Image color analysis, Shape,image composition, Image database
CITATION
Tao Chen, Ping Tan, Li-Qian Ma, Ming-Ming Cheng, A. Shamir, Shi-Min Hu, "PoseShop: Human Image Database Construction and Personalized Content Synthesis", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 5, pp. 824-837, May 2013, doi:10.1109/TVCG.2012.148
REFERENCES
[1] A. Shrivastava, T. Malisiewicz, A. Gupta, and A.A. Efros, "Data-Driven Visual Similarity for Cross-Domain Image Matching," ACM Trans. Graphics, vol. 30, no. 6, pp. 154:1-154:10, Dec. 2011.
[2] H. Liu, L. Zhang, and H. Huang, "Web-Image Driven Best Views of 3D Shapes," Visual Computer, vol. 28, no. 3, pp. 279-287, Mar. 2012.
[3] J.H. Hays and A.A. Efros, "Scene Completion Using Millions of Photographs," ACM Trans. Graphics, vol. 26, no. 3, pp. 4:1-4:7, 2007.
[4] D. Bitouk, N. Kumar, S. Dhillon, P. Belhumeur, and S.K. Nayar, "Face Swapping: Automatically Replacing Faces in Photographs," ACM Trans. Graphics, vol. 27, no. 3, pp. 39:1-39:8, 2008.
[5] L. Tao, L. Yuan, and J. Sun, "Skyfinder: Attribute-Based Sky Image Search," ACM Trans. Graphics, vol. 28, no. 3, pp. 68:1-68:5, 2009.
[6] B. Wang, Y. Yu, T.-T. Wong, C. Chen, and Y.-Q. Xu, "Data-Driven Image Color Theme Enhancement," ACM Trans. Graphics, vol. 29, no. 6, pp. 146:1-146:10, Dec. 2010.
[7] A.Y.-S. Chia, S. Zhuo, R.K. Gupta, Y.-W. Tai, S.-Y. Cho, P. Tan, and S. Lin, "Semantic Colorization with Internet Images," ACM Trans. Graphics, vol. 30, no. 6, pp. 156:1-156:8, Dec. 2011.
[8] J.-F. Lalonde, D. Hoiem, A.A. Efros, C. Rother, J. Winn, and A. Criminisi, "Photo Clip Art," ACM Trans. Graphics, vol. 26, no. 3, pp. 3:1-3:10, 2007.
[9] T. Chen, M.-M. Cheng, P. Tan, A. Shamir, and S.-M. Hu, "Sketch2photo: Internet Image Montage," ACM Trans. Graphics, vol. 28, no. 5, pp. 124:1-124:10, 2009.
[10] H. Huang, L. Zhang, and H.-C. Zhang, "Arcimboldo-Like Collage Using Internet Images," ACM Trans. Graphics, vol. 30, no. 6, pp. 155:1-155:8, Dec. 2011.
[11] M. Everingham, A. Zisserman, C.K.I. Williams, and L. Van Gool, "The PASCAL Visual Object Classes Challenge," Int'l J. Computer Vision, vol. 88, pp. 303-338, http://www.pascal-network.org/challenges/ VOC/voc2006results.pdf, 2006.
[12] G. Griffin, A. Holub, and P. Perona, "Caltech-256 Object Category Dataset," technical report, http://resolver.caltech.edu CaltechAUTHORS:CNS-TR-2007-001 , 2007.
[13] B.C. Russell, A. Torralba, K.P. Murphy, and W.T. Freeman, "Labelme: A Database and Web-Based Tool for Image Annotation," Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 157-173, 2008.
[14] N. Diakopoulos, I. Essa, and R. Jain, "Content Based Image Synthesis," Proc. Conf. Image and Video Retrieval (CIVR), pp. 299-307, 2004.
[15] M. Johnson, G.J. Brostow, J. Shotton, O. Arandjelović, V. Kwatra, and R. Cipolla, "Semantic Photo Synthesis," Computer Graphics Forum, vol. 25, no. 3, pp. 407-413, 2006.
[16] A. Georghiades, P. Belhumeur, and D. Kriegman, "From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[17] M. Enzweiler and D.M. Gavrila, "Monocular Pedestrian Detection: Survey and Experiments," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 12, pp. 2179-2195, Dec. 2008.
[18] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR), pp. 886-893, 2005.
[19] L. Sigal and M.J. Black, "Humaneva: Synchronized Video and Motion Capture Dataset for Evaluation of Articulated Human Motion," technical report, 2006.
[20] A. Sorokin and D. Forsyth, "Utility Data Annotation with Amazon Mechanical Turk," Proc. First IEEE Workshop Internet Vision at Computer Vision and Pattern Recognition, pp. 1-8, June 2008.
[21] J. Sullivan and S. Carlsson, "Recognizing and Tracking Human Action," Proc. European Conf. Computer Vision (ECCV), pp. 629-644, 2002.
[22] Y. Wang, H. Jiang, M.S. Drew, Z.-N. Li, and G. Mori, "Unsupervised Discovery of Action Classes," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1654-1661, 2006.
[23] N. Ikizler-Cinbis, R.G. Cinbis, and S. Sclaroff, "Learning Actions from the Web," Proc. IEEE Int'l Conf. Computer Vision (ICCV), pp. 995-1002, 2009.
[24] M.-W. Chao, C.-H. Lin, J. Assa, and T.-Y. Lee, "Human Motion Retrieval from Hand-Drawn Sketch," IEEE Trans. Visualization and Computer Graphics, vol. 18, no. 5, pp. 729-740, May 2012.
[25] M. Eitz, R. Richter, K. Hildebrand, T. Boubekeur, and M. Alexa, "Photosketcher: Interactive Sketch-Based Image Synthesis," IEEE Computer Graphics and Applications, vol. 31, no. 6, pp. 56-66, Nov./Dec. 2011.
[26] D. Kurlander, T. Skelly, and D. Salesin, "Comic Chat," Proc. ACM SIGGRAPH, pp. 225-236, 1996.
[27] A. Shamir, M. Rubinstein, and T. Levinboim, "Generating Comics from 3D Interactive Computer Graphics," IEEE Computer Graphics and Applications, vol. 26, no. 3, pp. 53-61, May 2006.
[28] J. Assa, Y. Caspi, and D. Cohen-Or, "Action Synopsis: Pose Selection and Illustration," ACM Trans. Graphics, pp. 667-676, 2005.
[29] S. Zhou, H. Fu, L. Liu, D. Cohen-Or, and X. Han, "Parametric Reshaping of Human Bodies in Images," ACM Trans. Graphics, vol. 29, pp. 126:1-126:10, July 2010.
[30] F. Xu, Y. Liu, C. Stoll, J. Tompkin, G. Bharaj, Q. Dai, H.-P. Seidel, J. Kautz, and C. Theobalt, "Video-Based Characters: Creating New Human Performances from a Multi-View Video Data base," ACM Trans. Graphics, vol. 30, pp. 32:1-32:10, Aug. 2011.
[31] P. Felzenszwalb, D. McAllester, and D. Ramanan, "A Discriminatively Trained, Multiscale, Deformable Part Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1-8, June 2008.
[32] P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Graph-Based Image Segmentation," Int'l J. Computer Vision, vol. 59, no. 2, pp. 167-181, 2004.
[33] M.J. Jones and J.M. Rehg, "Statistical Color Models with Application to Skin Detection," Int'l J. Computer Vision, vol. 46, no. 1, pp. 81-96, 2002.
[34] C. Rother, V. Kolmogorov, and A. Blake, "Grabcut: Interactive Foreground Extraction Using Iterated Graph Cuts," ACM Trans. Graphics, vol. 23, no. 3, pp. 309-314, 2004.
[35] Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, "Lazy Snapping," ACM Trans. Graphics, vol. 23, no. 3, pp. 303-308, 2004.
[36] A. Hernandez, M. Reyes, S. Escalera, and P. Radeva, "Spatio-Temporal Grabcut Human Segmentation for Face and Pose Recovery," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 33-40, June 2010.
[37] S. Belongie, J. Malik, and J. Puzicha, "Shape Matching and Object Recognition Using Shape Contexts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002.
[38] J. Ho, A. Peter, A. Rangarajan, and M.-H. Yang, "An Algebraic Approach to Affine Registration of Point Sets," Proc. IEEE Int'l Conf. Computer Vision (ICCV), pp. 1-8, 2009.
[39] P.F. Felzenszwalb and J.D. Schwartz, "Hierarchical Matching of Deformable Shapes," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1-8, 2007.
[40] D. Gavrila, "A Bayesian, Exemplar-Based Approach to Hierarchical Shape Matching," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 8, pp. 1408-1421, Aug. 2007.
[41] B.J. Frey and D. Dueck, "Clustering by Passing Messages between Data Points," Science, vol. 315, pp. 972-976, 2007.
[42] C. Harris and M. Stephens, "A Combined Corner and Edge Detection," Proc. Fourth Alvey Vision Conf., pp. 147-151, 1988.
[43] A. Levin, D. Lischinski, and Y. Weiss, "A Closed-Form Solution to Natural Image Matting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 228-242, Feb. 2008.
[44] E. Reinhard, M. Ashikhmin, B. Gooch, and P. Shirley, "Color Transfer between Images," IEEE Computer Graphics Applications, vol. 21, no. 5, pp. 34-41, 2001.
[45] Z. Lin and L. Davis, "Shape-Based Human Detection and Segmentation via Hierarchical Part-Template Matching," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 4, pp. 604-618, Apr. 2010.
[46] J.-F. Lalonde, A.A. Efros, and S.G. Narasimhan, "Estimating Natural Illumination from a Single Outdoor Image," Proc. IEEE Int'l Conf. Computer Vision (ICCV), pp. 183-190, 2009.
[47] Y. Zhang and R. Tong, "Environment-Sensitive Cloning in Images," Visual Computing, vol. 27, nos. 6-8, pp. 739-748, June 2011.
55 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool