The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - Sept. (2013 vol.19)
pp: 1552-1565
S. Hauswiesner , Inst. for Comput. Graphics & Vision, Graz Univ. of Technol., Graz, Austria
M. Straka , Inst. for Comput. Graphics & Vision, Graz Univ. of Technol., Graz, Austria
G. Reitmayr , Inst. for Comput. Graphics & Vision, Graz Univ. of Technol., Graz, Austria
ABSTRACT
Virtual try-on applications have become popular because they allow users to watch themselves wearing different clothes without the effort of changing them physically. This helps users to make quick buying decisions and, thus, improves the sales efficiency of retailers. Previous solutions usually involve motion capture, 3D reconstruction or modeling, which are time consuming and not robust for all body poses. Our method avoids these steps by combining image-based renderings of the user and previously recorded garments. It transfers the appearance of a garment recorded from one user to another by matching input and recorded frames, image-based visual hull rendering, and online registration methods. Using images of real garments allows for a realistic rendering quality with high performance. It is suitable for a wide range of clothes and complex appearances, allows arbitrary viewing angles, and requires only little manual input. Our system is particularly useful for virtual try-on applications as well as interactive games.
INDEX TERMS
Clothing, Databases, Cameras, Rendering (computer graphics), Feature extraction, Shape, Runtime,virtual try-on, Mixed reality, augmented reality, image-based rendering
CITATION
S. Hauswiesner, M. Straka, G. Reitmayr, "Virtual Try-On through Image-Based Rendering", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 9, pp. 1552-1565, Sept. 2013, doi:10.1109/TVCG.2013.67
REFERENCES
[1] A. Divivier, R. Trieb, and A.e.a. Ebert, "Virtual Try-On: Topics in Realistic, Individualized Dressing in Virtual Reality," Proc. Virtual and Augmented Reality Status Conf., 2004.
[2] S. Hauswiesner, M. Straka, and G. Reitmayr, "Image-Based Clothes Transfer," Proc. IEEE 10th Int'l Symp. Mixed and Augmented Reality (ISMAR), 2011.
[3] J. Gall, C. Stoll, E. de Aguiar, C. Theobalt, B. Rosenhahn, and H.-P. Seidel, "Motion Capture Using Joint Skeleton Tracking and Surface Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2009.
[4] A.O. Balan, L. Sigal, M.J. Black, J.E. Davis, and H.W. Haussecker, "Detailed Human Shape and Pose from Images," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2007.
[5] D. Vlasic, I. Baran, W. Matusik, and J. Popović, "Articulated Mesh Animation from Multi-View Silhouettes," ACM Trans. Graphics, vol. 27, no. 3, pp. 1-9, 2008.
[6] M. Straka, S. Hauswiesner, M. Ruther, and H. Bischof, "Rapid Skin: Estimating the 3D Human Pose and Shape in Real-Time," Proc. Second Int'l Conf. 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 41-48, 2012.
[7] G. Cheung, S. Baker, J. Hodgins, and T. Kanade, "Markerless Human Motion Transfer," Proc. ACM SIGGRAPH '04 Sketches, p. 31, http://doi.acm.org/10.11451186223.1186262 , 2004.
[8] M. Richter, K. Varanasi, N. Hasler, and C. Theobalt, "Real-Time Reshaping of Humans," Proc. Second Int'l Conf. 3D Imaging, Modeling, Processing, Visualization Transmission (3DIMPVT), pp. 340-347, 2012.
[9] V. Ganapathi, C. Plagemann, D. Koller, and S. Thrun, "Real Time Motion Capture Using a Single Time-of-Flight Camera," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2010.
[10] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake, "Real-Time Human Pose Recognition in Parts from Single Depth Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011.
[11] E. de Aguiar, L. Sigal, A. Treuille, and J.K. Hodgins, "Stable Spaces for Real-Time Clothing," Proc. ACM SIGGRAPH '10 Papers, pp. 106:1-106:9, http://doi.acm.org/10.11451833349.1778843 , 2010.
[12] D. Bradley, T. Popa, A. Sheffer, W. Heidrich, and T. Boubekeur, "Markerless Garment Capture," Proc. ACM SIGGRAPH '08 Papers, pp. 99:1-99:9, http://doi.acm.org/10.11451399504.1360698 , 2008.
[13] M. Salzmann, J. Pilet, S. Ilic, and P. Fua, "Surface Deformation Models for Nonrigid 3D Shape Recovery," IEEE Trans. Pattern Analysis Machine Intelligence, vol. 29, no. 8, pp. 1481-1487, http://dx.doi.org/10.1109TPAMI.2007.1080 , Aug. 2007.
[14] D. Vlasic, P. Peers, I. Baran, P. Debevec, J. Popović, S. Rusinkiewicz, and W. Matusik, "Dynamic Shape Capture Using Multi-View Photometric Stereo," Proc. ACM SIGGRAPH Asia '09 Papers, pp. 174:1-174:11, http://doi.acm.org/10.11451661412.1618520 , 2009.
[15] C. Stoll, J. Gall, E. de Aguiar, S. Thrun, and C. Theobalt, "Video-Based Reconstruction of Animatable Human Characters," Proc. ACM SIGGRAPH Asia '10 Papers, pp. 139:1-139:10, http://doi.acm.org/10.11451866158.1866161 , 2010.
[16] J. Li, J. Ye, Y. Wang, L. Bai, and G. Lu, "Technical Section: Fitting 3D Garment Models onto Individual Human Models," Computer Graphics, vol. 34, pp. 742-755, http://dx.doi.org/10.1016j.cag.2010.07.008 , Dec. 2010.
[17] J. Ehara and H. Saito, "Texture Overlay for Virtual Clothing Based on PCA of Silhouettes," Proc. Fifth Int'l Symp. Mixed and Augmented Reality (ISMAR '06), pp. 139-142, http://dx.doi.org/10.1109ISMAR.2006.297805 , 2006.
[18] H. Tanaka and H. Saito, "Texture Overlay onto Flexible Object with PCA of Silhouettes and K-Means Method for Search into Database," Proc. IAPR Conf. Machine Vision Applications, 2009.
[19] Z. Zhou, B. Shu, S. Zhuo, X. Deng, P. Tan, and S. Lin, "Image-Based Clothes Animation for Virtual Fitting," Proc. ACM SIGGRAPH Asia '12 Technical Briefs, pp. 33:1-33:4, http://doi.acm.org/10.11452407746.2407779 , 2012.
[20] P. Volino and N. Magnenat-Thalmann, "Accurate Garment Prototyping and Simulation," Computer-Aided Design & Applications, CAD Solutions, vol. 2, no. 5, p. 645, http://citeseerx.ist. psu.edu/viewdocsummary?doi=10.1.1.94.1310 , 2005.
[21] A. Weiss, D. Hirshberg, and M. Black, "Home 3D Body Scans from Noisy Image and Range Data," Proc. IEEE Int'l Conf. Computer Vision (ICCV), pp. 1951-1958, 2011.
[22] J. Tong, J. Zhou, L. Liu, Z. Pan, and H. Yan, "Scanning 3D Full Human Bodies Using Kinects," IEEE Trans. Visualization and Computer Graphics, vol. 18, no. 4, pp. 643-650, Apr. 2012.
[23] J. Li and G. Lu, "Customizing 3D Garments Based on Volumetric Deformation," Computers in Industry, vol. 62, no. 7, pp. 693-707, http://www.sciencedirect.com/science/article/ piiS0166361511000443, 2011.
[24] A. Hilton, D. Beresford, T. Gentils, R. Smith, and W. Sun, "Virtual People: Capturing Human Models to Populate Virtual Worlds," Proc. Computer Animation (CA '99), pp. 174-185, http://portal. acm.orgcitation.cfm?id=791217.791567 , 1999.
[25] A. Hilsmann and P. Eisert, "Tracking and Retexturing Cloth for Real-Time Virtual Clothing Applications," Proc. Fourth Int'l Conf. Computer Vision/Computer Graphics CollaborationTechniques (MIRAGE '09), pp. 94-105, http://dx.doi.org/10.1007978-3-642-01811-4_9 , 2009.
[26] P. Eisert, P. Fechteler, and J. Rurainsky, "3-D Tracking of Shoes for Virtual Mirror Applications," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-6, June 2008.
[27] I. Kemelmacher-Shlizerman, A. Sankar, E. Shechtman, and S.M. Seitz, "Being John Malkovich," Proc. 11th European Conf. Computer Vision (ECCV), pp. 341-353, 2010.
[28] K. Dale, K. Sunkavalli, M.K. Johnson, D. Vlasic, W. Matusik, and H. Pfister, "Video Face Replacement," ACM Trans. Graphics, vol. 30, no. 6, p. 130, 2011.
[29] M. Straka, S. Hauswiesner, M. Ruether, and H. Bischof, "A Free-Viewpoint Virtual Mirror with Marker-Less User Interaction," Proc. 17th Scandinavian Conf. Image Analysis, 2011.
[30] S. Hauswiesner, M. Straka, and G. Reitmayr, "Coherent Image-Based Rendering of Real-World Objects," Proc. Symp. Interactive 3D Graphics and Games, pp. 183-190, http://doi.acm.org/10.11451944745.1944776 , 2011.
[31] W. Matusik, C. Buehler, R. Raskar, S.J. Gortler, and L. McMillan, "Image-Based Visual Hulls," Proc. ACM SIGGRAPH '00, pp. 369-374, 2000.
[32] P. Debevec, Y. Yu, and G. Boshokov, "Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping," technical report, Univ. of California at Berkeley, Berkeley, CA, http://portal.acm.orgcitation.cfm?id=893689 , 1998.
[33] M. Botsch and O. Sorkine, "On Linear Variational Surface Deformation Methods," IEEE Trans. Visualization and Computer Graphics, vol. 14, no. 1, pp. 213-230, Jan./Feb. 2008.
[34] M.J.D. Powell, "Convergence Properties of Algorithms for Nonlinear Optimization," SIAM Rev., vol. 28, no. 4, pp. 487-500, http://www.jstor.org/stable2031100, 1986.
[35] H.Y. Shum, S.C. Chan, and S.B. Kang, Image-Based Rendering. Springer-Verlag, 2006.
[36] R. Mantiuk, K.J. Kim, A.G. Rempel, and W. Heidrich, "HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in All Luminance Conditions," Proc. ACM SIGGRAPH, pp. 40:1-40:14, http://doi.acm.org/10.11451964921.1964935 , 2011.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool