The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - Aug. (2013 vol.19)
pp: 1375-1385
Ruo-Feng Tong , State Key Lab. of CAD&CG, Zhejiang Univ., Hangzhou, China
Yun Zhang , State Key Lab. of CAD&CG, Zhejiang Univ., Hangzhou, China
Ke-Li Cheng , State Key Lab. of CAD&CG, Zhejiang Univ., Hangzhou, China
ABSTRACT
We propose "StereoPasting,” an efficient method for depth-consistent stereoscopic composition, in which a source 2D image is interactively blended into a target stereoscopic image. As we paint "disparity” on a 2D image, the disparity map of the selected region is gradually produced by edge-aware diffusion, and then blended with that of the target stereoscopic image. By considering constraints of the expected disparities and perspective scaling, the 2D object is warped to generate an image pair, which is then blended into the target image pair to get the composition result. The warping is formulated as an energy minimization, which could be solved in real time. We also present an interactive composition system, in which users can edit the disparity maps of 2D images by strokes, while viewing the composition results instantly. Experiments show that our method is intuitive and efficient for interactive stereoscopic composition. A lot of applications demonstrate the versatility of our method.
INDEX TERMS
Stereo image processing, Videos, Image color analysis, Image edge detection, Visualization, Equations, Paints, disparity map, StereoPasting, depth consistent, stereoscopic composition
CITATION
Ruo-Feng Tong, Yun Zhang, Ke-Li Cheng, "StereoPasting: Interactive Composition in Stereoscopic Images", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 8, pp. 1375-1385, Aug. 2013, doi:10.1109/TVCG.2012.319
REFERENCES
[1] X. An and F. Pellacini, "AppProp: All-Pairs Appearance-Space Edit Propagation," ACM Trans. Graphics, vol. 27, no. 3, pp. 40:1-40:9, 2008.
[2] C. Barnes, E. Shechtman, A. Finkelstein, and D.B. Goldman, "Patchmatch: A Randomized Correspondence Algorithm for Structural Image Editing," ACM Trans. Graphics, vol. 28, no. 3, pp. 24:1-24:11, 2009.
[3] M. Brown, R. Szeliski, and S.A.J. Winder, "Multi-Image Matching Using Multi-Scale Oriented Patches," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR), pp. 510-517, 2005.
[4] C.-H. Chang, C.-K. Liang, and Y.-Y. Chuang, "Content-Aware Display Adaptation and Interactive Editing for Stereoscopic Images," IEEE Trans. Multimedia, vol. 13, no. 4, pp. 589-601, Aug. 2011.
[5] T. Chen, M.-M. Cheng, P. Tan, A. Shamir, and S.-M. Hu, "Sketch2photo: Internet Image Montage," ACM Trans. Graphics, vol. 28, no. 5, pp. 124:1-124:10, 2009.
[6] M.-M. Cheng, G.-X. Zhang, N.J. Mitra, X. Huang, and S.-M. Hu, "Global Contrast Based Salient Region Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 409-416, 2011.
[7] Y.-Y. Chuang, B. Curless, D. Salesin, and R. Szeliski, "A Bayesian Approach to Digital Matting," Proc. Conf. Computer Vision and Pattern Recognition (CVPR), pp. 264-271, 2001.
[8] T.A. Davis, "Algorithm 849: A Concise Sparse Cholesky Factorization Package," ACM Trans. Math. Software, vol. 31, no. 4, pp. 587-591, Dec. 2005.
[9] P. Didyk, T. Ritschel, E. Eisemann, K. Myszkowski, and H.-P. Seidel, "A Perceptual Model for Disparity," ACM Trans. Graphics, vol. 30, no. 4, p. 96, 2011.
[10] M. Ding and R. Tong, "Content-Aware Copying and Pasting in Images," The Visual Computer, vol. 26, nos. 6-8, pp. 721-729, 2010.
[11] S.-P. Du, S.-M. Hu, and R.R. Martin, "Manipulating Perspective in Stereoscopic Images," Technical Report TR-120516, Tsinghua Univ., 2012.
[12] Z. Farbman, G. Hoffer, Y. Lipman, D. Cohen-Or, and D. Lischinski, "Coordinates for Instant Image Cloning," ACM Trans. Graphics, vol. 28, no. 3, pp. 67:1-67:9, 2009.
[13] E.S.L. Gastal and M.M. Oliveira, "Shared Sampling for Real-Time Alpha Matting," Computer Graphics Forum, vol. 29, no. 2, pp. 575-584, 2010.
[14] K. He, C. Rhemann, C. Rother, X. Tang, and J. Sun, "A Global Sampling Method for Alpha Matting," Proc. Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2049-2056, 2011.
[15] J. Jia, J. Sun, C.-K. Tang, and H.-Y. Shum, "Drag-and-Drop Pasting," ACM Trans. Graphics, vol. 25, no. 3, pp. 631-637, 2006.
[16] Y. Kim, H. Jung, S. Choi, J. Lee, and J. Noh, "A Single Image Representation Model for Efficient Stereoscopic Image Creation," Computer Graphics Forum, vol. 30, no. 7, pp. 2067-2076, 2011.
[17] J.-F. Lalonde, D. Hoiem, A.A. Efros, C. Rother, J.M. Winn, and A. Criminisi, "Photo Clip Art," ACM Trans. Graphics, vol. 26, no. 3, p. 3, 2007.
[18] M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M.H. Gross, "Nonlinear Disparity Mapping for Stereoscopic three-dimensional," ACM Trans. Graphics, vol. 29, no. 4, pp. 75:1-75:10, 2010.
[19] A. Levin, D. Lischinski, and Y. Weiss, "A Closed-Form Solution to Natural Image Matting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 228-242, Feb. 2008.
[20] B. Lévy, S. Petitjean, N. Ray, and J. Maillot, "Least Squares Conformal Maps for Automatic Texture Atlas Generation," ACM Trans. Graphics, vol. 21, no. 3, pp. 362-371, 2002.
[21] W.-Y. Lo, J. van Baar, C. Knaus, M. Zwicker, and M.H. Gross, "Stereoscopic Three-Dimensional Copy & Paste," ACM Trans. Graphics, vol. 29, no. 6, pp. 147:1-147:10, 2010.
[22] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[23] T. Oskam, A. Hornung, H. Bowles, K. Mitchell, and M.H. Gross, "OSCAM-Optimized Stereoscopic Camera Control for Interactive Three-Dimensional," ACM Trans. Graphics, vol. 30, no. 6, pp. 189:1-189:8, 2011.
[24] P. Pérez, M. Gangnet, and A. Blake, "Poisson Image Editing," ACM Trans. Graphics, vol. 22, no. 3, pp. 313-318, 2003.
[25] B. Pollock, M. Burton, J.W. Kelly, S. Gilbert, and E. Winer, "The Right View from the Wrong Location: Depth Perception in Stereoscopic Multi-User Virtual Environments," IEEE Trans. Visualisation and Computer Graphics, vol. 18, no. 4, pp. 581-588, Apr. 2012.
[26] J.R. Shewchuk, "Delaunay Refinement Algorithms for Triangular Mesh Generation," Computational Geometry, vol. 22, nos. 1-3, pp. 21-74, 2002.
[27] B.M. Smith, L. Zhang, and H. Jin, "Stereo Matching with Nonparametric Smoothness Priors in Feature Space," Proc. IEEE Computer Vision and Pattern Recognition (CVPR '09), pp. 485-492, 2009.
[28] M. Wan, N. Zhang, H. Qu, and A.E. Kaufman, "Interactive Stereoscopic Rendering of Volumetric Environments," IEEE Trans. Visualization and Computer Graphics, vol. 10, no. 1, pp. 15-28, Jan./Feb. 2004.
[29] J. Wang, M. Agrawala, and M.F. Cohen, "Soft Scissors: An Interactive Tool for Realtime High Quality Matting," ACM Trans. Graphics, vol. 26, no. 3, p. 9, 2007.
[30] J. Wang and M.F. Cohen, "Image and Video Matting: A Survey," Foundations and Trends in Computer Graphics and Vision, vol. 3, no. 2, pp. 97-175, 2007.
[31] L. Wang, H. Jin, R. Yang, and M. Gong, "Stereoscopic Inpainting: Joint Color and Depth Completion from Stereo Images," Proc. IEEE Computer Vision and Pattern Recognition (CVPR '08), 2008.
[32] O. Wang, M. Lang, M. Frei, A. Hornung, A. Smolic, and M.H. Gross, "Stereobrush: Interactive Two-Dimensional to Three-Dimensional Conversion Using Discontinuous Warps," Proc. Int'l Symp. Sketch-Based Interfaces and Modeling (SBIM '11), pp. 47-54, 2011.
[33] Y.-S. Wang, C.-L. Tai, O. Sorkine, and T.-Y. Lee, "Optimized Scale-and-Stretch for Image Resizing," ACM Trans. Graphics, vol. 27, no. 5, p. 118, 2008.
[34] B. Ward, S.B. Kang, and E.P. Bennett, "Depth Director: A System for Adding Depth to Movies," IEEE Computer Graphics and Applications, vol. 31, no. 1, pp. 36-48, Jan./Feb. 2011.
[35] A. Woods, T. Docherty, and R. Koch, "Image Distortions in Stereoscopic Video Systems," Stereoscopic Displays And Applications, vol. 1915, pp. 36-48, 1993.
[36] C. Xiao, Y. Nie, and F. Tang, "Efficient Edit Propagation Using Hierarchical Data Structure," IEEE Trans. Visualization and Computer Graphics, vol. 17, no. 8, pp. 1135-1147, Aug. 2011.
[37] K. Xu, Y. Li, T. Ju, S.-M. Hu, and T.-Q. Liu, "Efficient Affinity-Based Edit Propagation Using k-d Tree," ACM Trans. Graphics, vol. 28, no. 5, pp. 118:1-118:6, 2009.
[38] G.-X. Zhang, M.-M. Cheng, S.-M. Hu, and R.R. Martin, "A Shape-Preserving Approach to Image Resizing," Computer Graphics Forum, vol. 28, no. 7, pp. 1897-1906, 2009.
[39] G. Zhang, W. Hua, X. Qin, T.-T. Wong, and H. Bao, "Stereoscopic Video Synthesis from a Monocular Video," IEEE Trans. Visualisation and Computer Graphics, vol. 13, no. 4, pp. 686-696, July/Aug. 2007.
[40] Y. Zhang and R. Tong, "Environment-Sensitive Cloning in Images," The Visual Computer, vol. 27, nos. 6-8, pp. 739-748, 2011.
32 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool