Issue No.08 - Aug. (2013 vol.19)
pp: 1288-1297
Song-Pei Du , Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China
Shi-Min Hu , Dept. of Comput. Sci., Tsinghua Univ., Beijing, China
R. R. Martin , Sch. of Comput. Sci. & Inf., Cardiff Univ., Cardiff, UK
Traditional image editing techniques cannot be directly used to edit stereoscopic ("3D”) media, as extra constraints are needed to ensure consistent changes are made to both left and right images. Here, we consider manipulating perspective in stereoscopic pairs. A straightforward approach based on depth recovery is unsatisfactory: Instead, we use feature correspondences between stereoscopic image pairs. Given a new, user-specified perspective, we determine correspondence constraints under this perspective and optimize a 2D warp for each image that preserves straight lines and guarantees proper stereopsis relative to the new camera. Experiments verify that our method generates new stereoscopic views that correspond well to expected projections, for a wide range of specified perspective. Various advanced camera effects, such as dolly zoom and wide angle effects, can also be readily generated for stereoscopic image pairs using our method.
Stereo image processing, Cameras, Image segmentation, Humans, Image edge detection, Robustness, Visualization, warping, Stereoscopic images, stereopsis, perspective
Song-Pei Du, Shi-Min Hu, R. R. Martin, "Changing Perspective in Stereoscopic Images", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 8, pp. 1288-1297, Aug. 2013, doi:10.1109/TVCG.2013.14
[1] T. Oskam, A. Hornung, H. Bowles, K. Mitchell, and M. Gross, "OSCAM—Optimized Stereoscopic Camera Control for Interactive 3D," ACM Trans. Graphics, vol. 30, pp. 189:1-189:8, , 2011.
[2] S. Heinzle, P. Greisen, D. Gallup, C. Chen, D. Saner, A. Smolic, A. Burg, W. Matusik, and M. Gross, "Computational Stereo Camera System with Programmable Control Loop," ACM Trans. Graphics, vol. 30, no. 4, pp. 94:1-94:10, , Aug. 2011.
[3] M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, "Nonlinear Disparity Mapping for Stereoscopic 3D," ACM Trans. Graphics, vol. 29, pp. 75:1-75:10, , 2010.
[4] W.-Y. Lo, J. van Baar, C. Knaus, M. Zwicker, and M. Gross, "Stereoscopic 3D Copy & Paste," ACM Trans. Graphics, vol. 29, pp. 147:1-147:10, , 2010.
[5] A. Woods, T. Docherty, and R. Koch, "Image Distortions in Stereoscopic Video Systems," Proc. SPIE, vol. 1915, pp. 36-48, 1993.
[6] L. Wang, H. Jin, R. Yang, and M. Gong, "Stereoscopic Inpainting: Joint Color and Depth Completion from Stereo Images," Proc. IEEE Conf. Computer Vision Pattern Recognition, pp. 1-8, 2008.
[7] S. Koppal, C. Zitnick, M. Cohen, S.B. Kang, B. Ressler, and A. Colburn, "A Viewer-Centric Editor for 3D Movies," IEEE Computer Graphics and Applications, vol. 31, no. 1, pp. 20-35, Jan./Feb. 2011.
[8] R. Szeliski, "Image Alignment and Stitching: A Tutorial," Foundation Trends Computer Graphic Vision, vol. 2, pp. 1-104, http://dl.acm.orgcitation.cfm?id=1295184.1295185 , 2006.
[9] R. Carroll, A. Agarwala, and M. Agrawala, "Image Warps for Artistic Perspective Manipulation," ACM Trans. Graphics, vol. 29, pp. 127:1-127:9, , 2010.
[10] A. Agarwala, M. Agrawala, M. Cohen, D. Salesin, and R. Szeliski, "Photographing Long Scenes with Multi-Viewpoint Panoramas," ACM Trans. Graphics, vol. 25, pp. 853-861, , 2006.
[11] O. Wang, M. Lang, M. Frei, A. Hornung, A. Smolic, and M. Gross, "Stereobrush: Interactive 2D to 3D Conversion Using Discontinuous Warps," Proc. Eighth Eurographics Symp. Sketch-Based Interfaces and Modeling (SBIM '11), pp. 47-54, 2011.
[12] L.B. Stelmach, W.J. Tam, F. Speranza, R. Renaud, and T. Martin, "Improving the Visual Comfort of Stereoscopic Images," Proc. SPIE, vol. 5006, 2003.
[13] L. Meesters, W. IJsselsteijn, and P. Seuntiens, "A Survey of Perceptual Evaluations and Requirements of Three-Dimensional TV," IEEE Trans. Circuits and Systems for Video Technology, vol. 14, no. 3, pp. 381-391, Mar. 2004.
[14] B. Mendiburu, 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen. Focal Press, 2009.
[15] H.J. Kim, J.W. Choi, A.-J. Chang, and K.Y. Yu, "Reconstruction of Stereoscopic Imagery for Visual Comfort," Proc. SPIE, vol. 6803, 2008.
[16] H. Pan, C. Yuan, and S. Daly, "3D Video Disparity Scaling for Preference and Prevention of Discomfort," Proc. SPIE, vol. 7863, 2011.
[17] C. Wang and A.A. Sawchuk, "Disparity Manipulation for Stereo Images and Video," Proc. SPIE, vol. 6803, 2008.
[18] C.-H. Chang, C.-K. Liang, and Y.-Y. Chuang, "Content-Aware Display Adaptation and Interactive Editing for Stereoscopic Images," IEEE Trans. Multimedia, vol. 13, no. 4, pp. 589-601, Aug. 2011.
[19] A. Smolic, P. Kauff, S. Knorr, A. Hornung, M. Kunter, M. Müandller, and M. Lang, "Three-Dimensional Video Postproduction and Processing," Proc. IEEE, vol. 99, no. 4, pp. 607-625, Apr. 2011.
[20] Y.-S. Wang, C.-L. Tai, O. Sorkine, and T.-Y. Lee, "Optimized Scale-and-Stretch for Image Resizing," ACM Trans. Graphics, vol. 27, pp. 118:1-118:8, , 2008.
[21] Y.-S. Wang, H. Fu, O. Sorkine, T.-Y. Lee, and H.-P. Seidel, "Motion-Aware Temporal Coherence for Video Resizing," ACM Trans. Graphics, vol. 28, pp. 127:1-127:10, , 2009.
[22] P. Krähenbühl, M. Lang, A. Hornung, and M. Gross, "A System for Retargeting of Streaming Video," ACM Trans. Graphics, vol. 28, pp. 126:1-126:10, , 2009.
[23] F. Liu, M. Gleicher, H. Jin, and A. Agarwala, "Content-Preserving Warps for 3D Video Stabilization," ACM Trans. Graphics, vol. 28, pp. 44:1-44:9, , 2009.
[24] R. Carroll, M. Agrawal, and A. Agarwala, "Optimizing Content-Preserving Projections for Wide-Angle Images," ACM Trans. Graphics, vol. 28, pp. 43:1-43:9, , 2009.
[25] J. Wei, C.-F. Li, S.-M. Hu, R.R. Martin, and C.-L. Tai, "Fisheye Video Correction," IEEE Trans. Visualization and Computer Graphics, vol. 18, no. 10, pp. 1771-1783, Oct. 2012.
[26] R.F. Tong, Y. Zhang, and K.-L. Cheng, "StereoPasting: Interactive Composition in Stereoscopic Images," IEEE Trans. Visualization and Computer Graphics, accepted, 2012.
[27] T. Cootes, G. Edwards, and C. Taylor, "Active Appearance Models," IEEE Trans. Pattern Analysis Machine Intelligence, vol. 23, no. 6, pp. 681-685, June 2001.
[28] D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol. 47, pp. 7-42, http://dl.acm.orgcitation. cfm?id=598429.598475 , 2002.
[29], 2013.
[30] T. Sattler, B. Leibe, and L. Kobbelt, "SCRAMSAC: Improving Ransac's Efficiency with a Spatial Consistency Filter," Proc. IEEE Int'l Conf. Computer Vision, pp. 2090-2097, 2009.
[31] M. Brown, R. Szeliski, and S. Winder, "Multi-Image Matching Using Multi-Scale Oriented Patches," Proc. IEEE CS Conf. Computer Vision Pattern Recognition, vol. 1, pp. 510-517, 2005.
[32] Z.-N. Li, "Stereo Correspondence Based on line Matching in Hough Space Using Dynamic Programming," IEEE Trans. Systems, Man and Cybernetics, vol. 24, no. 1, pp. 144 -152, Jan. 1994.
[33] P. Bhat, C.L. Zitnick, M. Cohen, and B. Curless, "Gradientshop: A Gradient-Domain Optimization Framework for Image and Video Filtering," ACM Trans. Graphics, vol. 29, pp. 10:1-10:14, , 2010.
[34] N.J. Mitra, L.J. Guibas, and M. Pauly, "Partial and Approximate Symmetry Detection for 3D Geometry," ACM Trans. Graphics, vol. 25, pp. 560-568, 1141924 , 2006.
[35] B.M. Smith, L. Zhang, and H. Jin, "Stereo Matching with Nonparametric Smoothness Priors in Feature Space," Proc. IEEE CS Conf. Computer Vision Pattern Recognition, pp. 485-492, 2009.
[36] C.T. Loop and Z. Zhang, "Computing Rectifying Homographies for Stereo Vision," Computer Vision and Pattern Recognition, vol. 1, pp. 1125-1131, 1999.
[37] M. Lambooij, W. IJsselsteijn, and M. Fortuin, "Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review," J. Imaging Technology and Science, vol. 53, pp. 1-14, 2009.