The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - Aug. (2013 vol.19)
pp: 1354-1361
Yu-Shuen Wang , Dept. of Comput. Sci., Nat. Chiao-Tung Univ., Hsinchu, Taiwan
Feng Liu , Dept. of Comput. Sci., Portland State Univ., Portland, OR, USA
Pu-Sheng Hsu , Dept. of Comput. Sci. & Inf. Eng., Nat. Cheng-Kung Univ., Tainan, Taiwan
Tong-Yee Lee , Dept. of Comput. Sci. & Inf. Eng., Nat. Cheng-Kung Univ., Tainan, Taiwan
ABSTRACT
Properly handling parallax is important for video stabilization. Existing methods that achieve the aim require either 3D reconstruction or long feature trajectories to enforce the subspace or epipolar geometry constraints. In this paper, we present a robust and efficient technique that works on general videos. It achieves high-quality camera motion on videos where 3D reconstruction is difficult or long feature trajectories are not available. We represent each trajectory as a Bézier curve and maintain the spatial relations between trajectories by preserving the original offsets of neighboring curves. Our technique formulates stabilization as a spatial-temporal optimization problem that finds smooth feature trajectories and avoids visual distortion. The Bézier representation enables strong smoothness of each feature trajectory and reduces the number of variables in the optimization problem. We also stabilize videos in a streaming fashion to achieve scalability. The experiments show that our technique achieves high-quality camera motion on a variety of challenging videos that are difficult for existing methods.
INDEX TERMS
Trajectory, Streaming media, Cameras, Optimization, Smoothing methods, Robustness, Image reconstruction, Bézier curve, Video stabilization, warping, optimization
CITATION
Yu-Shuen Wang, Feng Liu, Pu-Sheng Hsu, Tong-Yee Lee, "Spatially and Temporally Optimized Video Stabilization", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 8, pp. 1354-1361, Aug. 2013, doi:10.1109/TVCG.2013.11
REFERENCES
[1] F. Liu, M. Gleicher, H. Jin, and A. Agarwala, "Content-Preserving Warps for 3D Video Stabilization," ACM Trans. Graphics, vol. 28, pp. 44:1-44:9, 2009.
[2] S. Liu, Y. Wang, L. Yuan, J. Bu, P. Tan, and J. Sun, "Video Stabilization with a Depth Camera," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2012.
[3] F. Liu, M. Gleicher, J. Wang, H. Jin, and A. Agarwala, "Subspace Video Stabilization," ACM Trans. Graphics, vol. 30, no. 1, pp. 1-10, 2011.
[4] A. Goldstein and R. Fattal, "Video Stabilization Using Epipolar Geometry," ACM Trans. Graphics, vol. 31, pp. 1-10, 2012.
[5] W.-Q. Yan and M.S. Kankanhalli, "Detection and Removal of Lighting & Shaking Artifacts in Home Videos," Proc. 10th ACM Int'l Conf. Multimedia, pp. 107-116, 2002.
[6] S. Wu and Z. Ren, "Video Stabilization by Multi-Trajectory Mapping and Smoothing," Proc. Fifth Int'l Conf. Information, Comm. and Signal Processing, pp. 542-545, 2005.
[7] M. Long, L. Xinggang, X. Lidong, and F. Fangwen, "Video Stabilizing System for Digital Camera," Proc. Int'l Conf. Signal Processing, pp. 1119-1122, 2004.
[8] M. Grundmann, V. Kwatra, and I. Essa, "Auto-Directed Video Stabilization with Robust l1 Optimal Camera Paths," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011.
[9] C. Morimoto and R. Chellappa, "Evaluation of Image Stabilization Algorithms," Proc. IEEE Int'l Conf. Acoustics, Speech and Signal Processing, pp. 2789-2792, 1998.
[10] Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H.-Y. Shum, "Full-Frame Video Stabilization with Motion Inpainting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1150-1163, July 2006.
[11] J. Yang, D. Schonfeld, C. Chen, and M.A. Mohamed, "Online Video Stabilization Based on Particle Filters," Proc. Int'l Conf. Image Processing (ICIP), pp. 1545-1548, 2006.
[12] B.-Y. Chen, K.-Y. Lee, J.-S. Lin, and W.-T. Huang, "Capturing Intention-Based Full-Frame Video Stabilization," Computer Graphics Forum, vol. 27, pp. 1805 -1814, 2008.
[13] M. Gleicher and F. Liu, "Re-Cinematography: Improving the Camerawork of Casual Video," ACM Trans. Multimedia Computing Comm. and Applications, vol. 5, no. 1, pp. 1-28, 2008.
[14] K.-Y. Lee, Y.-Y. Chuang, B.-Y. Chen, and M. Ouhyoung, "Video Stabilization Using Robust Feature Trajectories," Proc. IEEE 12th Int'l Conf. Computer Vision (ICCV), pp. 1397-1404, 2009.
[15] A. Fitzgibbon, Y. Wexler, and A. Zisserman, "Image-Based Rendering Using Image-Based Priors," Int'l J. Computer Vision, vol. 63, no. 2, pp. 141-151, 2005.
[16] P. Bhat, C.L. Zitnick, N. Snavely, A. Agarwala, M. Agrawala, M. Cohen, B. Curless, and S.B. Kang, "Using Photographs to Enhance Videos of a Static Scene," Proc. Eurographics Workshop Rendering, pp. 327-338, 2007.
[17] G. Zhang, W. Hua, X. Qin, Y. Shao, and H. Bao, "Video Stabilization Based on a 3D Perspective Camera Model," The Visual Computer, vol. 25, no. 11, pp. 997-1008, 2009.
[18] Z. Huang, F. He, X. Cai, Y. Chen, and X. Chen, "A 2D-3D Hybrid Approach to Video Stabilization," Proc. IEEE Int'l Conf. CS-Aided Design and CS Graphics, pp. 146-150, 2011.
[19] S. Baker, E.P. Bennett, S.B. Kang, and R. Szeliski, "Removing Rolling Shutter Wobble," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2392-2399, 2010.
[20] E. Ringaby and P.-E. Forssén, "Efficient Video Rectification and Stabilisation for Cell-Phones," Int'l J. Computer Vision, vol. 96, pp. 335-352, 2012.
[21] M. Grundmann, V. Kwatra, D. Castro, and I. Essa, "Calibration-Free Rolling Shutter Removal," Proc. Int'l Conf. Computational Photography, 2012.
[22] J. Shi and C. Tomasi, "Good Features to Track," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR), pp. 593-600, 1994.
[23] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[24] O. Sorkine and M. Alexa, "As-Rigid-as-Possible Surface Modeling," Proc. EUROGRAPHICS/ACM SIGGRAPH Symp. Geometry Processing, pp. 109-116, 2007,
[25] "Voodoo Camera Tracker: A Tool for the Integration of Virtual and Real Scenes," http://www.digilab.uni-hannover.de/docsmanual.html , 2013.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool