The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - Nov. (2012 vol.18)
pp: 1868-1879
K. Sunkavalli , Sch. of Eng. & Appl. Sci., Harvard Univ., Cambridge, MA, USA
N. Joshi , Microsoft Res., Redmond, WA, USA
Sing Bing Kang , Microsoft Res., Redmond, WA, USA
M. F. Cohen , Microsoft Res., Redmond, WA, USA
H. Pfister , Sch. of Eng. & Appl. Sci., Harvard Univ., Cambridge, MA, USA
ABSTRACT
We describe a unified framework for generating a single high-quality still image ("snapshot”) from a short video clip. Our system allows the user to specify the desired operations for creating the output image, such as super resolution, noise and blur reduction, and selection of best focus. It also provides a visual summary of activity in the video by incorporating saliency-based objectives in the snapshot formation process. We show examples on a number of different video clips to illustrate the utility and flexibility of our system.
INDEX TERMS
video signal processing, image enhancement, image fusion, image resolution, snapshot formation process, video snapshots, high-quality images, single high-quality still image generating, short video clip, output image, activity visual summary, saliency-based objectives incorporation, Spatial resolution, Cameras, Noise, Noise reduction, Image fusion, Image restoration, photomontage, Image fusion, image enhancement, super resolution, sharpening, deblurring, saliency
CITATION
K. Sunkavalli, N. Joshi, Sing Bing Kang, M. F. Cohen, H. Pfister, "Video Snapshots: Creating High-Quality Images from Video Clips", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 11, pp. 1868-1879, Nov. 2012, doi:10.1109/TVCG.2012.72
REFERENCES
[1] A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, "Interactive Digital Photomontage," ACM Trans. Graphics, vol. 23, no. 3, pp. 294-302, 2004.
[2] N. Joshi and M.F. Cohen, "Seeing Mt. Rainier: Lucky Imaging for Multi-Image Denoising Sharpening, and Haze Removal," Proc. IEEE Int'l Conf. Computational Photography (ICCP), 2010.
[3] R.Y. Tsai and T.S. Huang, "Multiframe Image Restoration and Registration," Advances in Computer Vision and Image Processing, vol. 1, pp. 317-339, 1984.
[4] S.C. Park, M.K. Park, and M.G. Kang, "Super-Resolution Image Reconstruction: A Technical Overview," IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21-36, May 2003.
[5] M. Irani and S. Peleg, "Improving Resolution by Image Registration," Graphical Models and Image Processing, vol. 53, pp. 231-239, May 1991.
[6] M.E. Tipping and C.M. Bishop, "Bayesian Image Super-Resolution," Proc. Neural Information Processing Systems, pp. 1279-1286, 2002.
[7] L. Pickup, D. Capel, S. Roberts, and A. Zisserman, "Bayesian Methods for Image Super-Resolution," The Computer J., vol. 52, pp. 101-113, 2007.
[8] S. Baker and T. Kanade, "Limits on Super-Resolution and How to Break Them," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 1167-1183, Sept. 2002.
[9] Z. Lin and H.-Y. Shum, "Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 83-97, Jan. 2004.
[10] H. Takeda, P. Milanfar, M. Protter, and M. Elad, "Super-Resolution without Explicit Subpixel Motion Estimation," IEEE Trans. Image Processing, vol. 18, no. 9, pp. 1958-1975, Sept. 2009.
[11] C. Liu and D. Sun, "A Bayesian Approach to Adaptive Video Super Resolution," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011.
[12] W.T. Freeman, T.R. Jones, and E.C. Pasztor, "Example-Based Super-Resolution," IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 56-65, Mar./Apr. 2002.
[13] J. Yang, J. Wright, T. Huang, and Y. Ma, "Image Super-Resolution via Sparse Representation," IEEE Trans. Image Processing, vol. 19, no. 11, pp. 2861-2873, Nov. 2010.
[14] M.F. Tappen, B.C. Russell, and W.T. Freeman, "Exploiting the Sparse Derivative Prior for Super-Resolution and Image Demosaicing," Proc. IEEE Workshop Statistical and Computational Theories of Vision, 2003.
[15] D. Glasner, S. Bagon, and M. Irani, "Super-Resolution from a Single Image," Proc. Int'l Conf. Computer Vision (ICCV), 2009.
[16] O. Shahar, A. Faktor, and M. Irani, "Super-Resolution from a Single Video," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2011.
[17] P. Chatterjee and P. Milanfar, "Is Denoising Dead?" IEEE Trans. Image Processing, vol. 19, no. 4, pp. 895-911, Apr. 2010.
[18] E. Simoncelli and E. Adelson, "Noise Removal via Bayesian Wavelet Coring," Proc. Int'l Conf. Image Processing (ICIP), vol. 1, pp. 379-382, Sept. 1996.
[19] J. Portilla, V. Strela, M. Wainwright, and E. Simoncelli, "Image Denoising Using Scale Mixtures of Gaussians in the Wavelet Domain," IEEE Trans. Image Processing, vol. 12, no. 11, pp. 1338-1351, Nov. 2003.
[20] P. Perona and J. Malik, "Scale-Space and Edge Detection Using Anisotropic Diffusion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 629-639, 1990.
[21] C. Tomasi and R. Manduchi, "Bilateral Filtering for Gray and Color Images," Proc. Int'l Conf. Computer Vision, pp. 839-846, 1998.
[22] S. Roth and M.J. Black, "Fields of Experts: A Framework for Learning Image Priors," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 860-867, 2005.
[23] M. Aharon, M. Elad, and A. Bruckstein, "The K-SVD: An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation," IEEE Trans. Signal Processing, vol. 54, no. 11, pp. 4311-4322, Nov. 2006.
[24] M. Elad and M. Aharon, "Image Denoising via Sparse and Redundant Representations over Learned Dictionaries," IEEE Trans. Image Processing, vol. 15, no. 12, pp. 3736-3745, Dec. 2006.
[25] A. Buades, B. Coll, and J.-M. Morel, "Nonlocal Image and Movie Denoising," Int'l J. Computer Vision, vol. 76, pp. 123-139, Feb. 2008.
[26] E.P. Bennett and L. McMillan, "Video Enhancement Using Per-Pixel Virtual Exposures," ACM Trans. Graphics, vol. 24, no. 3, pp. 845-852, 2005.
[27] C. Liu and W.T. Freeman, "A High-Quality Video Denoising Algorithm-Based on Reliable Motion Estimation," Proc. 11th European Conf. Computer Vision (ECCV), pp. 706-719, 2010.
[28] L. Teodosio and W. Bender, "Salient Video Stills: Content and Context Preserved," Proc. ACM Int'l Conf. Multimedia, pp. 39-46, 1993.
[29] M. Elad and A. Feuer, "Super-Resolution Reconstruction of Image Sequences," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 9, pp. 817-834, Sept. 1999.
[30] A. Levin, R. Fergus, F. Durand, and W.T. Freeman, "Image and Depth from a Conventional Camera with a Coded Aperture," ACM Trans. Graphics, vol. 26, no. 3, p. 70, 2007.
[31] P. Meer, "Robust Techniques for Computer Vision," Emerging Topics in Computer Vision, chapter 4, G. Medioni and S.B. Kang eds, Prentice Hall, July 2004.
[32] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[33] M.A. Fischler and R.C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. ACM, vol. 24, no. 6, pp. 381-395, 1981.
[34] L. Itti, C. Koch, and E. Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
[35] L. Itti and P. Baldi, "A Principled Approach to Detecting Surprising Events in Video," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 631-637, 2005.
[36] J.E. Cutting, "Representing Motion in a Static Image: Constraints and Parallels in Art, Science, and Popular Culture," Perception, vol. 31, no. 10, pp. 1165-1193, 2002.
[37] C. Liu, R. Szeliski, S.B. Kang, C.L. Zitnick, and W.T. Freeman, "Automatic Estimation and Removal of Noise from a Single Image," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 299-314, Feb. 2008.
[38] S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High Dynamic Range Video," ACM Trans. Graphics, vol. 22, no. 3, pp. 319-325, 2003.
[39] J. Fiss, A. Agarwala, and B. Curless, "Candid Portrait Selection from Video," ACM Trans. Graphics, vol. 30, pp. 128:1-128:8, Dec. 2011.
31 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool