This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Dynamosaicing: Mosaicing of Dynamic Scenes
October 2007 (vol. 29 no. 10)
pp. 1789-1801
ASCII Text
x 
 
Alex Rav-Acha, Yael Pritch, Dani Lischinski, Shmuel Peleg, "Dynamosaicing: Mosaicing of Dynamic Scenes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1789-1801, October, 2007.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2007.1091,
author = {Alex Rav-Acha and Yael Pritch and Dani Lischinski and Shmuel Peleg},
title = {Dynamosaicing: Mosaicing of Dynamic Scenes},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {29},
number = {10},
issn = {0162-8828},
year = {2007},
pages = {1789-1801},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2007.1091},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Dynamosaicing: Mosaicing of Dynamic Scenes
IS - 10
SN - 0162-8828
SP1789
EP1801
EPD - 1789-1801
A1 - Alex Rav-Acha,
A1 - Yael Pritch,
A1 - Dani Lischinski,
A1 - Shmuel Peleg,
PY - 2007
KW - video mosaicing
KW - dynamic scene
KW - video editing
KW - graph cuts
KW - panoramic mosaicing
KW - time manipulations
KW - space-time volume
VL - 29
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
This paper explores the manipulation of time in video editing, enabling to control the chronological time of events. These time manipulations include slowing down (or postponing) some dynamic events while speeding up (or advancing) others. When a video camera scans a scene, aligning all the events to a single time interval will result in a panoramic movie. Time manipulations are obtained by first constructing an aligned space-time volume from the input video, and then sweeping a continuous 2D slice (time front) through that volume, generating a new sequence of images. For dynamic scenes, aligning the input video frames poses an important challenge. We propose to align dynamic scenes using a new notion of "dynamics constancy", which is more appropriate for this task than the traditional assumption of "brightness constancy".Another challenge is to avoid visual seams inside moving objects and other visual artifacts resulting from sweeping the space-time volumes with time fronts of arbitrary geometry. To avoid such artifacts, we formulate the problem of finding optimal time front geometry as one of finding a minimal cut in a 4D graph, and solve it using max-flow methods.

[1] A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, “Interactive Digital Photomontage,” Proc. ACM SIGGRAPH '04, pp. 294-302, Aug. 2004.
[2] A. Agarwala, C. Zheng, C. Pal, M. Agrawala, M. Cohen, B. Curless, D. Salesin, and R. Szeliski, “Panoramic Video Textures,” Proc. ACM SIGGRAPH '05, pp. 821-827, July 2005.
[3] Z. Bar-Joseph, R. El-Yaniv, D. Lischinski, and M. Werman, “Texture Mixing and Texture Movie Synthesis Using Statistical Learning,” IEEE Trans. Visualization and Computer Graphics, vol. 7, no. 2, pp. 120-135, Apr.-June 2001.
[4] J.L. Barron, D.J. Fleet, S.S. Beauchemin, and T.A. Burkitt, “Performance of Optical Flow Techniques,” Proc. Computer Vision and Pattern Recognition, pp. 236-242, 1992.
[5] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hierarchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, May 1992.
[6] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[7] L.G. Brown, “Survey of Image Registration Techniques,” ACM Computing Surveys, vol. 24, no. 4, pp. 325-376, Dec. 1992.
[8] G. Doretto, A. Chiuso, S. Soatto, and Y.N. Wu, “Dynamic Textures,” Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, Feb. 2003.
[9] G. Doretto and S. Soatto, “Towards Plenoptic Dynamic Textures,” Proc. Workshop Textures, pp. 25-30, Oct. 2003.
[10] A. Efros and T. Leung, “Texture Synthesis by Non-Parametric Sampling,” Proc. Int'l Conf. Computer Vision, vol. 2, pp. 1033-1038, Sept. 1999.
[11] A.W. Fitzgibbon, “Stochastic Rigidity: Image Registration for Nowhere-Static Scenes,” Proc. Int'l Conf. Computer Vision, pp. 662-669, July 2001.
[12] W.T. Freeman and H. Zhang, “Shape-Time Photography,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 151-157, 2003.
[13] S. Hsu, H.S. Sawhney, and R. Kumar, “Automated Mosaics via Topology Inference,” IEEE Trans. Computer Graphics and Applications, vol. 22, no. 2, pp. 44-54, 2002.
[14] M. Irani and P. Anandan, “Robust Multi-Sensor Image Alignment,” Proc. Int'l Conf. Computer Vision, pp. 959-966, Jan. 1998.
[15] M. Irani, P. Anandan, J. Bergen, R. Kumar, and S. Hsu, “Mosaic Representations of Video Sequences and Their Applications,” Signal Processing: Image Comm., vol. 8, no. 4, pp. 327-351, May 1996.
[16] A. Klein, P. Sloan, A. Colburn, A. Finkelstein, and M. Cohen, “Video Cubism,” Technical Report MSR-TR-2001-45, Microsoft Research, 2001.
[17] V. Kolmogorov and R. Zabih, “What Energy Functions can be Minimized via Graph Cuts?” Proc. European Conf. Computer Vision, pp. 65-81, May 2002.
[18] V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, “Graphcut Textures: Image and Video Synthesis using Graph Cuts,” ACM Trans. Graphics, SIGGRAPH '03, vol. 22, no. 3, pp. 277-286, July 2003.
[19] H. Lombaert, Y. Sun, L. Grady, and C. Xu, “A Multilevel Banded Graph Cuts Method for Fast Image Segmentation,” Proc. Int'l Conf. Computer Vision, pp. 259-265, Oct. 2005.
[20] S. Peleg, M. Ben-Ezra, and Y. Pritch, “Omnistereo: Panoramic Stereo Imaging,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 279-290, Mar. 2001.
[21] S. Peleg, B. Rousso, A. Rav-Acha, and A. Zomet, “Mosaicing on Adaptive Manifolds,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1144-1154, Oct. 2000.
[22] A. Rav-Acha, Y. Pritch, D. Lischinski, and S. Peleg, “Dynamosaicing: Video Mosaics with Non-Chronological Time,” Proc. Conf. Computer Vision and Pattern Recognition, June 2005.
[23] A. Rav-Acha, Y. Pritch, and S. Peleg, “Online Registration of Dynamic Scenes Using Video Extrapolation,” Proc. Workshop Dynamical Vision at ICCV '05, Oct. 2005.
[24] P.H.S. Torr and A. Zisserman, “MLESAC: A New Robust Estimator with Application to Estimating Image Geometry,” J.Computer Vision and Image Understanding, vol. 78, no. 1, pp.138-156, 2000.
[25] M. Uyttendaele, A. Eden, and R. Szeliski, “Eliminating Ghosting and Exposure Artifacts in Image Mosaics,” Proc. Conf. Computer Vision and Pattern Recognition, vol. II, pp. 509-516, Dec. 2001.
[26] R. Vidal and A. Ravichandran, “Optical Flow Estimation and Segmentation of Multiple Moving Dynamic Textures,” Proc. Computer Vision and Pattern Recognition, pp. 516-521, 2005.
[27] Y. Weiss and W.T. Freeman, “On the Optimality of Solutions of the Max-Product Belief Propagation Algorithm in Arbitrary Graphs,” IEEE Trans. Information Theory, vol. 47, no. 2, pp. 723-735, 2001.
[28] Y. Wexler, E. Shechtman, and M. Irani, “Space-Time Video Completion,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 120-127, June 2004.
[29] Y. Wexler and D. Simakov, “Space-Time Scene Manifolds,” Proc. Int'l Conf. Computer Vision, Oct. 2005.
[30] A. Zomet, D. Feldman, S. Peleg, and D. Weinshall, “Mosaicing New Views: The Crossed-Slits Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 6, pp. 741-754, June 2003.

Index Terms:
video mosaicing, dynamic scene, video editing, graph cuts, panoramic mosaicing, time manipulations, space-time volume
Citation:
Alex Rav-Acha, Yael Pritch, Dani Lischinski, Shmuel Peleg, "Dynamosaicing: Mosaicing of Dynamic Scenes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1789-1801, Oct. 2007, doi:10.1109/TPAMI.2007.1091
Usage of this product signifies your acceptance of the Terms of Use.