The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2012 vol.18)
pp: 1156-1169
Lei Zhang , Beijing Institute of Technology, Beijing
Hua Huang , Xi'an Jiaotong University, Xi'an
Hongbo Fu , City University of Hong Kong, Hong Kong
ABSTRACT
We introduce the EXtract-and-COmplete Layering method (EXCOL)—a novel cartoon animation processing technique to convert a traditional animated cartoon video into multiple semantically meaningful layers. Our technique is inspired by vision-based layering techniques but focuses on shape cues in both the extraction and completion steps to reflect the unique characteristics of cartoon animation. For layer extraction, we define a novel similarity measure incorporating both shape and color of automatically segmented regions within individual frames and propagate a small set of user-specified layer labels among similar regions across frames. By clustering regions with the same labels, each frame is appropriately partitioned into different layers, with each layer containing semantically meaningful content. Then, a warping-based approach is used to fill missing parts caused by occlusion within the extracted layers to achieve a complete representation. EXCOL provides a flexible way to effectively reuse traditional cartoon animations with only a small amount of user interaction. It is demonstrated that our EXCOL method is effective and robust, and the layered representation benefits a variety of applications in cartoon animation processing.
INDEX TERMS
Cartoon animation, layer extraction, layer completion, label propagation.
CITATION
Lei Zhang, Hua Huang, Hongbo Fu, "EXCOL: An EXtract-and-COmplete Layering Approach to Cartoon Animation Reusing", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 7, pp. 1156-1169, July 2012, doi:10.1109/TVCG.2011.111
REFERENCES
[1] C. Bregler, L. Loeb, E. Chuang, and H. Deshpande, “Turning to the Masters: Motion Capturing Cartoons,” ACM Trans. Graphics, vol. 21, no. 3, pp. 399-407, 2002.
[2] C. de Juan and B. Bodenheimer, “Re-Using Traditional Animation: Methods for Semi-Automatic Segmentation and Inbetweening,” Proc. Symp. Computer Animation, pp. 223-232, 2006.
[3] D. Sýkora, J. Buriánek, and J. Zára, “Sketching Cartoons by Example,” Proc. Eurographics Workshop Sketch-Based Interfaces and Modeling, 2005.
[4] D. Sýkora, J. Buriánek, and J. Žára, “Colorization of Black-and-White Cartoons,” Image and Vision Computing, vol. 23, no. 9, pp. 767-852, 2005.
[5] S.H. Zhang, T. Chen, Y.F. Zhang, S.M. Hu, and R.R. Martin, “Vectorizing Cartoon Animations,” IEEE Trans. Visualization and Computer Graphics, vol. 15, no. 4, pp. 618-629, May 2009.
[6] M. Fradet, P. Pérez, and P. Robert, “Time-Sequential Extraction of Motion Layers,” Proc. Int'l Conf. Image Processing, pp. 3224-3227, 2008.
[7] J. Wang, Y.Q. Xu, H.Y. Shum, and M.F. Cohen, “Video Tooning,” ACM Trans. Graphics, vol. 23, no. 3, pp. 574-583, Aug. 2004.
[8] D. Sýkora, J. Dingliana, and S. Collins, “As-Rigid-as-Possible Image Registration for Hand-Drawn Cartoon Animations,” Proc. Non-Photorealistic Animation and Rendering, pp. 25-33, 2009.
[9] C. de Juan and B. Bodenheimer, “Cartoon Textures,” Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '04), 2004.
[10] J.J. Xiao and M. Shah, “Motion Layer Extraction in the Presence of Occlusion Using Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1644-1659, Oct. 2005.
[11] J. Wang and E. Adelson, “Representing Moving Images with Layers,” IEEE Trans. Image Processing, vol. 3, no. 5, pp. 625-638, Sept. 1994.
[12] Q.F. Ke and T. Kanade, “A Robust Subspace Approach to Layer Extraction,” Proc. Workshop Motion and Video Computing, pp. 37-43, 2002.
[13] A. Blake, A. Criminisi, G. Cross, and V. Kolmogorov, “Bilayer Segmentation of Live Video,” Proc. IEEE Conference Computer Vision and Pattern Recognition, pp. 53-60, 2006.
[14] T.H. Wang, J. Collomosse, D. Slatter, P. Cheatle, and D. Greig, “Video Stylization for Digital Ambient Displays of Home Movies,” Proc. Symp. Non-Photorealistic Animation and Rendering, pp. 3005-3008, 2010.
[15] T.H. Wang, J.Y. Guillemaut, and J. Collomosse, “Multi-Label Propagation for Coherent Video Segmentation and Artistic Stylization,” Proc. Int'l Conf. Image Processing, pp. 137-146, 2010.
[16] J. Wang, P. Bhat, R.A. Colburn, M. Agrawala, and M.F. Cohen, “Interactive Video Cutout,” ACM Trans. Graphics, vol. 24, no. 3, pp. 585-594, 2005.
[17] Y. Li, J. Sun, and H.-Y. Shum, “Video Object Cut and Paste,” ACM Trans. Graphics, vol. 24, no. 3, pp. 595-600, 2005.
[18] X. Bai, J. Wang, D. Simons, and G. Saprio, “Video Snapcut: Robust Video Object Cutout Using Localized Classifiers,” ACM Trans. Graphics, vol. 28, no. 3, 2009.
[19] J. McCann and N. Pollard, “Local Layering,” ACM Trans. Graphics, vol. 28, no. 3, pp. 1-7, Aug. 2009.
[20] Y. Wexler, E. Shechtman, and M. Irani, “Space-Time Video Completion,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 120-127, 2004.
[21] I. Drori, D. Cohen-Or, and H. Yeshurun, “Fragment-Based Image Completion,” ACM Trans. Graphics, vol. 22, no. 3, pp. 303-312, 2003.
[22] Y.T. Jia, S.M. Hu, and R.R. Martin, “Video Completion Using Tracking and Fragment Merging,” The Visual Computer, vol. 21, nos. 8-10, pp. 601-610, 2005.
[23] J.Y. Jia, T.P. Wu, Y.W. Tai, and C.K. Tang, “Video Repairing: Inference of Foreground and Background under Severe Occlusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 5, pp. 832-839, 2006.
[24] Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, “Lazy Snapping,” ACM Trans. Graphics, vol. 23, no. 3, pp. 303-308, 2004.
[25] C. Rother, A. Blake, and V. Kolmogorov, “Grabcut - Interactive Foreground Extraction Using Iterated Graph Cuts,” ACM Trans. Graphics, vol. 24, no. 3, pp. 309-314, 2004.
[26] D. Sýkora, J. Dingliana, and S. Collins, “Lazybrush: Flexible Painting Tool for Hand-Drawn Cartoons,” Computer Graphics Forum, vol. 28, no. 2, pp. 599-608, 2009.
[27] Y. Boykov and G. Funka-Lea, “Graph Cuts and Efficient n-d Image Segmentation,” Int'l J. Computer Vision, vol. 70, no. 2, pp. 109-131, 2006.
[28] S. Belongie, J. Malik, and J. Puzicha, “Shape Matching and Object Recognition Using Shape Contexts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002.
[29] V. Kolmogorov and R. Zabih, “What Energy Functions Can Be Minimized via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, Feb. 2004.
[30] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” Proc. Int'l Conf. Computer Vision, pp. 377-384, 1999.
[31] B. Zitová and J. Flusser, “Image Registration Methods: A Survey,” Image and Vision Computing, vol. 21, pp. 977-1000, 2003.
[32] D.G. Lowe, “Object Recognition from Local Scale-Invariant Features,” Proc. IEEE Int'l Conf. Computer Vision, pp. 1150-1157, 1999.
[33] M.A. Fischler and R.C. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Application to Image Analysis and Automated Catrography,” Comm. ACM, vol. 24, pp. 381-395, 1981.
[34] D. Sýkora, D. Sedláček, J.C. Sun, J. Dingliana, and S. Collins, “Adding Depth to Cartoons Using Sparse Depth (In)equalities,” Proc. Eurographics, 2010.
[35] P. Besl and N. McKay, “A Method for Registration of 3-D Shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
13 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool