The Community for Technology Leaders
RSS Icon
Issue No.11 - Nov. (2013 vol.19)
pp: 1935-1947
M. Flagg , Coll. of Comput. Building, Georgia Inst. of Technol., Atlanta, GA, USA
J. M. Rehg , Coll. of Comput. Building, Georgia Inst. of Technol., Atlanta, GA, USA
As a controllable medium, video-realistic crowds are important for creating the illusion of a populated reality in special effects, games, and architectural visualization. While recent progress in simulation and motion captured-based techniques for crowd synthesis has focused on natural macroscale behavior, this paper addresses the complementary problem of synthesizing crowds with realistic microscale behavior and appearance. Example-based synthesis methods such as video textures are an appealing alternative to conventional model-based methods, but current techniques are unable to represent and satisfy constraints between video sprites and the scene. This paper describes how to synthesize crowds by segmenting pedestrians from input videos of natural crowds and optimally placing them into an output video while satisfying environmental constraints imposed by the scene. We introduce crowd tubes, a representation of video objects designed to compose a crowd of video billboards while avoiding collisions between static and dynamic obstacles. The approach consists of representing crowd tube samples and constraint violations with a conflict graph. The maximal independent set yields a dense constraint-satisyfing crowd composition. We present a prototype system for the capture, analysis, synthesis, and control of video-based crowds. Several results demonstrate the system's ability to generate videos of crowds which exhibit a variety of natural behaviors.
Electron tubes, Trajectory, Layout, Computational modeling, Collision avoidance, Shape, Rendering (computer graphics),crowd constraint optimization, Crowd synthesis, video-based rendering, video object layout
M. Flagg, J. M. Rehg, "Video-Based Crowd Synthesis", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 11, pp. 1935-1947, Nov. 2013, doi:10.1109/TVCG.2012.317
[1] A. Treuille, S. Cooper, and Z. Popović, "Continuum Crowds," Proc. ACM SIGGRAPH, pp. 1160-1168, 2006.
[2] J. Ondřej, J. Pettré, A.-H. Olivier, and S. Donikian, "A Synthetic-Vision Based Steering Approach for Crowd Simulation," ACM Trans. Graphics, vol. 29, pp. 123:1-123:9, July 2010.
[3] E. Ju, M.G. Choi, M. Park, J. Lee, K.H. Lee, and S. Takahashi, "Morphable Crowds," ACM Trans. Graphics, vol. 29, 2010.
[4] A. Schödl and I.A. Essa, "Controlled Animation of Video Sprites," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA), pp. 121-127, 2002.
[5] M. Flagg, A. Nakazawa, Q. Zhang, S.B. Kang, Y.K. Ryu, I. Essa, and J.M. Rehg, "Human Video Textures," Proc. Symp. Interactive Three-dimensional Graphics and Games (IThree-dimensional '09). pp. 199-206, 2009.
[6] M. Software, "Massive, Crowd Animation Software for Visual Effects," http:/, 2003.
[7] B. Hiebert, J. Dave, T.-Y. Kim, I. Neulander, H. Rijpkema, and W. Telford, "The Chronicles of Narnia: The Lion, the Crowds and Rhythm and Hues," Proc. ACM SIGGRAPH, p. 1, 2006.
[8] J. Funge, X. Tu, and D. Terzopoulos, "Cognitive Modeling: Knowledge, Reasoning and Planning for Intelligent Characters," Proc. ACM SIGGRAPH, pp. 29-38, 1999.
[9] W. Shao and D. Terzopoulos, "Autonomous Pedestrians," Graphical Models, vol. 69, nos. 5/6, pp. 246-274, 2007.
[10] O.B. Bayazit, J. Lien, and N.M. Amato, "Better Group Behaviors in Complex Environments Using Global Roadmaps," Proc. Eighth Int'l Conf. Artificial Life, pp. 362-370, 2002.
[11] D. Helbing, L. Buzna, A. Johansson, and T. Werner, "Self-Organized Pedestrian Crowd Dynamics: Experiments, Simulations, and Design Solutions," Transportation Science, vol. 39, no. 1, pp. 1-24, 2005.
[12] D. Helbing, P. Molnar, I.J. Farkas, and K. Bolay, "Self-Organizing Pedestrian Movement," Environment and Planning B: Planning and Design, vol. 28, pp. 361-383, 2001.
[13] C. Loscos, D. Marchal, and A. Meyer, "Intuitive Crowd Behaviour in Dense Urban Environments Using Local Laws," Proc. Theory and Practice of Computer Graphics Conf. (TPCG '03), pp. 122-129, 2003.
[14] R.A. Metoyer and J.K. Hodgins, "Reactive Pedestrian Path Following from Examples," Proc. 16th Int'l Conf. Computer Animation and Social Agents (CASA '03), pp. 149-156, 2003.
[15] M. Sung, "Scalable, Controllable, Efficient and Convincing Crowd Simulation," PhD dissertation, Madison, WI, USA, supervisor-Gleicher, Michael L, 2005.
[16] C.W. Reynolds, "Flocks, Herds and Schools: A Distributed Behavioral Model," Proc. ACM SIGGRAPH, pp. 25-34, 1987.
[17] M. Sung, L. Kovar, and M. Gleicher, "Fast and Accurate Goal-Directed Motion Synthesis for Crowds," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '05), pp. 291-300, 2005.
[18] L. Kovar, M. Gleicher, and F. Pighin, "Motion Graphs," ACM Trans. Graphics, vol. 21, no. 3, pp. 473-482, 2002.
[19] J. Lee, J. Chai, P.S.A. Reitsma, J.K. Hodgins, and N.S. Pollard, "Interactive Control of Avatars Animated with Human Motion Data," ACM Trans. Graphics, vol. 21, pp. 491-500, July 2002.
[20] B. Yersin, J. Maïm, J. Pettré, and D. Thalmann, "Crowd Patches: Populating Large-Scale Virtual Environments for Real-Time Applications," Proc. Symp. Interactive Three-dimensional Graphics and Games (IThree-dimensional '09). pp. 207-214, 2009.
[21] K.H. Lee, M.G. Choi, and J. Lee, "Motion Patches: Building Blocks for Virtual Environments Annotated with Motion Data," ACM Trans. Graphics, vol. 25, no. 3, pp. 898-906, 2006.
[22] A. Lerner, Y. Chrysanthou, and D. Lischinski, "Crowds by Example," Computer Graphics Forum, vol. 26, no. 3, pp. 655-664, 2007.
[23] K.H. Lee, M.G. Choi, Q. Hong, and J. Lee, "Group Behavior from Video: A Data-Driven Approach to Crowd Simulation," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '07), pp. 109-118, 2007.
[24] N. Courty and T. Corpetti, "Crowd Motion Capture," Computer Animation Virtual Worlds, vol. 18, pp. 361-370, Sept. 2007.
[25] F. Tecchia, C. Loscos, and Y. Chrysanthou, "Image-Based Crowd Rendering," IEEE Computer Graphics Application, vol. 22, no. 2, pp. 36-43, Mar. 2002.
[26] L. Kavan, S. Dobbyn, S. Collins, J. Zara, and C. O'Sullivan, "Polypostors: Second Polygonal Impostors for third Crowds," Proc. ACM SIGGRAPH, pp. 149-155, Feb. 2008.
[27] J.-F. Lalonde, D. Hoiem, A.A. Efros, C. Rother, J. Winn, and A. Criminisi, "Photo Clip Art," ACM Trans. Graphics, vol. 26, no. 3, Aug. 2007.
[28] R. McDonnell, M. Larkin, S. Dobbyn, S. Collins, and C. O'Sullivan, "Clone Attack! Perception of Crowd Variety," Proc. ACM SIGGRAPH, pp. 1-8, 2008.
[29] M. Luby, "A Simple Parallel Algorithm for the Maximal Independent Set Problem," Proc. 17th Ann. ACM Symp. Theory of Computing (STOC '85), pp. 1-10, 1985.
[30] X. Bai, J. Wang, D. Simmons, and G. Sapiro, "Video Snapcut: Robust Video Object Cutout Using Localized Classifiers," Proc. ACM SIGGRAPH, pp. 1-11, 2009.
[31] B.L. Price, B.S. Morse, and S. Cohen, "Livecut: Learning-Based Interactive Video Segmentation by Evaluation of Multiple Propagated Cues," Proc. IEEE 12th Int'l Conf. Computer Vision (ICCV), 2009.
[32] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[33] R. McDonnell, M. Larkin, B. Hernández, I. Rudomin, and C. O'Sullivan, "Eye-Catching Crowds: Saliency Based Selective Variation," Proc. ACM SIGGRAPH, pp. 1-10, 2009.
[34] B. Celly and V. Zordan, "Animated People Textures," Proc. 17th Int'l Conf. Computer Animation and Social Agents, 2004.
[35] D. Mahajan, F.-C. Huang, W. Matusik, R. Ramamoorthi, and P. Belhumeur, "Moving Gradients: A Path-Based Method for Plausible Image Interpolation," Proc. ACM SIGGRAPH, pp. 42:1-42:11, 2009.
[36] F. Xu, Y. Liu, C. Stoll, J. Tompkin, G. Bharaj, Q. Dai, H.-P. Seidel, J. Kautz, and C. Theobalt, "Video-Based Characters: Creating New Human Performances From a Multi-View Video Database," ACM Trans. Graphics, vol. 30, pp. 32:1-32:10, Aug. 2011.
97 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool