This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
MPML3D: Scripting Agents for the 3D Internet
May 2011 (vol. 17 no. 5)
pp. 655-668
ASCII Text
x 
 
Helmut Prendinger, Sebastian Ullrich, Arturo Nakasone, Mitsuru Ishizuka, "MPML3D: Scripting Agents for the 3D Internet," IEEE Transactions on Visualization and Computer Graphics, vol. 17, no. 5, pp. 655-668, May, 2011.
 
 
BibTex
x
 
@article{ 10.1109/TVCG.2010.66,
author = {Helmut Prendinger and Sebastian Ullrich and Arturo Nakasone and Mitsuru Ishizuka},
title = {MPML3D: Scripting Agents for the 3D Internet},
journal ={IEEE Transactions on Visualization and Computer Graphics},
volume = {17},
number = {5},
issn = {1077-2626},
year = {2011},
pages = {655-668},
doi = {http://doi.ieeecomputersociety.org/10.1109/TVCG.2010.66},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Visualization and Computer Graphics
TI - MPML3D: Scripting Agents for the 3D Internet
IS - 5
SN - 1077-2626
SP655
EP668
EPD - 655-668
A1 - Helmut Prendinger,
A1 - Sebastian Ullrich,
A1 - Arturo Nakasone,
A1 - Mitsuru Ishizuka,
PY - 2011
KW - Artificial
KW - augmented
KW - and virtual realities
KW - graphical user interfaces
KW - synchronous interaction
KW - visualization
KW - markup languages
KW - scripting languages.
VL - 17
JA - IEEE Transactions on Visualization and Computer Graphics
ER -
 
Helmut Prendinger, National Institute of Informatics, Tokyo
Sebastian Ullrich, RWTH Aachen University, Aachen
Arturo Nakasone, National Institute of Informatics, Tokyo
Mitsuru Ishizuka, University of Tokyo, Tokyo
The aim of this paper is two-fold. First, it describes a scripting language for specifying communicative behavior and interaction of computer-controlled agents ("bots”) in the popular three-dimensional (3D) multiuser online world of "Second Life” and the emerging "OpenSimulator” project. While tools for designing avatars and in-world objects in Second Life exist, technology for nonprogrammer content creators of scenarios involving scripted agents is currently missing. Therefore, we have implemented new client software that controls bots based on the Multimodal Presentation Markup Language 3D (MPML3D), a highly expressive XML-based scripting language for controlling the verbal and nonverbal behavior of interacting animated agents. Second, the paper compares Second Life and OpenSimulator platforms and discusses the merits and limitations of each from the perspective of agent control. Here, we also conducted a small study that compares the network performance of both platforms.

[1] N. Stephenson, Snow Crash. Spectra, 1992.
[2] R. Bartle, Designing Virtual Worlds. New Riders, 2004.
[3] J.M. Smart, J. Cascio, and J. Paffendorf, “Metaverse Roadmap: Pathways to the 3D Web,” http:/www.metaverseroadmap.org, 2007.
[4] E. Castronova, Synthetic Worlds: The Business and Culture of Online Games. The Univ. of Chicago Press, 2005.
[5] W. Bainbridge, “The Scientific Research Potential of Virtual Worlds,” Science, vol. 317, pp. 472-476, 2007.
[6] T. Boellstorff, Coming of Age in Second Life: An Anthropologist Explores the Virtually Human. Princeton Univ. Press, 2008.
[7] P. Bourke, “Evaluating Second Life for the Collaborative Exploration of 3D Fractals,” Computers and Graphics, vol. 33, pp. 113-117, 2009.
[8] H. Prendinger, “The Global Lab: Towards a Virtual Mobility Platform for an Eco-Friendly Society,” Trans. Virtual Reality Soc. Japan, vol. 14, no. 2, pp. 163-170, 2009.
[9] A. Nakasone, H. Prendinger, S. Holland, P. Hut, J. Makino, and K. Miura, “Astrosim: Collaborative Visualization of an Astrophysics Simulation in Second Life,” IEEE Computer Graphics and Applications, vol. 29, no. 5, pp. 69-81, Sept./Oct. 2009.
[10] http:/secondlife.com/, 2010.
[11] http:/opensimulator.org, 2010.
[12] H. Prendinger, S. Descamps, and M. Ishizuka, “MPML: A Markup Language for Controlling the Behavior of Life-Like Characters,” J. Visual Languages and Computing, vol. 15, no. 2, pp. 183-203, 2004.
[13] M. Ishizuka and H. Prendinger, “Describing and Generating Multimodal Contents Featuring Affective Lifelike Agents with MPML,” New Generation Computing, vol. 24, pp. 97-128, 2006.
[14] M. Nischt, H. Prendinger, E. André, and M. Ishizuka, “MPML3D: A Reactive Framework for the Multimodal Presentation Markup Language,” Proc. Sixth Int'l Conf. Intelligent Virtual Agents (IVA '06), pp. 218-229, 2006.
[15] S. Ullrich, K. Brügmann, H. Prendinger, and M. Ishizuka, “Extending MPML3D to Second Life,” Proc. Eighth Int'l Conf. Intelligent Virtual Agents (IVA '08), pp. 281-288, 2008.
[16] S. Ullrich, H. Prendinger, and M. Ishizuka, “MPML3D: Agent Authoring Language for Virtual Worlds,” Proc. Advances in Computer Entertainment Technology (ACE '08), pp. 134-137, 2008.
[17] “MPML3D for Content Creators,” http://www.prendingerlab. netgloballab/, July 2009.
[18] “MPML3D for Developers,” http://forge.opensimulator.org/gf/project globallab/, July 2009.
[19] J. Cassell, H. Vilhjálmsson, and T. Bickmore, “BEAT: The Behavior Expression Animation Toolkit,” Proc. ACM SIGGRAPH '01, pp. 477-486, 2001.
[20] A. Marriott, “VHML—Virtual Human Markup Language,” Proc. Talking Head Technology Workshop, at OzCHI Conf., 2001.
[21] S. Kopp, B. Krenn, S. Marsella, A. Marshall, C. Pelachaud, H. Pirker, K. Thórisson, and H. Vilhjálmsson, “Towards a Common Framework for Multimodal Generation: The Behavior Markup Language,” Proc. Sixth Int'l Conf. Intelligent Virtual Agents (IVA '06), pp. 205-217, 2006.
[22] H. Prendinger and M. Ishizuka, Life-Like Characters: Tools, Affective Functions, and Applications. Springer-Verlag, 2004.
[23] M. Ishizuka, T. Tsutsui, S. Saeyor, H. Dohi, Y. Zong, and H. Prendinger, “MPML: A Multimodal Presentation Markup Language with Character Control Functions,” Proc. Agents '00 Workshop Achieving Human-Like Behavior in Interactive Animated Agents, pp. 50-54, 2000.
[24] Microsoft, Developing for Microsoft Agent. Microsoft Press, 1998.
[25] M. Nischt, H. Prendinger, E. André, and M. Ishizuka, “Creating Three-Dimensional Animated Characters: An Experience Report and Recommendations of Good Practice,” Upgrade. The European J. for the Informatics Professional, vol. VII, no. 2, pp. 36-41, 2006.
[26] K. Brügmann, H. Dohrn, H. Prendinger, M. Stamminger, and M. Ishizuka, “Phase-Based Gesture Motion Parametrization and Transitions for Conversational Agents with MPML3D,” Proc. Second Int'l Conf. Intelligent Technologies for Interactive Entertainment (INTETAIN '08), 2008.
[27] “Gathering of Animated Lifelike Agents,” http://hmi.ewi. utwente.nlgala/, 2010.
[28] H. Vilhjálmsson, N. Cantelmo, J. Cassell, N.E. Chafai, M. Kipp, S. Kopp, M. Mancini, S. Marsella, A.N. Marshall, C. Pelachaud, Z. Ruttkay, K. Thórisson, H. van Welbergen, and R. van der Werf, “The Behavior Markup Language: Recent Developments and Challenges,” Proc. Seventh Int'l Conf. Intelligent Virtual Agents (IVA '07), pp. 99-111, 2007.
[29] S. Kopp and I. Wachsmuth, “Synthesizing Multimodal Utterances for Conversational Agents,” Computer Animation and Virtual Worlds, vol. 15, no. 1, pp. 39-52, 2004.
[30] B.D. Carolis, C. Pelauchaud, I. Poggi, and M. Steedman, “APML: Mark-Up Language for Communicative Character Expressions,” Life-Like Characters: Tools, Affective Functions, and Applications, H. Prendinger and M. Ishizuka, eds., pp. 65-85, Springer-Verlag, 2004.
[31] P. Piwek, B. Krenn, M. Schröder, M. Grice, S. Baumann, and H. Pirker, “RRL: A Rich Representation Language for the Description of Agent Behavior in NECA,” Proc. First Int'l Joint Conf. Autonomous Agents and Multiagent Systems (AAMAS '02) Workshop Embodied Conversational Agents-Let's Specify and Evaluate Them!, 2002.
[32] K. Perlin, “Real Time Responsive Animation with Personality,” IEEE Trans. Visualization and Computer Graphics, vol. 1, no. 1, pp. 5-15, Mar. 1995.
[33] K. Perlin and A. Goldberg, “Improv: A System for Scripting Interactive Actors in Virtual Worlds,” Computer Graphics, vol. 30, pp. 205-216, 1996.
[34] H. Noser and D. Thalmann, “A Rule-Based Interactive Behavioral Animation System for Humanoids,” IEEE Trans. Visualization and Computer Graphics, vol. 5, no. 4, pp. 281-307, Oct.-Dec. 1999.
[35] N.I. Badler, R. Bindiganavale, J. Allbeck, W. Schuler, L. Zhao, and M. Palmer, “Parameterized Action Representation for Virtual Human Agents,” Embodied Conversational Agents, J. Cassell, J. Sullivan, S. Prevost, and E. Churchill, eds., pp. 256-284, MIT Press, 2000.
[36] F. Devillers and S. Donikian, “A Scenario Language to Orchestrate Virtual World Evolution,” Proc. ACM SIGGRAPH/Eurographis Symp. Computer Animation, pp. 265-275, 2003.
[37] J.F. Allen, “An Interval-Based Representation of Temporal Knowledge,” Proc. Int'l Joint Conf. Artificial Intelligence (IJCAI '81), pp. 221-226, 1981.
[38] P. Jorissen, M. Wijnants, and W. Lamotte, “Dynamic Interactions in Physically Realistic Collaborative Virtual Environments,” IEEE Trans. Visualization and Computer Graphics, vol. 11, no. 6, pp. 649-660, Nov.-Dec. 2005.
[39] “Walking in Stations,” http://myeve.eve-online.comdevblog. asp?a=blog&bid=401 , 2006.
[40] D. Friedman, A. Steed, and M. Slater, “Spatial Social Behavior in Second Life,” Proc. Seventh Int'l Conf. Intelligent Virtual Agents (IVA '07), pp. 252-264, 2007.
[41] R. Gayle and D. Manocha, “Navigating Virtual Agents in Online Virtual Worlds,” Proc. 13th Int'l Conf. 3D Web Technology (Web3D), pp. 53-56, 2008.
[42] M. Rehm and P. Rosina, “SecondLife As an Evaluation Platform for Multiagent Systems Featuring Social Interactions,” Proc. Seventh Int'l Joint Conf. Autonomous Agents and Multiagent Systems (AAMAS '08), pp. 1663-1664, 2008.
[43] D.J.H. Burden, “Deploying Embodied AI into Virtual Worlds,” Knowledge-Based Systems, vol. 22, no. 7, pp. 540-544, 2009.
[44] J.H.A. Gelissen, “Introduction to MPEG-V,” Virtual Worlds Research, vol. 2, no. 3, pp. 4-7, 2009.
[45] http:/www.openmetaverse.org, 2010.
[46] http://en.wikipedia.org/wikiBiovision_Hierarchy , 2010.
[47] http://secondlife.com/communityavatar.php , 2010.
[48] http:/www.h-anim.org/, 2010.
[49] “Open Source Platform for Interconnected Virtual Worlds,” http:/www.realxtend.org/, 2010.
[50] Avatar Puppeteering—Direct Manipulation of Avatars for Expression and Animation Editing, http:/avatarpuppeteering.com, 2010.
[51] “IBM and Linden Lab Interoperability Announcement,” http://blog.secondlife.com/2008/07/08, ibm-linden-lab- interoperability-announcement /, 2010.
[52] “An Introduction to MPML3D,” Prendinger Lab, http://www.prendingerlab.netgloballab/. Dec. 2008.
[53] “MPML3D Video in Global Lab Channel on YouTube,” http://www.youtube.comwatch?v=3ajgI-NVrBM , 2010.
[54] “MPML3D Examples in Various Videos in Prendinger's Channel on YouTube,” http://www.youtube.com/user/ prendingerlab#p u, 2010.
[55] W. Breitfuss, H. Prendinger, and M. Ishizuka, “Automatic Generation of Gaze and Gestures for Dialogues between Embodied Conversational Agents,” Int'l J. Semantic Computing, vol. 2, no. 1, pp. 71-90, 2008.
[56] A. Neviarouskaya, H. Prendinger, and M. Ishizuka, “Textual Affect Sensing for Sociable and Expressive Online Communication,” Proc. Second Int'l Conf. Affective Computing and Intelligent Interaction (ACII '07), pp. 218-229, 2007.
[57] F. Lamarche and S. Donikian, “Automatic Orchestration of Behaviours Through The Management of Resources and Priority Levels,” Proc. First Int'l Joint Conf. Autonomous Agents and Multiagent Systems (AAMAS '02), pp. 1309-1316, 2002.
[58] http:/openviewer.org/, 2010.
[59] S. Kumar, J. Chhugani, C. Kim, D. Kim, A. Nguyen, P. Dubey, C. Bienia, and Y. Kim, “Second Life and the New Generation of Virtual Worlds,” vol. 41, no. 9, Computer, pp. 46-53, Sept. 2008.
[60] http://slurl.com/secondlife/NIIsland/217/ 6925/, 2010.
[61] B. Brandherm, S. Ullrich, and H. Prendinger, “Simulation of Sensor-Based Tracking in Second Life (Demo paper),” Proc. Seventh Int'l Conf. Autonomous Agents and Multiagent Systems (AAMAS '08), pp. 1689-1690, 2008.
[62] W. Breitfuss, I. van der Sluis, H. Prendinger, S. Luz, and M. Ishizuka, “Evaluating an Algorithm for the Generation of Multimodal Referring Expressions in a Virtual World,” Proc. Ninth Int'l Conf. Intelligent Virtual Agents (IVA '09), 2009.
[63] H. Prendinger, P. Piwek, and M. Ishizuka, “Automatic Generation of Multi-Modal Dialogue from Text Based on Discourse Structure Analysis,” Int'l J. Semantic Computing, vol. 1, no. 3, pp. 319-334, 2007.

Index Terms:
Artificial, augmented, and virtual realities, graphical user interfaces, synchronous interaction, visualization, markup languages, scripting languages.
Citation:
Helmut Prendinger, Sebastian Ullrich, Arturo Nakasone, Mitsuru Ishizuka, "MPML3D: Scripting Agents for the 3D Internet," IEEE Transactions on Visualization and Computer Graphics, vol. 17, no. 5, pp. 655-668, May 2011, doi:10.1109/TVCG.2010.66
Usage of this product signifies your acceptance of the Terms of Use.