This Article 
 Bibliographic References 
 Add to: 
A Network Architecture Supporting Consistent Rich Behavior in Collaborative Interactive Applications
May/June 2006 (vol. 12 no. 3)
pp. 405-416

Abstract—Network architectures for collaborative virtual reality have traditionally been dominated by client-server and peer-to-peer approaches, with peer-to-peer strategies typically being favored where minimizing latency is a priority, and client-server where consistency is key. With increasingly sophisticated behavior models and the demand for better support for haptics, we argue that neither approach provides sufficient support for these scenarios and, thus, a hybrid architecture is required. We discuss the relative performance of different distribution strategies in the face of real network conditions and illustrate the problems they face. Finally, we present an architecture that successfully meets many of these challenges and demonstrate its use in a distributed virtual prototyping application which supports simultaneous collaboration for assembly, maintenance, and training applications utilizing haptics.

[1] F.P. Brooks Jr., “What's Real about Virtual Reality,” IEEE Computer Graphics and Applications, pp. 16-27, 1999.
[2] S. Jayaram, J. Vance, R. Gadh, U. Jayaram, and H. Srinivasan, “Assessment of VR Technology and Its Applications to Engineering Problems,” Computing and Information Science in Eng., vol. 1, no. 1, pp. 72-83, Mar. 2001.
[3] J. Krüger and R. Westermann, “Linear Algebra Operators for GPU Implementation of Numerical Algorithms,” ACM Trans. Graphics (TOG), vol. 22, no. 3, pp. 908-916, July 2003.
[4] M.J. Harris, W.V. Baxter, T. Scheuermann, and A. Lastra, “Simulation of Cloud Dynamics on Graphics Hardware,” Proc. SIGGRAPH/EUROGRAPHICS Workshop Graphics Hardware, pp. 92-101, 2003.
[5] Y. Liu, X. Liu, and E. Wu, “Real-Time 3D Fluid Simulation on the GPU with Complex Obstacles,” Pacific Graphics, pp. 247-256, Oct. 2004.
[6] C. Basdogan, C.-H. Ho, M.A. Shrinivasan, and M. Slater, “An Experimental Study on the Role of Touch in Shared Virtual Environments,” ACM Trans. Computer Human Interaction, vol. 7, no. 4, pp. 443-460, Dec. 2000.
[7] E.L. Sallnaes, K. Rassmus-Groehn, and C. Sjoestroem, “Supporting Presence in Collaborative Environments by Haptic Force Feedback,” ACM Trans. Computer-Human Interaction, vol. 7, no. 4, pp. 461-476, 2000.
[8] R.J. Adams, D. Klowden, and B. Hannaford, “Virtual Training for a Manual Assembly Task,” Haptics-e, vol. 2, no. 2, Oct. 2001.
[9] I. Oakley, S. Brewster, and P. Gray, “Can You Feel the Force? An Investigation of Haptic Collaboration in Shared Editors,” EuroHaptics, 2001.
[10] M.O. Ernst and M.S. Banks, “Humans Integrate Visual and Haptic Information in a Statistically Optimal Fashion,” Nature, vol. 415, no. 1, pp. 429-433, Jan. 2002.
[11] R. Komerska and C. Ware, “Haptic Task Constraints for 3D Interaction,” Proc. IEEE Symp. Virtual Reality, 2003.
[12] M.A. Otaduy and M.C. Lin, “Sensation Preserving Simplification for Haptic Rendering,” ACM SIGGRAPH Trans. Graphics, vol. 22, pp. 543-553, 2003.
[13] M.A. Otaduy, N. Jain, A. Sud, and M.C. Lin, “Haptic Rendering of Interaction between Textured Models,” Proc. SIGGRAPH: Sketches and Applications, 2004.
[14] J.D. Hwang, M.D. Williams, and G. Niemeyer, “Toward Event-Based Haptics: Rendering Contact Using Open-Loop Force Pulses,” Proc. 12th Int'l Symp. Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS), pp. 24-31, Mar. 2004.
[15] J.M. Hollerbach, E. Cohen, and W. Thompson, “Haptic Interfacing for Virtual Prototying of Mechanical CAD Designs,” Proc. DETC ASME Design Eng. Technical Conf., pp. 14-17, Sept. 1997.
[16] G. Zachmann and A. Rettig, “Natural and Robust Interaction in Virtual Assembly Simulation,” Proc. Eighth ISPE Int'l Conf. Concurrent Eng.: Research and Applications, July 2001.
[17] G.C. Burdea, “Haptic Feedback for Virtual Reality,” Proc. Virtual Reality and Prototyping Workshop, June 2003.
[18] P. Buttolo, R. Oboe, and B. Hannaford, “Architectures for Shared Haptic Virtual Environments,” Computers and Graphics, special issue, 1997.
[19] J.P. Hespanha, M. McLaughlin, G.S. Sukhatme, M. Akbarian, R. Garg, and W. Zhu, “Haptic Collaboration over the Internet,” Proc. Fifth PHANTOM Users Group Workshop, 2000.
[20] K. Montgomery, C. Bruyns, J. Brown, S. Sorkin, F. Mazzella, G. Thonier, A. Tellier, B. Lerman, and A. Menon, “Spring: A General Framework for Collaborative, Real-Time Surgical Simulation,” Medicine Meets Virtual Reality, Amsterdam: IOS Press, 2002.
[21] F. Bogsanyi and M.L. Miller, “Tool and Object Based Synchronization in Collaborative Haptics,” Proc. IEEE Int'l Workshop Haptic Audio and Visual Environments (HAVE), pp. 109-113, 2002.
[22] X. Shen, F. Bogsanyi, L. Ni, and N.D. Georganas, “A Heterogeneous Scalabale Architecture for Collaborative Haptics Environments,” Proc. IEEE Int'l Workshop Haptic Audio and Visual Environments (HAVE), pp. 113-118, Sept. 2003.
[23] M. Oliveira, J. Mortensen, J. Jordan, A. Steed, and M. Slater, “Considerations in the Design of Virtual Environment Systems: A Case Study,” Proc. Second Int'l Conf. Application and Development of Computer Games, Jan. 2003.
[24] K.S. Park and R.V. Kenyon, “Effects of Network Characteristics on Human Performance in a Collaborative Virtual Environment,” IEEE Virtual Reality, pp. 104-111, 1999.
[25] C. Gutwin, “The Effects of Network Delays on Group Work in Real-Time Groupware,” Proc. European Conf. Computer Supported Cooperative Work (CSCW), pp. 299-318, 2001.
[26] G.C. Burdea, Force and Touch Feedback for Virtual Reality. John Wiley and Sons, Inc., 1996.
[27] R.Q. van der Linde, P. Lammertse, E. Frederiksen, and B. Ruiter, “The HapticMaster, a New High-Performance Haptic Interface,” Proc. Eurohaptics Conf., pp. 1-5, 2002.
[28] S. Singhal and M. Zyda, Networked Virtual Environments Design and Implementation. ACM Press SIGGRAPH Series, Addison Wesley, 1999.
[29] M.R. Macedonia, M.J. Zyda, D.R. Pratt, P.T. Barham, and S. Zeswitz, “NPSNET: A Network Software Architecture for Large-Scale VES,” Presence Teleoperators and Virtual Environments, vol. 3, no. 4, pp. 265-287, 1994.
[30] S. Benford, J. Bowers, L.E. Fahlén, and C. Greenhalgh, “Managing Mutual Awareness in Collaborative Virtual Environments,” Virtual Reality Software and Technology, pp. 223-236, 1994.
[31] C. Greenhalgh and S. Benford, “MASSIVE: A Collaborative Virtual Environment for Teleconferencing,” ACM Trans. Computer-Human Interaction (TOCHI), vol. 2, no. 3, pp. 239-261, Sept. 1995.
[32] O. Hagsand, “Interactive Multiuser VEs in the DIVE System,” IEEE Multimedia, vol. 3, no. 1, pp. 30-39, 1996.
[33] L. Lamport, “Time, Clocks, and the Ordering of Events in a Distributed System,” Comm. ACM, vol. 21, no. 7, pp. 558-565, July 1978.
[34] D.R. Jefferson, “Virtual Time,” ACM Trans. Programming Languages and Systems, vol. 7, no. 3, pp. 404-425, 1985.
[35] M. Mauve, J. Vogel, V. Hilt, and W. Effelsberg, “Local-Lag and Timewarp: Providing Consistency for Replicated Continuous Applications,” IEEE Trans. Multimedia, vol. 6, no. 1, pp. 47-57, Feb. 2004.
[36] B. Mirtich, “Timewarp Rigid Body Simulation,” Proc. SIGGRAPH Computer Graphics, pp. 193-200, 2000.
[37] M. Mauve, “Consistency in Replicated Continuous Interactive Media,” Proc. ACM Conf. Computer Supported Cooperative Work (CSCW), pp. 181-190, Dec. 2000.
[38] R. Cozot, “From Multibody Systems Modeling to Distributed Real-Time Simulation,” Proc. 29th Ann. Simulation Symp., pp. 234-241, 1996.
[39] J.G. Cleary, M. Pearson, and H. Kinawi, “The Architecture of an Optimistic CPU: The WarpEngine,” Proc. 28th Hawaii Int'l Conf. System Sciences, vol. 1, pp. 163-172, Jan. 1995.
[40] S.R. Pettifer, “An Operating Environment for Large Scale Virtual Reality,” PhD dissertation, The University of Manchester, 1999.
[41] S. Pettifer, J. Cook, J. Marsh, and A. West, “DEVA3: Architecture for a Large-Scale Distributed Virtual Reality System,” Proc. ACM Symp. Virtual Reality Software and Technology, pp. 33-40, 2000.
[42] J. Marsh, “A Software Architecture for Interactive Multiuser Visualisation,” PhD dissertation, The University of Manchester, 2002.
[43] C. Clark, K. Fraser, S. Hand, J.G. Hanseny, E. July, C. Limpach, I. Pratt, and A. Wareld, “Live Migration of Virtual Machines,” Proc. Second Symp. Networked Systems Design and Implementation, 2005.
[44] H. Weiß, “Dynamics of Geometrically Nonlinear Rods: I. Mechanical Models and Equations of Motion,” Nonlinear Dynamics, vol. 30, pp. 357-381, 2002.
[45] H. Weiß, “Dynamics of Geometrically Nonlinear Rods: II. Mechanical Models and Equations of Motion,” Nonlinear Dynamics, vol. 30, pp. 383-415, 2002.
[46] J. Marsh, S. Pettifer, and A. West, “A Technique for Maintaining Continuity of Experience in Networked Virtual Environments,” Proc. UK Virtual Reality Special Interest Group (UKVRSIG), Sept. 1999.
[47] M. Fraser, T. Glover, I. Vaghi, S. Benford, C. Greenhalgh, J. Hindmarsh, and C. Heath, “Collaborative Virtual Environments,” Proc. Third Int'l Conf. Collaborative Virtual Environments, pp. 29-37, 2000.
[48] R. Sekuler, A.B. Sekuler, and R. Lau, “Sound Alters Visual Motion Perception,” Nature, vol. 385, p. 308, Jan. 1997.
[49] C.A. Ellis, S.J. Gibbs, and G. Rein, “Groupware: Some Issues and Experiences,” Comm. ACM, vol. 34, no. 1, pp. 39-58, Jan. 1991.
[50] S. Hemminger, “Network Emulator,” online document, /, Mar. 2005.

Index Terms:
Virtual reality, network architecture and design, haptic I/O, computer-supported collaborative work, computer-supported cooperative work, simulation, modeling, and visualization, client/server, distributed applications, computer-aided design.
James Marsh, Mashhuda Glencross, Steve Pettifer, Roger Hubbold, "A Network Architecture Supporting Consistent Rich Behavior in Collaborative Interactive Applications," IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 3, pp. 405-416, May-June 2006, doi:10.1109/TVCG.2006.40
Usage of this product signifies your acceptance of the Terms of Use.