This Article 
 Bibliographic References 
 Add to: 
Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair
October 2011 (vol. 17 no. 10)
pp. 1355-1368
Steven Henderson, Columbia University, New York
Steven Feiner, Columbia University, New York
We explore the development of an experimental augmented reality application that provides benefits to professional mechanics performing maintenance and repair tasks in a field setting. We developed a prototype that supports military mechanics conducting routine maintenance tasks inside an armored vehicle turret, and evaluated it with a user study. Our prototype uses a tracked headworn display to augment a mechanic's natural view with text, labels, arrows, and animated sequences designed to facilitate task comprehension, localization, and execution. A within-subject controlled user study examined professional military mechanics using our system to complete 18 common tasks under field conditions. These tasks included installing and removing fasteners and indicator lights, and connecting cables, all within the cramped interior of an armored personnel carrier turret. An augmented reality condition was tested against two baseline conditions: the same headworn display providing untracked text and graphics and a fixed flat panel display representing an improved version of the laptop-based documentation currently employed in practice. The augmented reality condition allowed mechanics to locate tasks more quickly than when using either baseline, and in some instances, resulted in less overall head movement. A qualitative survey showed that mechanics found the augmented reality condition intuitive and satisfying for the tested sequence of tasks.

[1] Interactive Electronic Technical Manuals—, General Content, Style, Format and User-Interaction Requirements, Dept. of Defense, 2007.
[2] “ARTESAS—Advanced Augmented Reality Technologies for Industrial Service Applications,” http:/, May 2009.
[3] M. Axholt, S. Peterson, and S.R. Ellis, “User Boresight Calibration Precision for Large-Format Head-Up Displays,” Proc. Virtual Reality Software and Technology (VRST '08), pp. 141-148, 2008.
[4] K.M. Baird and W. Barfield, “Evaluating the Effectiveness of Augmented Reality Displays for a Manual Assembly Task,” Virtual Reality, vol. 4, pp. 250-259, 1999.
[5] F. Biocca, A. Tang, C. Owen, and F. Xiao, “Attention Funnel: Omnidirectional 3D Cursor for Mobile Augmented Reality Platforms,” Proc. SIGCHI Conf. Human Factors in Comp. Systems (CHI '06), pp. 1115-1122, 2006.
[6] T.P. Caudell and D.W. Mizell, “Augmented Reality: An Application of Heads-Up Display Technology to Manual Manufacturing Processes,” Proc. Hawaii Int'l Conf. System Sciences, vol. 2, pp. 659-669, 1992.
[7] J.M. Chambers, Graphical Methods for Data Analysis. Duxbury Resource Center, 1983.
[8] D. Curtis, D. Mizell, P. Gruenbaum, and A. Janin, “Several Devils in the Details: Making an AR Application Work in the Airplane Factory,” Proc. Int'l Workshop Augmented Reality (IWAR '98), pp. 47-60, 1999.
[9] S.M. Donelson and C.C. Gordon, “1995 Matched Anthropometric Database of US Marine Corps Personnel: Summary Statistics,” Technical Report TR-96-036, Natick Research Development and Eng. Center, 1996.
[10] S. Feiner, “Apex: An Experiment in the Automated Creation of Pictorial Explanations,” IEEE Computer Graphics and Applications, vol. 5, no. 11, pp. 29-37, Nov. 1985.
[11] S. Feiner, B. MacIntyre, and D. Seligmann, “Knowledge-Based Augmented Reality,” Comm. ACM, vol. 36, pp. 53-62, 1993.
[12] W. Friedrich, “ARVIKA-Augmented Reality for Development, Production and Service,” Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR '02), pp. 3-4, 2002.
[13] C.C. Gordon, S.P. Paquette, J.D. Brantley, H.W. Case, and D.J. Gaeta, A Supplement to the 1995 Matched Anthropometric Database of US Marine Corps Personnel: Summary Statistics, Natick, 1997.
[14] J. Heiser, D. Phan, M. Agrawala, B. Tversky, and P. Hanrahan, “Identification and Validation of Cognitive Design Principles for Automated Generation of Assembly Instructions,” Proc. Working Conf. Advanced Visual Interfaces, pp. 311-319, 2004.
[15] S. Henderson and S. Feiner, “Evaluating the Benefits of Augmented Reality for Task Localization in Maintenance of an Armored Personnel Carrier Turret,” Proc. Int'l Symp. Mixed Augmented Reality (ISMAR '09), pp. 135-144. 2009.
[16] J.A. Jones, I.I.J. Edward Swan, G. Singh, E. Kolstad, and S.R. Ellis, “The Effects of Virtual Reality, Augmented Reality, and Motion Parallax on Egocentric Depth Perception,” Proc. Applied Perception in Graphics and Visualization, pp. 9-14, 2008.
[17] C. Knöpfle, J. Weidenhausen, L. Chauvigné, and I. Stock, “Template Based Authoring for AR Based Service Scenarios,” Proc. IEEE Virtual Reality (VR '05), pp. 249-252, 2005.
[18] A.M. Law and W.D. Kelton, Simulation Modeling and Analysis. McGraw-Hill Higher Education, 1997.
[19] U. Neumann and A. Majoros, “Cognitive, Performance, and Systems Issues for Augmented Reality Applications in Manufacturing and Maintenance,” Proc. IEEE Virtual Reality (VR '98), pp. 4-11, 1998.
[20] S. Nilsson and B. Johansson, “Fun and Usable: Augmented Reality Instructions in a Hospital Setting,” Proc. Australasian Conf. Computer-Human Interaction, pp. 123-130, 2007.
[21] J.J. Ockerman and A.R. Pritchett, “Preliminary Investigation of Wearable Computers for Task Guidance in Aircraft Inspection,” Proc. Int'l Symp. Wearable Computers (ISWC '98), pp. 33-40, 1998.
[22] S.K. Ong, M.L. Yuan, and A.Y.C. Nee, “Augmented Reality Applications in Manufacturing: A Survey,” Int'l J. Production Research, vol. 46, pp. 2707-2742, 2008.
[23] J. Platonov, H. Heibel, P. Meier, and B. Grollmann, “A Mobile Markerless AR System for Maintenance and Repair,” Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR '06), pp. 105-108, 2006.
[24] A. Raczynski and P. Gussmann, “Services and Training Through Augmented Reality,” Proc. European Conf. Visual Media Production (CVMP '04), pp. 263-271, 2004.
[25] D. Reiners, D. Stricker, G. Klinker, and S. Müller, “Augmented Reality for Construction Tasks: Doorlock Assembly,” Proc. Int'l Workshop Augmented Reality (IWAR '98), pp. 31-46, 1999.
[26] C.M. Robertson, B. MacIntyre, and B.N. Walker, “An Evaluation of Graphical Context When the Graphics are Outside of the Task Area,” Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR '08), pp. 73-76, 2008.
[27] J.P. Rolland, C.A. Burbeck, W. Gibson, and D. Ariely, “Towards Quantifying Depth and Size Perception in 3D Virtual Environments,” Presence, vol. 4, pp. 24-48, 1995.
[28] T. Salonen and J. Sääski, “Dynamic and Visual Assembly Instruction for Configurable Products Using Augmented Reality Techniques,” Advanced Design and Manufacture to Gain a Competitive Edge, pp. 23-32, Springer, 2008.
[29] B. Schwald and B.D. Laval, “An Augmented Reality System for Training and Assistance to Maintenance in the Industrial Context,” Proc. Winter School of Computer Graphics, pp. 425-432, 2003.
[30] B. Schwerdtfeger and G. Klinker, “Supporting Order Picking with Augmented Reality,” Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR '08), pp. 91-94, 2008.
[31] B. Schwerdtfeger, R. Reif, W.A. Gunthner, G. Klinker, D. Hamacher, L. Schega, I. Bockelmann, F. Doil, and J. Tumler, “Pick-by-Vision: A First Stress Test,” Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR '09), pp. 115-124, 2009.
[32] D.D. Seligmann and S. Feiner, “Automated Generation of Intent-Based 3D Illustrations,” Proc. Conf. Computer Graphics and Interactive Techniques, pp. 123-132, 1991.
[33] A. Smailagic and D. Siewiorek, “User-Centered Interdisciplinary Design of Wearable Computers,” ACM SIGMOBILE Mobile Computing and Comm. Rev., vol. 3, pp. 43-52, 1999.
[34] A. Tang, C. Owen, F. Biocca, and W. Mou, “Comparative Effectiveness of Augmented Reality in Object Assembly,” Proc. SIGCHI Conf. Human Factors in Comp. Systems (CHI '03), pp. 73-80, 2003.
[35] M. Tönnis and G. Klinker, “Effective Control of a Car Driver's Attention for Visual and Acoustic Guidance Towards the Direction of Imminent Dangers,” Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR '06), pp. 13-22, 2006.
[36] J. Tümler, R. Mecke, M. Schenk, A. Huckauf, F. Doil, G. Paul, E. Pfister, I. Böckelmann, and A. Roggentin, “Mobile Augmented Reality in Industrial Applications: Approaches for Solution of User-Related Issues,” Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR '08), pp. 87-90, 2008.
[37] U.S. Army, Maintenance Operations and Procedures (Field Manual 4-30.3), 2007.
[38] U.S. Marine Corps, Light Armored Vehicle Operator's Manual (LAV-25A1), 2003.
[39] J. Wither, S. DiVerdi, and T. Hollerer, “Evaluating Display Types for AR Selection and Annotation,” Proc. Int'l Symp. Mixed and Augmented Reality, pp. 1-4, 2007.
[40] J. Zauner, M. Haller, A. Brandl, and W. Hartman, “Authoring of a Mixed Reality Assembly Instructor for Hierarchical Structures,” Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR '03), pp. 237-246, 2003.

Index Terms:
Industrial, military, user interfaces, virtual and augmented reality.
Steven Henderson, Steven Feiner, "Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair," IEEE Transactions on Visualization and Computer Graphics, vol. 17, no. 10, pp. 1355-1368, Oct. 2011, doi:10.1109/TVCG.2010.245
Usage of this product signifies your acceptance of the Terms of Use.