This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Optical Merger of Direct Vision with Virtual Images for Scaled Teleoperation
March/April 2006 (vol. 12 no. 2)
pp. 277-285
ASCII Text
x 
 
Samuel T. Clanton, David C. Wang, Vikram S. Chib, Yoky Matsuoka, George D. Stetten, "Optical Merger of Direct Vision with Virtual Images for Scaled Teleoperation," IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 2, pp. 277-285, March/April, 2006.
 
 
BibTex
x
 
@article{ 10.1109/TVCG.2006.35,
author = {Samuel T. Clanton and David C. Wang and Vikram S. Chib and Yoky Matsuoka and George D. Stetten},
title = {Optical Merger of Direct Vision with Virtual Images for Scaled Teleoperation},
journal ={IEEE Transactions on Visualization and Computer Graphics},
volume = {12},
number = {2},
issn = {1077-2626},
year = {2006},
pages = {277-285},
doi = {http://doi.ieeecomputersociety.org/10.1109/TVCG.2006.35},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Visualization and Computer Graphics
TI - Optical Merger of Direct Vision with Virtual Images for Scaled Teleoperation
IS - 2
SN - 1077-2626
SP277
EP285
EPD - 277-285
A1 - Samuel T. Clanton,
A1 - David C. Wang,
A1 - Vikram S. Chib,
A1 - Yoky Matsuoka,
A1 - George D. Stetten,
PY - 2006
KW - Artificial
KW - augmented
KW - and virtual realities
KW - image display
KW - medical information systems
KW - real time.
VL - 12
JA - IEEE Transactions on Visualization and Computer Graphics
ER -
 

Abstract—Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time. The method adapts a concept already used successfully in a new medical device called the Sonic Flashlight, which permits direct in situ visualization of ultrasound during invasive procedures. The Sonic Flashlight uses a flat-panel monitor and a half-silvered mirror to merge the visual outer surface of a patient with a simultaneous ultrasound scan of the patient's interior. Adapting the concept to scaled teleoperation involves removing the imaging device and the target to a remote location and adding a master-slave control device. This permits the operator to see his hands, along with what appears to be the tool, and the target, merged in a workspace that preserves natural hand-eye coordination. Three functioning prototypes are described, one based on ultrasound and two on light microscopy. The limitations and potential of the new approach are discussed.

[1] G.D. Stetten, “System and Method for Location-Merging of Real-Time Tomographic Slice Images with Human Vision,” US Patent no. 6,599,247, July 2003.
[2] G.D. Stetten, V.S. Chib, and R.J. Tamburo, “Tomographic Reflection to Merge Ultra-Sound Images with Direct Vision,” Proc. Applied Imagery Pattern Recognition Workshop, pp. 200-205, 2000.
[3] G. Stetten, V. Chib, D. Hildebrand, and J. Bursee, “Real Time Tomographic Reflection: Phantoms for Calibration and Biopsy,” Proc. IEEE and ACM Int'l Symp. Augmented Reality, pp. 11-18, Oct. 2001.
[4] G.D. Stetten and V.S. Chib, “Overlaying Ultrasound Images on Direct Vision,” J. Ultrasound in Medicine, vol. 20, pp. 235-240, 2001.
[5] W. Chang, G.D. Stetten, L. Lobes, and D. Shelton, “Guidance of Retrobulbar Injection with Real Time Tomographic Reflection,” J. Ultrasound in Medicine, vol. 21, pp. 1131-1135, 2002.
[6] A. State, M. Livingston, W. Garret, G. Hirota, M. Whitton, E. Pisano, and H. Fuchs, “Technologies for Augmented Reality Systems: Realizing Ultrasound-Guided Needle Biopsies,” Proc. ACM SIGGRAPH, pp. 439-446, 1996.
[7] M. Rosenthal, A. State, J. Lee, G. Hirota, J. Ackerman, K. Keller, E. Pisano, M. Jiroutek, K. Muller, and H. Fuchs, “Augmented Reality Guidance for Needle Biopsies: A Randomized, Controlled Trial in Phantoms,” Medical Image Analysis, vol. 6, no. 3, pp. 313-320, 2002.
[8] F. Sauer, A. Khamene, B. Bascle, L. Schimmang, F. Wenzel, and S. Vogt, “Augmented Reality Visualization of Ultrasound Images: System Description, Calibration, and Features,” Proc. Int'l Symp. Augmented Reality 2001, pp 30-39, 2001.
[9] S. Nicolau, A. Garcia, X. Pennec, L. Soler, and N. Ayache, “Augmented Reality Guided Radio-Frequency Tumor Ablation,” Computer Animation and Virtual World, vol. 16, no. 1, 2005.
[10] K. Knowlton, “Computer Displays Optically Superimposed on Input Devices,” The Bell System Technical J., vol. 56, no. 3, pp. 367-383, Mar. 1977.
[11] C. Schmandt, “Spatial Input/Display Correspondence in a Stereoscopic Computer Graphic Work Station,” Proc. ACM SIGGRAPH, vol. 17, no. 3, pp. 253-261, July 1983.
[12] A. DiGioia, B. Colgan, and N. Koerbel, Cybersurgery, R. Satava, ed., pp. 121-139. Wiley, 1998.
[13] M. Blackwell, F. Morgan, and A. DiGioia, “Augmented Reality and Its Future in Orthopaedics,” Clinical Orthopaedic and Related Research, vol. 345, pp. 111-122, 1998.
[14] K. Albrecht, M. Braun, S. Conrad, G. Goebbels, A. Grab, N. Hanssen, F. Hasenbrink, A. Ivanovic, E. Keeve, Z. Krol, R. Sader, and H.-F. Zeilhofer, “ARSyS-Tricorder: Development of an Augmented Reality System for Intra-Operative Navigation with Respect to the Requirements of Individual Transplant Design in Maxillo-Facial Surgery,” Abstract D-0020, European Congress of Radiology, Vienna, Mar. 2002.
[15] S. Hofstein, “Ultrasonic Scope,” US Patent no. 4,200,885, Apr. 1980.
[16] K. Masamune, G. Fichtinger, A. Deguet, D. Matsuka, and R. Taylor, “An Image Overlay System with Enhanced Reality for Percutaneous Therapy Performed inside CT Scanner,” Proc. Conf. Medical Image Computing and Computer-Assisted Intervention 2002, pp. 77-84, 2002.
[17] W. Birkfellner, M. Figl, K. Huber, F. Watzinger, F. Wanschitz, R. Hanel, A. Wagner, D. Rafolt, R. Ewers, and H. Bergmann, “The Varioscope AR— A Head-Mounted Operating Microscope for Augmented Reality,” Proc. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 869-877, 2000.
[18] P.J. Edwards, D.J. Hawkes, and D.L. Hill, “Augmentation of Reality Using an Operating Microscope for Otolaryngology and Neurosurgical Guidance,” J. Image Guided Surgery, vol. 1, pp. 172-178, 1995.
[19] B.J. Nelson and B. Vikramaditya, “Visually Servoed Micropositioning for Robotic Micromanipulation,” Microcomputer Applications, vol. 18, pp. 23-31, 1999.
[20] R. Taylor, W. Robinett, V. Chi, F. Brooks, W. Wright, R. Williams, and E. Snyder, “The Nanomanipulator: A Virtual-Reality Interface for a Scanning Tunneling Microscope,” Proc. SIGGRAPH '93, pp. 127-134, 1993.
[21] R. Superfine, M. Falvo, R. Taylor, and S. Washburn, “Nanomanipulation: Buckling, Transport and Rolling at the Nanoscale,” CRC Handbook of Nanoscience, Eng., and Technology, S. Lyshevski, D. Brenner, J. Lafrate, and W. Goddard, eds., CRC Press LLC, 2002.
[22] A. Nowatzyk, D. Shelton, J. Galeotti, and G. Stetten, “Extending the Sonic Flashlight to Real Time Tomographic Holography,” Proc. AMI-ARCS 2004, Workshop Augmented Environments for Medical Imaging Including Augmented Reality in Computer-Aided Surgery, Sept. 2004.

Index Terms:
Artificial, augmented, and virtual realities, image display, medical information systems, real time.
Citation:
Samuel T. Clanton, David C. Wang, Vikram S. Chib, Yoky Matsuoka, George D. Stetten, "Optical Merger of Direct Vision with Virtual Images for Scaled Teleoperation," IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 2, pp. 277-285, March-April 2006, doi:10.1109/TVCG.2006.35
Usage of this product signifies your acceptance of the Terms of Use.