Issue No.04 - Fourth Quarter (2012 vol.5)
pp: 289-300
D. Prattichizzo , Dept. of Inf. Eng., Univ. of Siena, Siena, Italy
C. Pacchierotti , Dept. of Inf. Eng., Univ. of Siena, Siena, Italy
G. Rosati , Sch. of Eng., Univ. of Padua, Padua, Italy
A novel sensory substitution technique is presented. Kinesthetic and cutaneous force feedback are substituted by cutaneous feedback (CF) only, provided by two wearable devices able to apply forces to the index finger and the thumb, while holding a handle during a teleoperation task. The force pattern, fed back to the user while using the cutaneous devices, is similar, in terms of intensity and area of application, to the cutaneous force pattern applied to the finger pad while interacting with a haptic device providing both cutaneous and kinesthetic force feedback. The pattern generated using the cutaneous devices can be thought as a subtraction between the complete haptic feedback (HF) and the kinesthetic part of it. For this reason, we refer to this approach as sensory subtraction instead of sensory substitution. A needle insertion scenario is considered to validate the approach. The haptic device is connected to a virtual environment simulating a needle insertion task. Experiments show that the perception of inserting a needle using the cutaneous-only force feedback is nearly indistinguishable from the one felt by the user while using both cutaneous and kinesthetic feedback. As most of the sensory substitution approaches, the proposed sensory subtraction technique also has the advantage of not suffering from stability issues of teleoperation systems due, for instance, to communication delays. Moreover, experiments show that the sensory subtraction technique outperforms sensory substitution with more conventional visual feedback (VF).
haptic interfaces, force feedback, communication delays, cutaneous force feedback, sensory subtraction technique, haptics, CF, index finger, teleoperation task, force pattern, teleoperation systems, Sensors, Needles, Force feedback, Robot sensing systems, Visualization, Biological tissues, tactile force feedback, Sensory substitution, cutaneous force feedback, wearable devices, haptic devices, needle insertion
D. Prattichizzo, C. Pacchierotti, G. Rosati, "Cutaneous Force Feedback as a Sensory Subtraction Technique in Haptics", IEEE Transactions on Haptics, vol.5, no. 4, pp. 289-300, Fourth Quarter 2012, doi:10.1109/TOH.2012.15
[1] A. Okamura, "Haptic Feedback in Robot-Assisted Minimally Invasive Surgery," Current Opinion in Urology, vol. 19, no. 1, pp. 102-107, 2009.
[2] J. Bishoff, D. Stoianovici, B. Lee, J. Bauer, R. Taylor, L. Whitcomb, J. Cadeddu, D. Chan, and L. Kavoussi, "Rcm-Paky: Clinical Application of a New Robotic System for Precise Needle Placement," J. Endourology, vol. 12, p. S82, 1998.
[3] A. Zivanovic and B. Davies, "A Robotic System for Blood Sampling," IEEE Trans. Information Technology in Biomedicine, vol. 4, no. 1, pp. 8-14, Mar. 2000.
[4] K. Masamune, E. Kobayashi, Y. Masutani, M. Suzuki, T. Dohi, H. Iseki, and K. Takakura, "Development of an Mri-Compatible Needle Insertion Manipulator for Stereotactic Neurosurgery," Computer Aided Surgery, vol. 1, no. 4, pp. 242-248, 1995.
[5] P. Rizun, P. McBeth, D. Louw, and G. Sutherland, "Robot-Assisted Neurosurgery," Surgical Innovation, vol. 11, no. 2, pp. 99-106, 2004.
[6] S. DiMaio and S. Salcudean, "Needle Insertion Modeling and Simulation," IEEE Trans. Robotics and Automation, vol. 19, no. 5, pp. 864-875, Oct. 2003.
[7] J. Hing, A. Brooks, and J. Desai, "Reality-Based Needle Insertion Simulation for Haptic Feedback in Prostate Brachytherapy," Proc. IEEE Int'l Conf. Robotics and Automation (ICRA), vol. 1, pp. 619-624, 2006.
[8] N. Abolhassani, R. Patel, and M. Moallem, "Needle Insertion into Soft Tissue: A Survey," Medical Eng. and Physics, vol. 29, no. 4, pp. 413-431, 2007.
[9] D. Lorenzo, E. Momi, I. Dyagilev, R. Manganelli, A. Formaglio, D. Prattichizzo, M. Shoham, and G. Ferrigno, "Force Feedback in a Piezoelectric Linear Actuator for Neurosurgery," The Int'l J. Medical Robotics and Computer Assisted Surgery, vol. 7, no. 3, pp. 268-275, 2011.
[10] I. Nisky, A. Pressman, C. Pugh, F. Mussa-Ivaldi, and A. Karniel, "Perception and Action in Teleoperated Needle Insertion," IEEE Trans. Haptics, vol. 4, no. 3, pp. 155-166, July-Sept. 2011.
[11] D. Lawrence, "Stability and Transparency in Bilateral Teleoperation," IEEE Trans. Robotics and Automation, vol. 9, no. 5, pp. 624-637, Oct. 1993.
[12] K. Hashtrudi-Zaad and S. Salcudean, "Transparency in Time-Delayed Systems and the Effect of Local Force Feedback for Transparent Teleoperation," IEEE Trans. Robotics and Automation, vol. 18, no. 1, pp. 108-114, Feb. 2002.
[13] D. Lee and M. Spong, "Passive Bilateral Teleoperation with Constant Time Delay," IEEE Trans. Robotics, vol. 22, no. 2, pp. 269-281, Apr. 2006.
[14] G. Niemeyer and J. Slotine, "Stable Adaptive Teleoperation," IEEE J. Oceanic Eng., vol. 16, no. 1, pp. 152-162, Jan. 1991.
[15] Y. Ye and P. Liu, "Improving Haptic Feedback Fidelity in Wave-Variable-Based Teleoperation Orientated to Telemedical Applications," IEEE Trans. Instrumentation and Measurement, vol. 58, no. 8, pp. 2847-2855, Aug. 2009.
[16] R. Anderson and M. Spong, "Bilateral Control of Teleoperators with Time Delay," IEEE Trans. Automatic Control, vol. 34, no. 5, pp. 494-501, May 1989.
[17] R. Adams, M. Moreyra, and B. Hannaford, "Stability and Performance of Haptic Displays: Theory and Experiments," Proc. ASME Int'l Mechanical Eng. Congress and Exhibition, vol. 1, pp. 227-234, 1998.
[18] B. Black and W. Book, "Dynamic Compensating Controller for Passive Haptic Manipulators in Teleoperation," Proc. IEEE Int'l Conf. Robotics and Automation, vol. 1, pp. 1485-1491, 2009.
[19] A. Lecuyer, S. Coquillart, A. Kheddar, P. Richard, and P. Coiffet, "Pseudo-Haptic Feedback: Can Isometric Input Devices Simulate Force Feedback?" Proc. IEEE Virtual Reality, vol. 1, pp. 83-90, 2000.
[20] J. Kim and J. Ryu, "Robustly Stable Haptic Interaction Control Using an Energy-Bounding Algorithm," The Int'l J. Robotics Research, vol. 29, no. 6, pp. 666-679, 2010.
[21] F. Conti and O. Khatib, "A New Actuation Approach for Haptic Interface Design," The Int'l J. Robotics Research, vol. 28, no. 6, pp. 834-848, 2009.
[22] R. Schoonmaker and C. Cao, "Vibrotactile Force Feedback System for Minimally Invasive Surgical Procedures," Proc. IEEE Int'l Conf. Systems, Man and Cybernetics, vol. 3, pp. 2464-2469, 2006.
[23] M. Massimino and T. Sheridan, "Sensory Substitution for Force Feedback in Teleoperation," Presence: Teleoperators and Virtual Environments, vol. 2, no. 4, pp. 344-352, 1993.
[24] M. Tavakoli, R. Patel, and M. Moallem, "Haptic Feedback and Sensory Substitution during Telemanipulated Suturing," Proc. Eurohaptics and Symp. Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics '05), vol. 1, pp. 543-544, 2005.
[25] M. Kitagawa, D. Dokko, A. Okamura, and D. Yuh, "Effect of Sensory Substitution on Suture-Manipulation Forces for Robotic Surgical Systems," J. Thoracic and Cardiovascular Surgery, vol. 129, no. 1, pp. 151-158, 2005.
[26] C. Wagner, S. Lederman, and R. Howe, "A Tactile Shape Display Using RC Servomotors," Proc. 10th Symp. Haptic Interfaces for Virtual Environment and Teleoperator Systems., vol. 1, pp. 354-355, 2002.
[27] N. Garcia-Hernandez, N. Tsagarakis, and D. Caldwell, "Feeling through Tactile Displays: A Study on the Effect of the Array Density and Size on the Discrimination of Tactile Patterns," IEEE Trans. Haptics, vol. 4, no. 2, pp. 100-110, Apr.-June 2011.
[28] K. Minamizawa, S. Fukamachi, H. Kajimoto, N. Kawakami, and S. Tachi, "Gravity Grabber: Wearable Haptic Display to Present Virtual Mass Sensation," Proc. ACM SIGGRAPH '07, vol. 1, pp. 250-258, 2007.
[29] M. Wijntjes, A. Sato, V. Hayward, and A. Kappers, "Local Surface Orientation Dominates Haptic Curvature Discrimination," IEEE Trans. Haptics, vol. 2, no. 2, pp. 94-102, Apr.-June 2009.
[30] A. Ferber, M. Peshkin, and J. Colgate, "Using Kinesthetic and Tactile Cues to Maintain Exercise Intensity," IEEE Trans. Haptics, vol. 2, no. 4, pp. 224-235, Oct.-Dec. 2009.
[31] K. Minamizawa, D. Prattichizzo, and S. Tachi, "Simplified Design of Haptic Display by Extending One-Point Kinesthetic Feedback to Multipoint Tactile Feedback," Proc. IEEE Haptic Symp., vol. 1, pp. 257-260, 2010.
[32] D. Prattichizzo, C. Pacchierotti, S. Cenci, K. Minamizawa, and G. Rosati, "Using a Fingertip Tactile Device to Substitute Kinesthetic Feedback in Haptic Interaction," Haptics: Generating and Perceiving Tangible Sensations, Lecture Notes in Computer Science, A. Kappers, J. van Erp, W. Bergmann Tiest, and F. van der Helm, eds., pp. 125-130, vol. 6191, Springer, 2010.
[33] I. Birznieks, P. Jenmalm, A.W. Goodwin, and R.S. Johansson, "Encoding of Direction of Fingertip Forces by Human Tactile Afferents," J. Neuroscience, vol. 21, no. 20, pp. 8222-8237, 2001.
[34] V. Hayward, O. Astley, M. Cruz-Hernandez, D. Grant, and G. Robles-De-La-Torre, "Haptic Interfaces and Devices," Sensor Rev., vol. 24, no. 1, pp. 16-29, 2004.
[35] F. Chinello, M. Malvezzi, C. Pacchierotti, and D. Prattichizzo, "A Three DoFs Wearable Tactile Display for Exploration and Manipulation of Virtual Objects," Proc. IEEE Haptic Symp., 2012.
[36] B. Dasgupta and T.S. Mruthyunjaya, "The Stewart Platform Manipulator: A Review," Mechanism and Machine Theory, vol. 35, no. 1, pp. 15-40, 2000.
[37] L. Rosenberg, "Virtual Fixtures: Perceptual Tools for Telerobotic Manipulation," Proc. IEEE Virtual Reality Ann. Int'l Symp., vol. 1, pp. 76-82, 1993.
[38] J. Abbott, P. Marayong, and A. Okamura, "Haptic Virtual Fixtures for Robot-Assisted Manipulation," Robotics Research, vol. 28, no. 1, pp. 49-64, 2007.
[39] B. Davies, "Robotic Devices in Surgery," Minimally Invasive Therapy and Allied Technologies, vol. 12, no. 1, pp. 5-13, 2003.
[40] S. Cotin, H. Delingette, and N. Ayache, "Real-Time Elastic Deformations of Soft Tissues for Surgery Simulation," IEEE Trans. Visualization and Computer Graphics, vol. 5, no. 1, pp. 62-73, Jan.-Mar. 1999.
[41] C. Zilles and J. Salisbury, "A Constraint-Based God-Object Method for Haptic Display," Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, vol. 3, pp. 146-151, 1995.
[42] I. Nisky, A. Pressman, C.M. Pugh, F.A. Mussa-Ivaldi, and A. Karniel, "Perception and Action in Teleoperated Needle Insertion," IEEE Trans. Haptics, vol. 4, no. 3, pp. 155-166, July-Sept. 2011.
[43] T. Lam, M. Mulder, and M. Van Paassen, "Haptic Feedback in UAV Tele-Operation with Time Delay," J. Guidance, Control and Dynamics, vol. 31, no. 6, pp. 1728-1739, 2008.
[44] E. Kandel, J. Schwartz, and T. Jessell, Principles of Neural Science, fourth ed. McGraw-Hill, 2000.
[45] G. Robles-De-La-Torre, "The Importance of the Sense of Touch in Virtual and Real Environments," IEEE Multimedia, vol. 13, no. 3, pp. 24-30, July-Sept. 2006.
[46] J. Jeka et al., "The Structure of Somatosensory Information for Human Postural Control," Motor Control, vol. 2, no. 1, pp. 13-33, 1998.
[47] S. Hirche, A. Bauer, and M. Buss, "Transparency of Haptic Telepresence Systems with Constant Time Delay," Proc. IEEE Conf. Control Applications, vol. 1, pp. 328-333, 2005.