The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - July-September (2009 vol.2)
pp: 160-169
Maria Karam , Ryerson University, Toronto
Frank A. Russo , Ryerson University, Toronto
Deborah I. Fels , Ryerson University, Toronto
ABSTRACT
We present a Model Human Cochlea (MHC), a sensory substitution technique and system that translates auditory information into vibrotactile stimuli using an ambient, tactile display. The model is used in the current study to translate music into discrete vibration signals displayed along the back of the body using a chair form factor. Voice coils facilitate the direct translation of auditory information onto the multiple discrete vibrotactile channels, which increases the potential to identify sections of the music that would otherwise be masked by the combined signal. One of the central goals of this work has been to improve accessibility to the emotional information expressed in music for users who are Deaf or hard of hearing. To this end, we present our prototype of the MHC, two models of sensory substitution to support the translation of existing and new music, and some of the design challenges encountered throughout the development process. Results of a series of experiments conducted to assess the effectiveness of the MHC are discussed, followed by an overview of future directions for this research.
INDEX TERMS
Human factors, user interfaces, models and principles, music, sensory aids.
CITATION
Maria Karam, Frank A. Russo, Deborah I. Fels, "Designing the Model Human Cochlea: An Ambient Crossmodal Audio-Tactile Display", IEEE Transactions on Haptics, vol.2, no. 3, pp. 160-169, July-September 2009, doi:10.1109/TOH.2009.32
REFERENCES
[1] G.V. Bekesy, Experiments in Hearing. McGraw Hill, 1960.
[2] E. Bigand, S. Vieillard, F. Madurell, J. Marozeau, and A. Dacquet, “Multidimensional Scaling of Emotional Responses to Music: The Effect of Musical Expertise and of the Duration of the Excerpts,” Cognition and Emotion, vol. 19, no. 8, pp. 1113-1139, 2005.
[3] P. Brooks and B. Frost, “Evaluation of a Tactile Vocoder Device for Word Recognition,” J. Acoustical Soc. of Am., vol. 74, pp. 34-40, 1983.
[4] P. Brooks, B. Frost, J. Mason, and D. Gibson, “Continuing Evaluation of the Queen's University Tactile Vocoder i: Identification of Open-Set Sentences and Tracking Narrative,” J. Rehabilitation Research and Development, vol. 23, no. 1, pp. 129-138, 1986.
[5] E. Chew and A.R.J. Francois, “Interactive Multi-Scale Visualizations of Tonal Evolution in Musa.rt Opus 2,” Computers in Entertainment, vol. 3, no. 4, pp. 1-16, 2005.
[6] R.W. Cholewiak, A.A. Collins, and J.C. Brill, “Spatial Factors in Vibrotactile Pattern Perception,” Proc. Eurohaptics Conf., 2001.
[7] R.W. Cholewiak and C.M. McGrath, “Vibrotactile Targeting in Multimodal Systems: Accuracy and Interaction,” Proc. Virtual Reality 2006 Conf. in Conjunction with the IEEE Conf., pp. 22-23, 2006.
[8] J.C. Craig and P. Evans, “Vibrotactile Masking and the Persistence of Tactual Features,” Perception and Psychophysics, vol. 42, pp. 309-317, 1987.
[9] J.J. Gibson, The Ecological Approach to Visual Perception. Lawrence Erlbaum Assoc., 1979.
[10] E. Gunther, G. Davenport, and S. O'Modhrain, “Cutaneous Grooves: Composing for the Sense of Touch,” Proc. 2002 Conf. New Interfaces for Musical Expression (NIME '02), pp. 1-6, 2002.
[11] M. Karam and D.I. Fels, “Designing a Model Human Cochlea: Issues and Challenges in Crossmodal Audio-Haptic Displays,” Proc. 2008 Ambi-Sys Workshop Haptic User Interfaces in Ambient Media Systems (HAS '08), pp. 1-9, 2008.
[12] M. Karam, G. Nespoli, F. Russo, and D.I. Fels, “Modelling Perceptual Elements of Music in a Vibrotactile Display for Deaf Users: A Field Study,” Proc. Int'l Conf. Advances in Computer-Human Interaction, pp. 249-254, 2009.
[13] C. Kayser, C. Petkov, M. Augath, and N. Logothetis, “Integration of Touch and Sound in Auditory Cortex,” Neuron, vol. 48, no. 2, pp. 373-384, 2005.
[14] J.H. Kirman, “Vibrotactile Frequency Recognition: Forward and Backward Masking Effects,” J. General Psychology, vol. 113, no. 2, pp. 47-58, 1986.
[15] V. Lévesque, J. Pasquero, V. Hayward, and M. Legault, “Display of Virtual Braille Dots by Lateral Skin Deformation: Feasibility Study,” ACM Trans. Applied Perception, vol. 2, no. 2, pp. 132-149, 2005.
[16] J. Loomis and S. Lederman, Tactual Perception, pp. 31-41. Wiley, 1986.
[17] D.A. Mahns, N.M. Perkins, V. Sahai, L. Robinson, and M.J. Rowe, “Vibrotactile Frequency Discrimination in Human Hairy Skin,” J.Neurophysiology, vol. 95, pp. 1442-1450, 2006.
[18] M.T. Marshall and M.M. Wanderley, “Vibrotactile Feedback in Digital Musical Instruments,” Proc. 2006 Conf. New Interfaces for Musical Expression (NIME '06), pp. 226-229, 2006.
[19] J.B. Mitroo, N. Herman, and N.I. Badler, “Movies from Music: Visualizing Musical Compositions,” Proc. ACM SIGGRAPH '79, pp. 218-225, 1979.
[20] E. Rusconi, B. Kwan, B.L. Giordano, C. Umilita, and B. Butterworth, “Spatial Representation of Pitch Height: The Smarc Effect,” Cognition, vol. 99, pp. 113-129, 2006.
[21] J.A. Russell, “Evidence of Convergent Validity on the Dimensions of Affect,” J. Personality and Social Psychology, vol. 36, pp. 1152-1168, 1978.
[22] F.A. Russo, L.L. Cuddy, A. Galembo, and W.F. Thompson, “Sensitivity to Tonality across the Pitch Range,” Perception, vol. 36, pp. 781-790, 2007.
[23] D.C. Sinclair, Mechanisms of Cutaneous Sensation, chapter xi, p. 363. Oxford Univ. Press, 1981.
[24] C. Strumf, Tonpsychologie, vol. 1. S. Hirzel, 1883.
[25] Tadoma, The Tadoma Method, Web Resource, 2007.
[26] W. Thompson, F. Russo, and D. Sinclair, “Effects of Underscoring on the Perception of Closure in Film Excerpts,” Psychomusicology, vol. 13, pp. 9-27, 1994.
[27] T. Tommerdahl, K. Hester, E. Felix, M. Hollins, O. Favorov, P. Quibrera, and B. Whitsel, “Human Vibrotactile Frequency Discriminative Capacity After Adaptation to 25 Hz or 200 Hz Stimulation,” Brain Research, vol. 1057, pp. 1-9, 2005.
[28] J.B.F. Van Erp, H.A.H.C. Van Veen, C. Jansen, and T. Dobbins, “Waypoint Navigation with a Vibrotactile Waist Belt,” ACM Trans. Applied Perception, vol. 2, no. 2, pp. 106-117, 2005.
[29] P.B.Y. Rita, “Tactile Vision Substitution: Past and Future,” Int'l J. Neuroscience, vol. 19, pp. 29-36, 1983.
360 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool