The Community for Technology Leaders
RSS Icon
Issue No.03 - July-September (2012 vol.3)
pp: 285-297
Iman Mohammad Rezazadeh , Islamic Azad University, Tehran
Mohammad Firoozabadi , Islamic Azad University, Tehran and Tarbiat Modares University, Tehran
Huosheng Hu , University of Essex, Colchester
S. Mohammad Reza Hashemi Golpayegani , Islamic Azad University, Tehran and Amir Kabir University of Technology, Tehran
The real-time adaptation between human and assistive devices can improve the quality of life for amputees, which, however, may be difficult to achieve since physical and mental states vary over time. This paper presents a co-adaptive human-machine interface (HMI) that is developed to control virtual forearm prosthesis over a long period of operation. Direct physical performance measures for the requested tasks are calculated. Bioelectric signals are recorded using one pair of electrodes placed on the frontal face region of a user to extract the mental (affective) measures (the entropy of the alpha band of the forehead electroencephalography signals) while performing the tasks. By developing an effective algorithm, the proposed HMI can adapt itself to the mental states of a user, thus improving its usability. The quantitative results from 16 users (including an amputee) show that the proposed HMI achieved better physical performance measures in comparison with the traditional (nonadaptive) interface ({\rm p\hbox{-}value}<0.001). Furthermore, there is a high correlation (correlation coefficient < 0.9, {\rm p\hbox{-}value} < .01) between the physical performance measures and self-report feedbacks based on the NASA TLX questionnaire. As a result, the proposed adaptive HMI outperformed a traditional HMI.
Training, Prosthetics, Electroencephalography, Entropy, Educational institutions, Real time systems, virtual reality, Human-machine interface, affective measure, forehead bioelectric signals, prosthetics, real-time adaptation
Iman Mohammad Rezazadeh, Mohammad Firoozabadi, Huosheng Hu, S. Mohammad Reza Hashemi Golpayegani, "Co-Adaptive and Affective Human-Machine Interface for Improving Training Performances of Virtual Myoelectric Forearm Prosthesis", IEEE Transactions on Affective Computing, vol.3, no. 3, pp. 285-297, July-September 2012, doi:10.1109/T-AFFC.2012.3
[1] D.G. Kamper, R L. Harvey, S. Suresh, and W.Z Rymer, “Relative Contributions of Neural Mechanisms Versus Muscle Mechanics in Promoting Finger Extension Deficits Following Stroke,” J. Muscle Nerve, vol. 28, pp. 309-318, 2003.
[2] D.G. Kamper and W.Z. Rymer, “Impairment of Voluntary Control of Finger Motion Following Stroke: Role of Inappropriate Muscle Co-Activation,” J. Muscle Nerve, vol. 24, pp. 673-681, 2001.
[3] K. Eng, E. Siekierka, P. Pyk, E. Chevrier, Y. Hauser, M. Cameirao, L. Holper, K. Hägni, L. Zimmerli, A. Duff, C. Schuster, C. Bassetti, P. Verschure, and D. Kiper, “Interactive Visuo-Motor Therapy for Stroke Rehabilitation,” J. Medical, Biological Eng. and Computing, vol. 45, pp. 901-907, 2007.
[4] M. Kelly-Hayes, J.T. Robertson, J.P. Bronderick, P.W. Duncan, L.A. Hershey, E.J. Roth, W.H. Thies, and C.A. Trombly, “The American Heart Association Stroke Outcome Classification,” J. Stroke, vol. 29, pp. 1274-1280, 1998.
[5] D. Jack, R. Boian, A.S. Merians, M. Tremaine, G.C. Burdea, S.V. Adamovich, M. Recce, and M.H. Poizner, “Virtual Reality-Enhanced Stroke Rehabilitation,” IEEE Trans. Neural Systems and Rehabilitation Eng., vol. 9, no. 3, pp. 308-318, Sept. 2001.
[6] M. Zecca, J. Carpaneto, S. Micera, M.C. Carrozza, P. Dario, K. Itoh, and A. Takanishi, “Evolutionary Design of a Fuzzy Classifier for EMG-Based Control—Control of a Multi-DoFs Underactuated Hand Prosthesis,” Proc. Robotics and Mechatronics Conf., pp. 26-28, May 2006.
[7] D.H. Silcox, M.D. Rooks, R.R. Vogel, and L.L. Fleming, “Myoelectric Protheses,” The J. Joint and Bone Surgery, vol. 75, no. 12, pp. 1781-1791, 1993.
[8] P. Shenoy, K.J. Miller, B. Crawford, and R.N. Rao, “Online Electromyographic Control of a Robotic Prosthesis,” IEEE Trans. Biomedical Eng. vol. 55, no. 3, pp. 1128-1135, Mar. 2008.
[9] M. Zecca, S. Micera, M.C. Carrozza, and P. Dario, “On the Control of Multifunctional Prosthetic Hands by Processing the Electromyographic Signal,” Critical Rev. in Biomedical Eng., vol. 30, nos. 4-6, pp. 459-485, 2002.
[10] M.A. Oskoei and H. Hu, “Myoelectric Control Systems—A Survey,” J. Biomedical Signal Processing and Control, vol. 2, no. 4, pp. 275-294, 2007.
[11] J.L. Pons, Wearable Robots: Biomechatronic Exoskeleton, first ed., pp. 1-16. Wiley & Sons, 2008.
[12] R. Kato, T. Fujita, H. Yokoi, and T. Arai, “Adaptable EMG Prosthetic Hand Using On-Line Learning Method-Investigation of Mutual Adaptation between Human and Adaptable Machine,” Proc. IEEE 15th Int'l Symp. Robot and Human Interactive Comm., pp. 599-604, Sept. 2006.
[13] K. Momen, S. Krishnan, and T. Chau, “Real-Time Classification of Forearm Electromyographic Signals Corresponding to User-Selected Intentional Movements for Multifunction Control,” IEEE Trans. Neural Systems and Rehabilitation Eng., vol. 15, no. 4, pp. 535-542, Dec. 2007.
[14] A.B. Ajiboye and R.F. Weir, “A Heuristic Fuzzy Logic Approach to EMG Pattern Recognition for Multi-functional Prosthesis Control,” IEEE Trans. Neural Systems and Rehabilitation Eng., vol. 13, no. 3, pp. 280-291, Sept. 2005.
[15] D. Nishikawa, “On-Line Learning Based Electromyogram to Forearm Motion Classifier with Motor Skill Evaluation,” JSME Int'l J. Series C, vol. 43, no. 4, pp. 906-915, 2000.
[16] O. Fukuda, T. Tsuji, M. Kaneko, and A. Otsuka, “A Human-Assisting Manipulator Tele-Operated by EMG Signals and Arm Motions,” IEEE Trans. Robotics and Automation, vol. 19, no. 2, pp. 210-222, Apr. 2003.
[17] M. Holden, “Virtual Environments for Motor Rehabilitation: Review,” J. Cyberpsychology and Behavior, vol. 8, no. 3, pp. 187-211, 2005.
[18] I. Mohammad Rezazadeh, X. Wang, M. Firoozabadi, and M. Hashemi Golpayegani, “Using Affective Human-Machine Interface to Increase the Operation Performance in Virtual Construction Crane Training System: A Novel Approach Automation in Construction,” Automation in Construction, vol. 20, pp. 289-298, 2011.
[19] I. Mohammad Rezazadeh, X. Wang, R. Wang, and S.M.P. Firoozabadi, “Toward Affective Handsfree Human-Machine Interface Approach in Virtual Environment-Based Equipment Operation Training,” Proc. Ninth Int'l Conf. Construction Applications of Virtual Reality, 2009.
[20] E.P. Doherty, G. Cockton, J. Rizzo, B. Blondina, and B. Davis, “Yes/No or Maybe—Further Evaluation of an Interface for Brain-Injured Individuals,” Interacting with Computers, vol. 14, no. 4, pp. 341-358, 2002.
[21] T. Surdilovic and Y-Q Zhang, “Convenient Intelligent Cursor Control Web Systems for Internet Users with Severe Motor-Impairments,” Int'l J. Medical Informatics, vol. 75, no. 1, pp. 86-100, 2006.
[22] Cyberlink,, Mar. 2011.
[23] Biopac, http:/, Mar. 2011.
[24] V. Moertini, “Introduction to Five Data Clustering Algorithm,” Integral, vol. 7, no. 2, 2002.
[25] A. Priyona, M. Ridwan, A. Alias, R. Atiq, R. Rahmat, A. Hassan, and M. Ali, “Generation of Fuzzy Rules with Subtractive Clustering,” Universiti Teknologi Malaysia, Jurnal Teknologi, vol. 43(D) Dis, pp. 143-153, 2003.
[26] K. Polat, S. Yosunkaya, and S. Gunes, “Comparison of Different Classifier Algorithms on the Automated Detection of Obstructive Sleep Apnea Syndrome,” J. Medical Systems, vol. 32, pp. 243-250, 2008.
[27] M. Khezri and M. Jahed, “Real-Time Intelligent Pattern Recognition Algorithm for Surface EMG Signals,” Biomedical Eng. Online, vol. 3, no. 6, p. 45, 2003.
[28] N.J. ovec, K.J. ovec, and I. Gerlic, “The Influence of Mozart's Music on Brain Activity in the Process of Learning,” Clinical Neurophysiology, vol. 117, pp. 2703-2714, 2006.
[29] S. Micheloyannis, M. Vourkas, M. Bizas, P. Simos, and C.J. Stam, “Changes in Linear and Nonlinear EEG Measures as a Function of Task Complexity: Evidence for Local and Distant Signal Synchronization,” J. Brain Topography, vol. 15, no. 4, pp. 239-247, Summer 2003.
[30] J. Hu, D. Xiao, and Z. Mu, “Application of Energy Entropy in Motor Imagery EEG Classification,” Int'l J. Digital Content Technology and Its Appl., vol. 3, no. 2, pp. 83-90, June 2009.
[31] S. Aydin, H.M. Saraoglu, and S. Kara, “Log Energy Entropy-Based EEG Classification with Multilayer Neural Networks in Seizure,” Annals of Biomedical Eng., vol. 37, no. 12, pp. 2626-2630, Dec. 2009.
[32] MAYA, www.autodesc.commaya, May 2010.
[33] Mathworks,, May 2010.
[34] R. Hornero, D.E. Abasolo, N. Jimeno, and P. Espino, “Applying Approximate Entropy and Central Tendency Measure to Analyze Time Series Generated by Schizophrenic Patients,” Proc. 25th Ann. IEEE Int'l Conf. Eng. in Medicine and Biology Soc., Sept. 2003.
[35] J. Liu, C. Zhang, and C. Zheng, “EEG-Based Estimation of Mental Fatigue by Using KPCA-HMM and Complexity Parameters,” J. Biomedical Signal Processing and Control, vol. 5, pp. 124-130, 2010.
[36] L.A. Frey Law and R.K. Shields, “Mathematical Models of Human Paralyzed Muscle After Long-Term Training,” J. Biomechanics, vol. 40, pp. 2587-2595, 2007.
[37] M. Serruyaa and M.J. Kahanab, “Techniques and Devices to Restore Cognition,” Behavioural Brain Research, vol. 192, pp. 149-165, 2008.
[38] Q.H. Mach, M.D. Hunter, and R.S. Grewal, “Neurophysiological Correlates in Interface Design: An HCI Perspective,” Computers in Human Behavior, vol. 26, pp. 371-376, 2010.
[39] B.H. Kantowitz, “Mental Workload,” Human Factors Psychology, P.A. Hancock ed., pp. 81-121, Elsevier, 1987.
[40] J.A. Coan and J.J. Allen, “Frontal EEG Asymmetry as a Moderator and Mediator of Emotion,” Biological Psychology, vol. 67, nos. 1/2, pp. 7-49, 2004.
[41] H. Matsunaga and H. Nakazawa, “Design Method Consideration Human Satisfaction: Development of Adaptive Human-Machine Interface Based on Satisfaction Measures,” Human Factors and Ergonomics in Manufacturing, vol 9, no. 3, pp. 253-266, 1999.
[42] M.J. Larson, D.A. Good, and J.E. Fair, “The Relationship between Performance Monitoring, Satisfaction with Life, and Positive Personality Traits,” Biological Psychology, vol. 83, pp. 222-228, 2010.
[43] J. Sleigh, L. Voss, and J. Barnard, “What Are Electroencephalogram Entropies Really Measuring?” Int'l Congress Series, vol. 1283, pp. 231-234, 2005.
[44] O.A. Rosso, “Entropy Changes in Brain Function,” Int'l J. Psychophysiology, vol. 64, pp. 75-80, 2007.
[45] H. Yoo, M. Kim, and O. Kwon, “Emotional Index Measurement Method for Context-Aware Service,” Expert Systems with Applications, vol. 38, no. 1, pp. 785-793, 2010.
[46] J. Coan, J.J. Allen, and E. Harmon-Jones, “Voluntary Facial Expression and Hemispheric Asymmetry over the Frontal Cortex,” Psychophysiology, vol. 38, pp. 912-925, 2010.
[47] T. Liu, J. Shi, D. Zhao, and J. Yang, “The Relationship between EEG Band Power, Cognitive Processing and Intelligence in School-Age Children,” Psychology Science Quarterly, vol. 50, no. 2, pp. 259-268, 2008.
[48] C.S. Nam, S. Johnson, Y. Li, and Y. Seong, “Evaluation of Human-Agent User Interfaces in Multi-Agent Systems,” Int'l J. Industrial Ergonomics, vol. 39, pp. 192-201, 2009.
[49] S. Rubio, E. Diaz, and J. Martin, “Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA-TLX, and Workload Profile Methods,” Applied Psychology: An Int'l Rev., vol. 53, pp. 61-86, 2004.
[50] D. Perani, F. Fazio, N.A. Borghese, M. Tettamanti, S. Ferrari, J. Decety, and M.C. Gilardi, “Different Brain Correlates for Watching Real and Virtual Hand Actions,” NeuroImage, vol. 14, pp. 749-758, 2001.
[51] S. Subramanian, L.A. Knaut, C. Beaudoin, B.J. McFadyen, A.G. Feldman, and M.F. Levin, “Virtual Reality Environments for Post-Stroke Arm Rehabilitation,” J. NeuroEng. and Rehabilitation, vol. 4, no. 2, p. 20, 2007.
[52] H. Sveistrup, “Motor Rehabilitation Using Virtual Reality,” J. NeuroEng. and Rehabilitation, vol. 1, no. 1, p. 10, 2004.
[53] C. Berka, “EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks,” Aviation, Space, and Environmental Medicine, vol. 78, no. 5, pp. B231-B244, May 2007.
[54] A.W.K. Gaillard, “Stress, Workload, and Fatigue as Three Biobehavioural States: A General Overview,” Stress, Workload, and Fatigue, P.A. Hancock and P.A. Desmond eds., pp. 623-639, Erlbaum, 2001.
[55] M. Wyczesany1, J. Kaiser1, and A.M.L. Coenen, “Subjective Mood Estimation Co-Varies with Spectral Power EEG Characteristics,” Acta Neurobiologiae Experimentalis, vol. 68, pp. 180-192, 2008.
[56] G. Chanel, C. Rebetez, M. Betrancourt, and T. Pun, “Emotion Assessment from Physiological Signals for Adaptation of Games Difficulty,” IEEE Trans. Systems, Man, and Cybernetics—Part A: Systems and Humans, vol. 41, pp. 1052-1063, Nov. 2011.
[57] G. Chanel, C. Rebetez, M. Betrancourt, and T. Pun, “Boredom, Engagement and Anxiety as Indicators for Adaptation to Difficulty in Games,” Proc. 12th Int'l Conf. Entertainment and Media in the Ubiquitous Era, Oct. 2008.
[58] SENIAM Project Surface ElectroMyoGraphy for the Non-Invasive Assessment of Muscles, http:/, Nov. 2011.
28 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool