The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan.-March (2013 vol.4)
pp: 2-14
Peng Ren , Florida International University, Miami
Armando Barreto , Florida International University, Miami
Ying Gao , University of Wisconsin- Platteville, Platteville
Malek Adjouadi , Florida International University, Miami
ABSTRACT
Previous research found that the pupil diameter (PD) can be an indication of affective state, but this approach to the detection of the affective state of a computer user has not been investigated fully. We propose a new affective sensing approach to evaluate the computer user's affective states as they transition from "relaxation” to "stress,” through processing the PD signal. Wavelet denoising and Kalman filtering were used to preprocess the PD signal. Then, three features were extracted from it and five classification algorithms were used to evaluate the overall performance of the identification of "stress” states in the computer users, achieving an average accuracy of 83.16 percent, with the highest accuracy of 84.21 percent reached with a Multilayer Perceptron and a Naive Bayes classifier. The Galvanic Skin Response (GSR) signal was also analyzed to study the comparative efficiency of affective sensing through the PD signal. We compared the discriminating power of the three features derived from the preprocessed PD signal to three features derived from the preprocessed GSR signal in terms of their Receiver Operating Characteristic curves. The results confirm that the PD signal should be considered a powerful physiological factor to involve in future automated affective classification systems for human-computer interaction.
INDEX TERMS
Human computer interaction, Multiresolution analysis, Low pass filters, Noise reduction, Kalman filters, Physiology, Receivers, Walsh transform, Human computer interaction, Multiresolution analysis, Low pass filters, Noise reduction, Kalman filters, Physiology, Receivers, receiver operating characteristic (ROC) curves, Affective computing, human-computer interaction (HCI), pupil diameter (PD), wavelet denoising, Kalman filter
CITATION
Peng Ren, Armando Barreto, Ying Gao, Malek Adjouadi, "Affective Assessment by Digital Processing of the Pupil Diameter", IEEE Transactions on Affective Computing, vol.4, no. 1, pp. 2-14, Jan.-March 2013, doi:10.1109/T-AFFC.2012.25
REFERENCES
[1] R.W. Picard, Affective Computing. MIT Press, 1997.
[2] C. Peter and R. Beale, Affect and Emotion in Human-Computer Interaction. Springer, 2008.
[3] A.R. Damasio, Descartes' Error: Emotion, Reason, and the Human Brain. Harper Perennial, 1995.
[4] B. Reeves and C. Nass, The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge Univ. Press, 1996.
[5] Univ. of California-Irvine "Short-Term Stress Can Affect Learning and Memory," ScienceDaily, Mar. 2008.
[6] A. De Santos, C. Sánchez-Avila, J. Guerra-Casanova, and G.B.-D. Pozo, "Real-Time Stress Detection by Means of Physiological Signals," Recent Application in Biometrics, J. Yang and N. Poh, eds., 2011.
[7] J.M. Fellous and M.A. Arbib, Who Needs Emotions? The Brain Meets the Robot. Oxford Univ. Press, 2005.
[8] W.A.N. Dorland, Dorland's Illustrated Medical Dictionary, 29th ed. Saunders, 2007.
[9] F.H. Martini and J.L. Nath, Fundamentals of Anatomy & Physiology, eighth ed. Benjamin Cummings, 2008.
[10] T. Partala and V. Surakka, "Pupil Size Variation as an Indication of Affective Processing," Int'l J. Human-Computer Studies, vol. 59, pp. 185-198, 2003.
[11] K. Holmqvist, M. Nystrom, R. Andersson, R. Dewhurst, H. Jarodzka, and J. van de Weijer, Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford Univ. Press, 2011.
[12] Z. Zhu and Q. Ji, "Robust Real-Time Eye Detection and Tracking under Variable Lighting Conditions and Various Face Orientations," Computer Vision and Image Understanding, vol. 98, no 1, pp. 124-154, Apr. 2005.
[13] C.L. Lim, C. Rennie, R.J. Barry, H. Bahramali, I. Lazzaro, and B. Manor, "Decomposing Skin Conductance into Tonic and Phasic Components," Int'l J. Psychophysiology, vol. 24, no. 2, pp. 97-109, Feb. 1997.
[14] Y. Shi, N. Ruiz, R. Taib, E. Choi, and F. Chen, "Galvanic Skin Response as an Index of Cognitive Load," Proc. Computer-Human Interaction Conf. Human Factors in Computing System, pp. 2651-2656, 2007.
[15] H. Frank, "An Experimental Comparison of the Psychological Stress Evaluator and the Galvanic Skin Response in Detection of Deception," J. Applied Psychology, vol. 63, no. 3, pp. 338-344, June 1978.
[16] M.E. Bitterman and W. Holtzman, "Conditioning and Extinction of the Galvanic Skin Response as a Function of Anxiety," J. Abnormal and Social Psychology, vol. 47, no. 3, pp. 615-623, July 1952.
[17] J. Beatty and B. Lucero-Wagoner, "The Pupillary System," Handbook of Psychophysiology, J.T. Cacioppo, L.G. Tassinary, G.G. Berntson, eds., second ed., pp. 142-162, Cambridge Univ. Press, 2000.
[18] S.R. Steinhauer, G.J. Siegle, R. Condray, and M. Pless, "Sympathetic and Parasympathetic Innervation of Pupillary Dilation during Sustained Processing," Int'l J. Psychophysiology, vol. 52, pp. 77-86, 2004.
[19] P.C. Bressloff and C.V. Wood, "Spontaneous Oscillations in a Nonlinear Delayed-Feedback Shunting Model of the Pupillary Light Reflex," Physical Rev., vol. 58, pp. 3597-3605, 1998.
[20] C. Darwin, The Expression of the Emotions in Man and Animals. Univ. of Chicago Press, 1965.
[21] J.R. Stroop, "Studies of Interference in Serial Verbal Reactions," J. Experimental Psychology, vol. 18, pp. 643-662, 1935.
[22] J.H. Tulen, P. Moleman, H.G. van Steenis, and F. Boomsma, "Characterization of Stress Reactions to the Stroop Color Word Test," Pharmacology Biochemistry and Behaviour, vol. 32, no. 1, pp. 9-15, Jan. 1989.
[23] P. Hjemdahl, U. Freyschuss, A. Juhlin-Dannfelt, and B. Linde, "Differentiated Sympathetic Activation during Mental Stress Evoked by the Stroop Test," Acta Physiol Scand Suppl., vol. 527, pp. 25-29, 1984.
[24] F.-T. Sun, C. Kuo, H.-T. Cheng, S. Buthpitiya, P. Collins, and M.L. Griss, "Activity-Aware Mental Stress Detection Using Physiological Sensors," Silicon Valley Campus, Paper 23, 2010.
[25] R.M. Stern, W.J. Ray, and K.S. Quigley, Psychophysiological Recording. Oxford Univ. Press, 2001.
[26] P. Renaud and J.P. Blondin, "The Stress of Stroop Performance: Physiological and Emotional Responses to Color-Word Interference, Task Pacing, and Pacing Speed," Int'l J. Psychophysiology, vol. 27, pp. 87-97, 1997.
[27] Y. Gao, A. Barreto, and M. Adjouadi, "Affective Assessment of a Computer User through the Processing of the Pupil Diameter Signal," Innovations in Computing Sciences and Software Eng., pp. 189-194, Springer Science, 2010.
[28] Y. Gao, A. Barreto, J. Zhai, and N. Rishe, "Digital Filtering of Pupil Diameter Variations for the Detection of Stress in Computer Users," Proc. 11th World Multi-Conf. Systemics, Cybernetics, and Informatics, p. 30, vol. 2, 2007.
[29] S.-J.S. Tsai, "Power Transformer Partial Discharge (PD) Acoustic Signal Detection Using Fiber Sensors and Wavelet Analysis, Modeling, and Simulation," master's thesis, Dept. of Electrical and Computer Eng., Virginia Polytechnic Inst. and State Univ., Blacksburg, 2002.
[30] M. Chabert, J.-Y. Tourneret, and F. Castanie, "Additive and Multiplicative Abrupt Jump Detection Using the Continuous Wavelet Transform," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 1996.
[31] S. Mallat, A Wavelet Tour of Signal Processing, third ed. Academic Press, 2008.
[32] M. Misiti, Y. Misiti, G. Oppenheim, and J. Poggi, "Wavelet Toolbox Manual-User's Guide," The Math Works Inc., 2011.
[33] I. Daubechies, Ten Lectures on Wavelets. SIAM, 1992.
[34] N.A. Thacker and A. Lacey, "Tutorial: The Kalman Filter," Imaging Science and Biomedical Eng. Division, Medical School, Univ. of Manchester, 1998.
[35] M. St-Pierre and D. Gingras, "Comparison between the Unscented Kalman Filter and the Extended Kalman Filter for the Position Estimation Module of an Integrated Navigation Information System," Proc. IEEE Intelligent Vehicles Symp., vol. 16, pp. 831-835, 2004.
[36] Y. Gao, A. Barreto, and M. Adjouadi, "Comparative Analysis of Noninvasively Monitored Biosignals for Affective Assessment of a Computer User," Proc. Southern Biomedical Eng. Conf., vol. 24, pp. 255-260, 2009.
[37] H.A. Broadbent and Y.A. Maksik, "Analysis of Periodic Data Using Walsh Functions," Behavior Research Methods, Instruments, & Computers, vol. 24, no. 2, pp. 238-247, 1992.
[38] M. Adjouadi, M. Cabrerizo, M. Ayala, D. Sanchez, I. Yaylali, P. Jayakar, and A. Barreto, "Interictal Spike Detection Using the Walsh Transform," IEEE Trans. Biomedical Eng., vol. 51, no. 5, pp. 868-873, May 2004.
[39] B. Meffert and O. Hochmuth, "The Application of the Walsh Transform in Biosignal Processing," Probl. Techn. Med., 1987.
[40] H. Larsen and D.C. Lai, "Walsh Spectral Estimates with Applications to the Classification of EEG Signals," IEEE Trans. Biomedical Eng., vol. 27, no. 9, pp. 485-492, Sept. 1980.
[41] T. Lin, M. Omata, W. Hu, and A. Imamiya, "Do Physiological Data Relate to Traditional Usability Indexes?" Proc. 17th Australia Conf. Computer-Human Interaction, pp. 1-10, 2005.
[42] M.M. Moore and U. Dua, "A Galvanic Skin Response Interface for People with Severe Motor Disabilities," Proc. ACM SIGACCESS Conf. Computers and Accessibility, pp. 48-84, 2004.
[43] W. Picard and J.A. Healey, "Wearable and Automotive Systems for Affect Recognition from Physiology," technical report, MIT, 2000.
[44] A. de Santos Sierra, C.S. Ávila, J.G. Casanova, and G. Bailador del Pozo, "A Stress-Detection System Based on Physiological Signals and Fuzzy Logic," IEEE Trans. Industrial Electronics, vol. 58, no. 10, pp. 4857-4865, Oct. 2011.
[45] D.C. Montgomery, Design and Analysis of Experiments, seventh ed. Wiley, July 2008.
[46] A. Field, Discovering Statistics Using SPSS, third ed. Sage Publications, Ltd, Jan. 2009.
[47] J. Chilo, G. Horvath, T. Lindblad, and R. Olsson, "Electronic Nose Ovarian Carcinoma Diagnosis Based on Machine Learning Algorithms," Proc. Industrial Conf. Advances in Data Mining. Applications and Theoretical Aspects, vol. 5633, pp. 13-23, 2009.
[48] I.H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, second ed. Morgan Kaufmann, June 2005.
[49] J.G. Cleary and L.E. Trigg, "${\rm K}^\ast$ : An Instance-Based Learner Using an Entropic Distance Measure," Proc. 12th Int'l Conf. Machine Learning, pp. 108-114, 1995.
[50] L. Fausett, Fundamentals of Neural Network. Prentice Hall, 1994.
[51] S. Theodoridis and K. Koutroumbas, Pattern Recognition, third ed. Academic Press, 2006.
[52] L. Breiman, "Random Forests," J. Machine Learning, vol. 45, pp. 5-32, 2001.
[53] W.W. Cohen, "Fast Effective Rule Induction," Proc. Int'l Conf. Machine Learning, pp. 115-123, 1995.
[54] J. Fürnkranz and G. Widmer, "Incremental Reduced Error Pruning," Proc. Int'l Conf. Machine Learning, pp. 70-77, 1994.
[55] B. Efron and R. Tibshirani, An Introduction to the Bootstrap. Chapman and Hall, 1993.
[56] R. Kohavi, "A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection," Proc. Int'l Joint Conf. Artificial Intelligence, p. 1137, 1995.
[57] J.A. Swets, Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers. Psychology Press, 1996.
[58] M. Sebag, J. Azé, and N. Lucas, "ROC-Based Evolutionary Learning: Application to Medical Data Mining," Proc. Artificial Evolution Conf., pp. 384-396, 2004.
[59] A. Barreto, J. Zhai, N. Rishe, and Y. Gao, "Significance of Pupil Diameter Measurements for the Assessment of Affective State in Computer Users," Advances and Innovations in Systems, Computing Sciences and Software Eng., pp. 59-64, Springer, 2007.
[60] G. Ridgway, "Receiver Operating Characteristic Curve with Convex Hull," MATLAB Central File Exchange, Apr. 2010.
[61] P. Patida, M. Gurta, S. Srivastava, and A.K. Nagawat, "Imaging Denoising by Various Filters for Different Noise," Int'l J. Computer Application, vol. 9, no. 4, pp. 45-50, Nov. 2010.
[62] M. Adjouadi and F. Candocia, "A Stereo Matching Paradigm Based on the Walsh Transformation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 12, pp. 1212-1219, Dec. 1994.
92 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool