The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - September (2011 vol.23)
pp: 1359-1372
Liang Wang , Nanjing University, Nanjing
Zhanqing Wu , Nanjing University, Nanjing
Xianping Tao , Nanjing University, Nanjing
Tao Gu , University of Southern Denmark, Odense
ABSTRACT
Recognizing human activities from sensor readings has recently attracted much research interest in pervasive computing due to its potential in many applications, such as assistive living and healthcare. This task is particularly challenging because human activities are often performed in not only a simple (i.e., sequential), but also a complex (i.e., interleaved or concurrent) manner in real life. Little work has been done in addressing complex issues in such a situation. The existing models of interleaved and concurrent activities are typically learning-based. Such models lack of flexibility in real life because activities can be interleaved and performed concurrently in many different ways. In this paper, we propose a novel pattern mining approach to recognize sequential, interleaved, and concurrent activities in a unified framework. We exploit Emerging Pattern—a discriminative pattern that describes significant changes between classes of data—to identify sensor features for classifying activities. Different from existing learning-based approaches which require different training data sets for building activity models, our activity models are built upon the sequential activity trace only and can be applied to recognize both simple and complex activities. We conduct our empirical studies by collecting real-world traces, evaluating the performance of our algorithm, and comparing our algorithm with static and temporal models. Our results demonstrate that, with a time slice of 15 seconds, we achieve an accuracy of 90.96 percent for sequential activity, 88.1 percent for interleaved activity, and 82.53 percent for concurrent activity.
INDEX TERMS
Human activity recognition, pattern analysis, emerging pattern, classifier design and evaluation.
CITATION
Liang Wang, Zhanqing Wu, Xianping Tao, Tao Gu, "A Pattern Mining Approach to Sensor-Based Human Activity Recognition", IEEE Transactions on Knowledge & Data Engineering, vol.23, no. 9, pp. 1359-1372, September 2011, doi:10.1109/TKDE.2010.184
REFERENCES
[1] S. Katz, A.B. Ford, R.W. Moskowitz, B.A. Jackson, and M.W. Jaffe, "Studies of Illness in the Aged. The Index of ADL: A Standardized Measure of Biological and Psychological Function," J. Am. Medical Assoc., vol. 185, pp. 914-919, Sept. 1963.
[2] L. Bao and S.S. Intille, "Activity Recognition from User-Annotated Acceleration Data," Proc. Second Int'l Conf. Pervasive Computing (PERVASIVE '04), pp. 1-17, 2004.
[3] E.M. Tapia, S.S. Intille, and K. Larson, "Activity Recognition in the Home Setting Using Simple and Ubiquitous Sensors," Proc. Second Int'l Conf. Pervasive Computing (PERVASIVE '04), pp. 158-175, 2004.
[4] B. Logan, J. Healey, M. Philipose, E.M. Tapia, and S.S. Intille, "A Long-Term Evaluation of Sensing Modalities for Activity Recognition," Proc. Ninth Int'l Conf. Ubiquitous Computing (UbiComp), Sept. 2007.
[5] T. Huynh, U. Blanke, and B. Schiele, "Scalable Recognition of Daily Activities from Wearable Sensors," Proc. Int'l Symp. Location and Context-Awareness (LoCA), Sept. 2007.
[6] M. Stikic, T. Huynh, K. Van-Laerhoven, and B. Schiele, "ADL Recognition Based on the Combination of RFID and Accelerometer Sensing," Proc. Int'l Conf. Pervasive Computing Technologies for Healthcare, 2008.
[7] Y. Nakauchi, K. Noguchi, P. Somwong, and T. Matsubara, "Human Intention Detection and Activity Support System for Ubiquitous Sensor Room," J. Robotics and Mechatronics, vol. 16, no. 5, pp. 545-551, 2004.
[8] C. Lombriser, N.B. Bharatula, D. Roggen, and G. Tröster, "On-Body Activity Recognition in a Dynamic Sensor Network," Proc. Int'l Conf. Body Area Networks (BodyNets), 2007.
[9] J.B.J. Bussmann, W.L.J. Martens, J.H.M. Tulen, F. Schasfoort, H.J.G. van den Berg-Emons, and H. Stam, "Measuring Daily Behavior Using Ambulatory Accelerometry: The Activity Monitor," Behavior Research Methods, Instruments, and Computers, vol. 33, no. 3, pp. 349-356, 2001.
[10] M. Philipose, K.P. Fishkin, M. Perkowitz, D.J. Patterson, D. Fox, H. Kautz, and D. Hähnel, "Inferring Activities from Interactions with Objects," IEEE Pervasive Computing, vol. 3, no. 4, pp. 50-57, Oct. 2004.
[11] D. Wilson and C. Atkeson, "Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors," Proc. Int'l Conf. Pervasive Computing, pp. 62-79, 2005.
[12] J. Lester, T. Choudhury, and G. Borriello, "A Practical Approach to Recognizing Physical Activities," Proc. Int'l Conf. Pervasive Computing, 2006.
[13] J.A. Ward, P. Lukowicz, G. Tröster, and T.E. Starner, "Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1553-1567, Oct. 2006.
[14] D. Patterson, D. Fox, H. Kautz, and M. Philipose, "Fine-Grained Activity Recognition by Aggregating Abstract Object Usage," Proc. IEEE Int'l Symp. Wearable Computers, Oct. 2005.
[15] D. Wyatt, M. Philipose, and T. Choudhury, "Unsupervised Activity Recognition Using Automatically Mined Common Sense," Proc. Am. Assoc. for Artificial Intelligence (AAAI) Conf., July 2005.
[16] W. Pentney, A.M. Popescu, S. Wang, H. Kautz, and M. Philipose, "Sensor-Based Understanding of Daily Life via Large-Scale Use of Common Sense," Proc. Am. Assoc. for Artificial Intelligence (AAAI) Conf., July 2006.
[17] S. Wang, W. Pentney, A.M. Popescu, T. Choudhury, and M. Philipose, "Common Sense Based Joint Training of Human Activity Recognizers," Proc. Int'l Joint Conf. Artificial Intelligence, Jan. 2007.
[18] D.L. Vail, M.M. Veloso, and J.D. Lafferty, "Conditional Random Fields for Activity Recognition," Proc. Int'l Conf. Autonomous Agents and Multi-Agent Systems (AAMAS), 2007.
[19] T.Y. Wu, C.C. Lian, and J.Y. Hsu, "Joint Recognition of Multiple Concurrent Activities Using Factorial Conditional Random Fields," Proc. Am. Assoc. for Artificial Intelligence (AAAI) Workshop Plan, Activity, and Intent Recognition, July 2007.
[20] T.L.M. van Kasteren, A.K. Noulas, G. Englebienne, and B.J.A. Kröse, "Accurate Activity Recognition in a Home Setting," Proc. 10th Int'l Conf. Ubiquitous Computing (UbiComp), Sept. 2008.
[21] D.H. Hu and Q. Yang, "CIGAR: Concurrent and Interleaving Goal and Activity Recognition," Proc. Am. Assoc. for Artificial Intelligence (AAAI) Conf., 2008.
[22] D.H. Hu, S.J. Pan, V.W. Zheng, N.N. Liu, and Q. Yang, "Real World Activity Recognition with Multiple Goals," Proc. 10th Int'l Conf. Ubiquitous Computing (UbiComp), Sept. 2008.
[23] J. Modayil, T.X. Bai, and H. Kautz, "Improving the Recognition of Interleaved Activities Research Note," Proc. 10th Int'l Conf. Ubiquitous Computing (UbiComp), Sept. 2008.
[24] T. Huynh, M. Fritz, and B. Schiele, "Discovery of Activity Patterns Using Topic Models," Proc. 10th Int'l Conf. Ubiquitous Computing (UbiComp), Sept. 2008.
[25] R. Hamid, S. Maddi, A. Bobick, and I. Essa, "Unsupervised Analysis of Activity Sequences Using Event Motifs," Proc. ACM Int'l Workshop Video Surveillance and Sensor, 2006.
[26] R. Hamid, S. Maddi, A. Johnson, A. Bobick, I. Essa, and C. Isbell, "A Novel Sequence Representation for Unsupervised Analysis of Human Activities," Artificial Intelligence, vol. 173, no. 14, pp. 1221-1244, 2009.
[27] S.S. Intille, K. Larson, E.M. Tapia, J. Beaudin, P. Kaushik, J. Nawyn, and R. Rockinson, "Using a Live-In Laboratory for Ubiquitous Computing Research," Proc. Int'l Conf. Pervasive Computing, pp. 349-365, 2006.
[28] Y. Yacoob and M.J. Black, "Parameterized Modeling and Recognition of Activities," Proc. IEEE Int'l Conf. Computer Vision, 1998.
[29] D. Moore, I. Essa, and M. Hayes, "Exploiting Human Actions and Object Context for Recognition Tasks," Proc. IEEE Int'l Conf. Computer Vision, 1999.
[30] Y.A. Ivanov and A.F. Bobick, "Recognition of Visual Activities and Interactions by Stochastic Parsing," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 852-872, Aug. 2000.
[31] I. Haritaoglu, D. Harwood, and L.S. Davis, "W4: Real-Time Surveillance of People and Their Activities," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 809-830, Aug. 2000.
[32] J. Yamato, J. Ohya, and K. Ishii, "Recognizing Human Action in Time-Sequential Images Using Hidden Markov Model," Proc. Int'l IEEE CS Conf. Computer Vision and Pattern Recognition, 1992.
[33] N. Oliver, A. Garg, and E. Horvitz, "Layered Representations for Learning and Inferring Office Activity from Multiple Sensory Channels," Computer Vision and Image Understanding, vol. 96, no. 2, pp. 163-180, 2004.
[34] X.D. Sun, C.W. Chen, and B.S. Manjunath, "Probabilistic Motion Parameter Models for Human Activity Recognition," Proc. IEEE Int'l Conf. Pattern Recognition, 2002.
[35] J.B. Arie, Z.Q. Wang, P. Pandit, and S. Rajaram, "Human Activity Recognition Using Multidimensional Indexing," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1091-1104, Aug. 2002.
[36] M. Brand and V. Kettnaker, "Discovery and Segmentation of Activities in Video," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 844-851, Aug. 2000.
[37] N. Robertson and I. Reid, "A General Method for Human Activity Recognition in Video," Computer Vision and Image Understanding, vol. 104, no. 2, pp. 232-248, 2006.
[38] F. Fusier, V. Valentin, F. Brémond, M. Thonnat, M. Borg, D. Thirde, and J. Ferryman, "Video Understanding for Complex Activity Recognition," Machine Vision and Applications, vol. 18, nos. 3/4, pp. 167-188, Aug. 2007.
[39] T. Huang, D. Koller, J. Malik, G.H. Ogasawara, B. Rao, S.J. Russell, and J. Weber, "Automatic Symbolic Traffic Scene Analysis Using Belief Networks," Proc. Nat'l Conf. Artificial Intelligence, 1994.
[40] A.A. Efros, A.C. Berg, G. Mori, and J. Malik, "Recognizing Action at a Distance," Proc. IEEE Int'l Conf. Computer Vision, pp. 726-733, 2003.
[41] E. Shechtman and M. Irani, "Space-Time Behavior Based Correlation," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 405-412, 2005.
[42] H. Kautz, "A Formal Theory of Plan Recognition," PhD dissertation, Univ. of Rochester, 1987.
[43] G.Z. Dong and J.Y. Li, "Efficient Mining of Emerging Patterns: Discovering Trends and Differences," Proc. ACM Int'l Conf. Knowledge Discovery and Data Mining, pp. 43-52, Aug. 1999.
[44] G.Z. Dong, X. Zhang, L. Wong, and J.Y. Li, "CAEP: Classification by Aggregating Emerging Patterns," Proc. Second Int'l Conf. Discovery Science, Dec. 1999.
[45] J.Y. Li, H.Q. Liu, J.R. Downing, A.E. Yeoh, and L. Wong, "Simple Rules Underlying Gene Expression Profiles of More Than Six Subtypes of Acute Lymphoblastic Leukemia (ALL) Patients," Bioinformatics, vol. 19, no. 1, p. 71-78, 2003.
[46] J.Y. Li, H.Q. Liu, S.K. Ng, and L. Wong, "Discovery of Significant Rules for Classifying Cancer Diagnosis Data," Bioinformatics, vol. 19, pp. ii93-ii102, 2003.
[47] J.Y. Li, G.M. Liu, and L. Wong, "Mining Statistically Important Equivalence Classes and Delta-Discriminative Emerging Patterns," Proc. ACM Int'l Conf. Knowledge Discovery and Data Mining, pp. 430-439, 2007.
[48] U. Fayyad and K. Irani, "Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning," Proc. Int'l Joint Conf. Artificial Intelligence, 1993.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool