This Article 
 Bibliographic References 
 Add to: 
A Nonsupervised Learning Framework of Human Behavior Patterns Based on Sequential Actions
April 2010 (vol. 22 no. 4)
pp. 479-492
Sang Wan Lee, IBM-KAIST Bio-computing Research Center, Korea Advanced Institute of Science and Technology, Daejeon
Yong Soo Kim, Daejeon University, Daejeon
Zeungnam Bien, Ulsan National Institute of Science and Technology, Ulsan
In designing autonomous service systems such as assistive robots for the aged and the disabled, discovery and prediction of human actions are important and often crucial. Patterns of human behavior, however, involve ambiguity, uncertainty, complexity, and inconsistency caused by physical, logical, and emotional factors, and thus their modeling and recognition are known to be difficult. In this paper, a nonsupervised learning framework of human behavior patterns is suggested in consideration of human behavioral characteristics. Our approach consists of two steps. In the first step, a meaningful structure of data is discovered by using Agglomerative Iterative Bayesian Fuzzy Clustering (AIBFC) with a newly proposed cluster validity index. In the second step, the sequence of actions is learned on the basis of the structure discovered in the first step and by utilizing the proposed Fuzzy-state Q--learning (FSQL) process. These two learning steps are incorporated in an amalgamated framework, AIBFC-FSQL, which is capable of learning human behavior patterns in a nonsupervised manner and predicting subsequent human actions. Through a number of simulations with typical benchmark data sets, we show that the proposed learning method outperforms several well-known methods. We further conduct experiments with two challenging real-world databases to demonstrate its usefulness from a practical perspective.

[1] M. Philipose et al., "Inferring Activities from Interactions with Objects," Proc. Pervasive Computing, pp. 50-57, 2004.
[2] D.H. Wilson, A.C. Long, and C. Atkeson, "A Context-Aware Recognition Survey for Data Collection Using Ubiquitous Sensors in the Home," Proc. Conf. Human Factors in Computing Systems (CHI): Late Breaking Results, 2005.
[3] D.H. Wilson and C. Atkeson, "Simultaneous Tracking and Activity Recognition (STAR) Using May Anonymous, Binary Sensors," Proc. Pervasive Computing, 2005.
[4] J.-H. Choi et al., "A Technology of Tracking Activities of the Aged for U-Healthcare," Korea Information Processing Soc. Rev., vol. 15, no. 1, pp. 34-43, Jan. 2008.
[5] S. Kubo et al., "Structural Equation Modeling for Comfort and Thermal Sensation," J. Japan Soc. for Fuzzy Theory and Intelligent Informatics, vol. 20, no. 2, pp. 164-170, Apr. 2008.
[6] M. Sasajima et al., "Toward Task-Oriented Mobile Internet Service Navigation—Ontology-Based User Modeling Method with Obstacles in Daily Life," J. Japan Soc. for Fuzzy Theory and Intelligent Informatics, vol. 20, no. 2, pp. 171-189, Apr. 2008.
[7] G. Kawakami et al., "Everyday Life Behavior Monitoring Based on Spatio-Temporal Expansion of Behavior Metrics Sensing Using Location-EMG Sensor," J. Japan Soc. for Fuzzy Theory and Intelligent Informatics, vol. 20, no. 2, pp. 190-200, Apr. 2008.
[8] T. Tajima et al., "Development of a Marketing System for Recognizing Customer Buying Behavior in a Store Using Ultrasound Sensor," J. Japan Soc. for Fuzzy Theory and Intelligent Informatics, vol. 20, no. 2, pp. 201-210, Apr. 2008.
[9] Z. Bien and M.-G. Chun, "A Fuzzy Petri Net Model," Handbook of Fuzzy Computation, C2.4, IOP Publishing Ltd., 1998.
[10] K. Reineke and A. Bernstein, "Predicting User Interface Preferences of Culturally Ambiguous Users," Proc. Conf. Human Factors in Computing Systems (CHI), 2008.
[11] C. Kidd et al., "The Aware Home: A Living Laboratory for Ubiquitous Computing Research," Proc. Second Int'l Workshop Cooperative Buildings, pp. 191-198, 1999.
[12] Smart Medical Home Research Laboratory, Univ. of Rochester, http://www.futurehealth.rochester.edusmart_home /, 2009.
[13] V. Stanford, "Using Pervasive Computing to Deliver Eldercare," IEEE Pervasive Comp., vol. 1, no. 1, pp. 10-13, Jan.-Mar. 2002.
[14] J. Porteus and S. Brownsell, Using Telecare: Exploring Technologies for Independent Living for Older People. Anchor Trust, 2000.
[15] J. Coughlin et al., "Old Age, New Technology and Future Innovations in Disease Management and Home Health Care," Home Healthcare Management and Practice, vol. 18, pp. 196-207, 2006.
[16] Z. Bien, "Learning System Techniques for Human-Friendly Man-Machine Interaction," Keynote Speech in the Proc. Second Int'l Conf. Artificial Intelligence in Eng. and Technology (iCAiET '04), Aug. 2004.
[17] S.W. Lee, D.-J. Kim, Y.S. Kim, and Z. Bien, "Adaptive Gabor Wavelet Neural Network for Facial Expression Recognition— Training of Feature Extractor by Novel Feature Separability Criterion," Proc. 11th World Congress of Int'l Fuzzy Systems Assoc., pp. 1309-1315, July 2005.
[18] H. Jang, J.-H. Do, J.-W. Jung, and Z.Z. Bien, "Two-Staged Hand-Posture Recognition Method for Softremocon System," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, Oct. 2005.
[19] J.-H. Do, H. Jang, S.H. Jung, J. Jung, and Z. Bien, "Soft Remote Control System in the Intelligent Sweet Home," Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems (IROS '05), pp. 2193-2198, Aug. 2005.
[20] Z.Z. Bien et al., "Steward Robot for Human-Friendly Assistive Home Environment," Promoting Independence for Older Persons with Disabilities, vol. 18, pp. 75-84, IOS Press, Jan. 2006.
[21] P.K. Simpson, "Fuzzy Min-Max Neural Networks - Part I: Classification," IEEE Trans. Neural Networks, vol. 3, no. 5, pp. 776-786, Sept. 1992.
[22] H. Hagras, F. Doctor, and A. Lopez, "An Incremental Adaptive Life Long Learning Approach for Type-2 Fuzzy Embedded Agents in Ambient Intelligent Environments," IEEE Trans. Fuzzy Systems, vol. 15, no. 1, pp. 41-55, Feb. 2007.
[23] H.-E. Lee, K.-H. Park, and Z.Z. Bien, "Iterative Fuzzy Clustering Algorithm with Supervision to Construct Probabilistic Fuzzy Rule Base from Numerical Data," IEEE Trans. Fuzzy Systems, vol. 16, no. 1, pp. 263-277, Feb. 2008.
[24] A.V. Petrovsky et al., Psychology. Prosveshenye, 1986.
[25] R. BurkhardtJr., Patterns of Behavior. Chicago Univ. Press, 2005.
[26] W. Lee, Decision Theory and Human Behavior. John Wiley & Sons, Inc., 1971.
[27] N.R. Pal, K. Pal, J.M. Keller, and J.C. Bezdek, "A Possibilistic Fuzzy c-Means Clustering Algorithm," IEEE Trans. Fuzzy Systems, vol. 13, no. 4, pp. 517-530, Aug. 2005.
[28] T. Huntsberger and P. Ajjimarangsee, "Parallel Self-Organizing Feature Maps for Unsupervised Pattern Recognition," Int'l J. General Systems, vol. 16, no. 14, pp. 357-372, 1990.
[29] E. Tsao, J. Bezdek, and N. Pal, "Fuzzy Kohonen Clustering Network," Pattern Recognition, vol. 27, no. 5, pp. 757-764, 1994.
[30] F.L. Chung and T. Lee, "Fuzzy Competitive Learning," Neural Networks, vol. 7, pp. 539-551, 1994.
[31] G. Carpenter, S. Grossberg, and D. Rosen, "Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System," Neural Networks, vol. 4, pp. 759-771, 1991.
[32] P. Simpson, "Fuzzy Min-Max Neural Networks—Part 2: Clustering," IEEE Trans. Fuzzy Systems, vol. 1, no. 1, pp. 32-45, Feb. 1993.
[33] N.B. Kraryiannis, "A Methodology for Constructing Fuzzy Algorithms for Learning Vector Quantization," IEEE Trans. Neural Networks, vol. 8, no. 3, pp. 505-518, May 1997.
[34] J.C.H.W. Christopher and D. Peter, "Q-Learning," Machine Learning, vol. 8, pp. 279-292, 1992.
[35] M. Yoichiro, "Modified Q-Learning Method with Fuzzy State Division and Adaptive Rewards," Proc. IEEE Int'l Conf. Fuzzy Systems, pp. 1556-1561, 2002.
[36] R.B. Hamid, "Fuzzy Q-Learning: A New Approach for Fuzzy Dynamic Programming," Proc. IEEE World Congress on Computational Intelligence, pp. 486-491, 1994.
[37] I.H. Suh, J.-H. Kim, and C.-H. Frank Rhee, "Fuzzy Q-Learning for Autonomous Robot Systems," Proc. IEEE Int'l Conf. Neural Network, pp. 1738-1743, 1997.
[38] S.W. Lee, Y.S. Kim, and Z. Bien, "Agglomerative Fuzzy Clustering Based on Bayesian Interpretation," Proc. IEEE Int'l Conf. Information Reuse and Integration, 2007.
[39] R.A. Johnson and D.W. Winchern, Applied Multivariate Statistical Analysis. Pearson Education, 2002.
[40] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc., vol. 39, no. 1, pp. 1-38, 1977.
[41] J.C. Bezdek, "Fuzziness vs. Probability—Again," IEEE Trans. Fuzzy Systems, vol. 2, no. 1, pp. 1-3, Feb. 1994.
[42] B. Kosko, "The Probability Monopoly," IEEE Trans. Fuzzy Systems, vol. 2, no. 1, pp. 32-33, Feb. 1994.
[43] C. Sutton and A. McCallum, "An Introduction to Conditional Random Fields for Relational Learning," Introduction to Statistical Relational Learning, MIT Press, 2007.
[44] H. Wallach, "Efficient Training of Conditional Random Fields," MSc thesis, Division of Informatics, Univ. of Edinburgh, 2002.
[45] T.G. Dietterich, "Machine Learning for Sequential Data: A Review," Structural, Syntactic, and Statistical Pattern Recognition, pp. 15-30, Springer-Verlag, 2002.
[46] J. Lafferty, A. McCallum, and F. Pereira, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data," Proc. Int'l Conf. Machine Learning, pp. 282-289, 2001.
[47] R. Duda et al., Pattern Classification. John Wiley & Sons, 2001.
[48] J.C. Bezdek and N.R. Pal, "Some New Indexes of Cluster Validity," IEEE Trans. Systems, Man, and Cybernetics-Part B: Cybernetics, vol. 28, no. 3, pp. 301-315, June 1998.
[49] X.L. Xie and G. Beni, "A Validity Measure for Fuzzy Clustering," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 8, pp. 841-847, Aug. 1991.
[50] S. Bandyopadhyay, "Simulated Annealing Using a Reversible Jump Markov Chain Monte Carlo Algorithm for Fuzzy Clustering," IEEE Trans. Knowledge and Data Eng., vol. 17, no. 4, pp. 479-490, Apr. 2005.
[51] M. Girolami, "Mercer Kernel-Based Clustering in Feature Space," IEEE Trans. Neural Networks, vol. 13, no. 3, pp. 780-784, May 2002.
[52] R. Sutton and A. Barto, Reinforcement Learning. MIT Press, 1998.
[53] K. Murphy, "HMM Toolbox for Matlab,"∼murphyk/Software/ HMMhmm.html, 2008.
[54] K. Murphy, "Kevin Murphy's MATLAB CRF Code," crf/, 2009.
[55] J. Han, W. Bang, and Z.Z. Bien, "Feature Set Extraction Algorithm Based on Soft Computing Techniques and Its Application to EMG Pattern Classification," Fuzzy Optimization and Decision Making, vol. 1, no. 1, pp. 269-286, 2002.
[56] M.A. Feki, S.W. Lee, M. Mokhtari, and Z. Bien, "Context Aware Life Pattern Prediction Using Fuzzy-State Q-Learning," Proc. Fifth Int'l Conf. Smart Home and Telecom., June, 2007.

Index Terms:
Fuzzy clustering, knowledge acquisition, learning, human behavior.
Sang Wan Lee, Yong Soo Kim, Zeungnam Bien, "A Nonsupervised Learning Framework of Human Behavior Patterns Based on Sequential Actions," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 4, pp. 479-492, April 2010, doi:10.1109/TKDE.2009.123
Usage of this product signifies your acceptance of the Terms of Use.