The Community for Technology Leaders
RSS Icon
Issue No.01 - Jan. (2014 vol.13)
pp: 61-74
Chuan Qin , Dept. of Comput. Sci. & Eng., Univ. of South Carolina, Columbia, SC, USA
Xuan Bao , Dept. of Electr. & Comput. Eng., Duke Univ., Durham, NC, USA
Romit Roy Choudhury , Dept. of Electr. & Comput. Eng., Duke Univ., Durham, NC, USA
Srihari Nelakuditi , Dept. of Comput. Sci. & Eng., Univ. of South Carolina, Columbia, SC, USA
Mobile phones are becoming the convergent platform for personal sensing, computing, and communication. This paper attempts to exploit this convergence toward the problem of automatic image tagging. We envision TagSense, a mobile phone-based collaborative system that senses the people, activity, and context in a picture, and merges them carefully to create tags on-the-fly. The main challenge pertains to discriminating phone users that are in the picture from those that are not. We deploy a prototype of TagSense on eight Android phones, and demonstrate its effectiveness through 200 pictures, taken in various social settings. While research in face recognition continues to improve image tagging, TagSense is an attempt to embrace additional dimensions of sensing toward this end goal. Performance comparison with Apple iPhoto and Google Picasa shows that such an out-of-band approach is valuable, especially with increasing device density and greater sophistication in sensing and learning algorithms.
Sensors, Accelerometers, Cameras, Compass, Tagging, Face recognition, Smart phones,context-awareness, Sensors, Accelerometers, Cameras, Compass, Tagging, Face recognition, Smart phones, activity recognition, Image tagging, face recognition, sensing, smartphone
Chuan Qin, Xuan Bao, Romit Roy Choudhury, Srihari Nelakuditi, "TagSense: Leveraging Smartphones for Automatic Image Tagging", IEEE Transactions on Mobile Computing, vol.13, no. 1, pp. 61-74, Jan. 2014, doi:10.1109/TMC.2012.235
[1] Amazon, "Amazon Mechanical Turk,", 2013.
[2] "Google Image Labeler,", 2011.
[3] L.V. Ahn and L. Dabbish, "Labeling Images with a Computer Game," Proc. ACM SIGCHI Conf. Human Factors Computing Systems, 2004.
[4] T. Yan et al., "CrowdSearch: Exploiting Crowds for Accurate Real-Time Image Search on Mobile Phones," Proc. Eighth Int'l Conf. Mobile Systems, Applications, and Services (MobiSys), 2010.
[5] T. Nakakura et al., "Neary: Conversation Field Detection Based on Similarity of Auditory Situation," Proc. ACM 10th Workshop Mobile Computing Systems and Applications (HotMobile), 2009.
[6] H. Lu et al., "SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones," Proc. ACM MobiSys, 2009.
[7] A. Engstrom et al., "Mobile Collaborative Live Video Mixing," Proc. Mobile Multimedia Workshop (with MobileHCI), Sept. 2008.
[8] "Google Goggles,", 2013.
[9] D.H. Hu et al., "Real World Activity Recognition with Multiple Goals," Proc. ACM 10th Int'l Conf. Ubiquitous Computing (UbiComp), 2008.
[10] "CyanogenMod," http:/, 2013.
[11] "WiFi Direct," , 2013.
[12] C. Liu, "Beyond Pixels: Exploring New Representations and Applications for Motion Analysis," doctoral thesis, MIT, 2009.
[13] E. Miluzzo et al., "Sensing Meets Mobile Social Networks: The Design, Implementation and Evaluation of CenceMe Application," Proc. ACM Int'l Conf. Embedded Networked Sensor Systems (Sensys), 2008.
[14] M. Azizyan et al., "SurroundSense: Mobile Phone Localization via Ambience Fingerprinting," Proc. ACM MobiCom, 2009.
[15] M. Braun and R. Spring, "Enkin," http:/, 2013.
[16] Monsoon Solutions Inc., http:/, 2013.
[17] A. Carroll and G. Heiser, "An Analysis of Power Consumption in a Smartphone," Proc. USENIX Conf. USENIX Ann. Technical Conf. (ATC), 2010.
[18] A.A. Sani et al., "Directional Antenna Diversity for Mobile Devices: Characterizations and Solutions," Proc. ACM MobiCom, 2010.
[19] C. Peng, G. Shen, Z. Han, Y. Zhang, Y. Li, and K. Tan, "A Beepbeep Ranging System on Mobile Phones," Proc. ACM Fifth Int'l Conf. Embedded Networked Sensor Systems (SenSys), 2007.
[20] "ALIPR (Automatic Photo Tagging and Visual Image Search)," http:/, 2013.
[21] M. Naaman et al., "Leveraging Context to Resolve Identity in Photo Albums," Proc. ACM/IEEE CS Fifth Joint Conf. Digital Libraries (JCDL), 2005.
[22] R. Sarvas et al., "Metadata Creation System for Mobile Images," Proc. ACM MobiSys, 2004.
[23] S.N. Patel et al., "The ContextCam: Automated Point of Capture Video Annotation," Proc. ACM Int'l Conf. Ubiquitous Computing (UbiComp), 2004.
[24] R. Want, "When Cell Phones Become Computers," IEEE Pervasive Computing, vol. 8, no. 2, pp. 2-5, Apr.-June 2009.
[25] P. Mohan et al., "Nericell: Rich Monitoring of Road and Traffic Conditions Using Mobile Smartphones," Proc. ACM Conf. Embedded Networked Sensor Systems (Sensys), 2008.
[26] L. Bao and S.S. Intille, "Activity Recognition from User-Annotated Acceleration Data," Proc. Second Int'l Conf. Pervasive Computing, 2004.
[27] T. van Kasteren et al., "Accurate Activity Recognition in a Home Setting," Proc. ACM 10th Int'l Conf. Ubiquitous Computing (UbiComp), 2008.
[28] M. Leo et al., "Complex Human Activity Recognition for Monitoring Wide Outdoor Environments," Proc. IEEE 17th Int'l Conf. Pattern Recognition (ICPR), 2004.
[29] B. Logan, "Mel Frequency Cepstral Coefficients for Music Modeling," Proc. Int'l Symp. Music Information Retrieval (ISMIR), 2000.
95 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool