| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
Viz-A-Vis: Toward Visualizing Video through Computer Vision
November/December 2008 (vol. 14 no. 6)
pp. 1261-1268
In the established procedural model of information visualization, the first operation is to transform raw data into data tables [1]. The transforms typically include abstractions that aggregate and segment relevant data and are usually defined by a human, user or programmer. The theme of this paper is that for video, data transforms should be supported by low level computer vision. High level reasoning still resides in the human analyst, while part of the low level perception is handled by the computer. To illustrate this approach, we present Viz-A-Vis, an overhead video capture and access system for activity analysis in natural settings over variable periods of time. Overhead video provides rich opportunities for long-term behavioral and occupancy analysis, but it poses considerable challenges. We present initial steps addressing two challenges. First, overhead video generates overwhelmingly large volumes of video impractical to analyze manually. Second, automatic video analysis remains an open problem for computer vision.
[1] S. Card, J. Mackinlay, and B. Shneiderman, Readings in information visualization: using vision to think. San Francisco, Calif.: Morgan Kaufmann Publishers, 1999.
[2] M. Pantic, A. Pentland, A. Nijholt, and T. Huang, "Human Computing and Machine Understanding of Human Behavior: A Survey," in Artifical Intelligence for Human Computing, vol. 4451/2007, Lecture Notes in Computer Science: Springer Berlin / Heidelberg, 2007, pp. 47–71.
[3] C. Marshall and G. B. Rossman, Designing Qualitative Research, Fourth ed. Thousand Oaks: Sage Publications, 2006.
[4] K. Miller, Principles of Everyday Behavior Analysis, Fourth ed: Wadsworth Publishing, 2005.
[5] S. R. Barley, "Images of Imaging: Notes on Doing Longitudinal Field Work," Organization Science. Special Issue: Longitudinal Field Research Methods for Studying Processes of Organizational Change, vol. 1, pp. 220–247, 1990.
[6] B. T. Truong and S. Venkatesh, "Video abstraction: A systematic review and classification," ACM Trans. Multimedia Comput. Commun. Appl., vol. 3, pp. 3, 2007.
[7] P. Stéphane, G. Pierre, and P. Justin, "Prototyping of interactive satellite image analysis tools using a real-time data-flow computer," in Image Analysis and Processing, vol. 974/1995, Lecture Notes in Computer Science, 1996, pp. 683–688.
[8] A. Wilson, "Robust Vision-Based Detection of Pinching for One and Two-Handed Gesture Input," presented at UIST, 2006.
[9] K. Truong, G. Abowd, and J. Brotherton, "Who, What, When, Where, How: Design Issues of Capture and Access Applications," in Proceedings of the 3rd international conference on Ubiquitous Computing. Atlanta, Georgia, USA: Springer-Verlag, 2001.
[10] J. Hare, P. Lewis, P. Enser, and C. Sandom, "Mind the Gap: Another look at the problem of the semantic gap in image retrieval," presented at Multimedia Content Analysis, Management and Retrieval 2006, San Jose, California, 2006.
[11] G. Hayes, "Documenting and understanding everyday activities through the selective archiving of live experiences," in CHI '06 extended abstracts on Human factors in computing systems. Montreal, Quebec, Canada: ACM, 2006.
[12] C. Neustaedter, S. Greenberg, and M. Boyle, "Blur filtration fails to preserve privacy for home-based video conferencing," ACM Trans. Comput.-Hum. Interact., vol. 13, pp. 1–36, 2006.
[13] V. Bellotti and A. Sellen, "Design for Privacy in Ubiquitous Computing Environments," presented at Proceedings of the Third European Conference on Computer Supported Cooperative Work ({ECSCW}'93), 1993.
[14] G. Iachello and G. Abowd, "Privacy and proportionality: adapting legal evaluation techniques to inform design in ubiquitous computing," in Proceedings of the SIGCHI conference on Human factors in computing systems. Portland, Oregon, USA: ACM, 2005.
[15] Y. Ivanov, C. Wren, A. Sorokin, and I. Kaur, " Visualizing the History of Living Spaces," IEEE Transactions on Visualization and Computer Graphics, vol. 13, pp. 1153–1160, 2007.
[16] T. Kapler and W. Wright, "GeoTime Information Visualization," presented at Information Visualization, INFOVIS 2004, Austin, Texas, 2004.
[17] M. P. Kwan and J. Lee, "Geovisualization of human activity patterns using 3D GIS: a time-geographic approach," in Spatially Integrated Social Science: Examples in Best Practice, M. F. Goodchild and D. G. Janelle, Eds. New York: Oxford University Press, 2004, pp. 48–66.
[18] G. Daniel and M. Chen, "Video Visualization," in Proceedings of the 14th IEEE Visualization 2003 (VIS'03): IEEE Computer Society, 2003.
[19] A. W. Klein, P.-P. J. Sloan, A. Finkelstein, and M. F. Cohen, "Stylized video cubes," in Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation. San Antonio, Texas: ACM, 2002.
[20] S. Fels, E. Lee, and K. Mase, "Techniques for interactive video cubism (poster session)," in Proceedings of the eighth ACM international conference on Multimedia. Marina del Rey, California, United States: ACM, 2000.
[21] E. P. Bennett and L. McMillan, "Proscenium: a framework for spatio-temporal video editing," in Proceedings of the eleventh ACM international conference on Multimedia. Berkeley, CA, USA: ACM, 2003.
[22] M. Terry, G. J. Brostow, G. Ou, J. Tyman, and D. Gromala, "Making space for time in time-lapse photography," in ACM SIGGRAPH 2004 Sketches. Los Angeles, California: ACM, 2004.
[23] S. Kiranyaz, K. Caglar, E. Guldogan, O. Guldogan, and M. Gabbouj, "MUVIS: a content-based multimedia indexing and retrieval framework," presented at Seventh International Symposium on Signal Processing and its Applications, ISSPA 2003, Paris, France, 2003.
[24] A. Bobick, "Movement, Activity and Action: the Role of Knowledge in the Perception of Motion," Royal Society Workshop on Knowledge-based Vision in Man and Machine, vol. 352, pp. 1257–1265, 1997.
[25] M. Sonka, V. Hlavec, and R. Boyle, Image Processing Analysis, and Machine Vision: PWS Publishing, 1999.
[26] M. Romero, Z. Pousman, and M. Mateas, "Alien presence in the home: the design of Tableau Machine," Personal and Ubiquitous Computing, vol. 12, 2007.
[27] C. Kidd, R. Orr, A. G., A. C., I. Essa, B. MacIntyre, E. Mynatt, T. Starner, and W. Newstetter, "The Aware Home: A Living Laboratory for Ubiquitous Computing Research," presented at Proceedings of the Second International Workshop on Cooperative Buildings - CoBuild'99, 1999.
[28] A. Bobick and J. Davis, "Real-time recognition of activity using temporal templates," presented at 3rd IEEE Workshop on Applications of Computer Vision (WACV '96), 1996.
[29] J. Summet, M. Flagg, T. Cham, J. Rehg, and R. Sukthankar, "Shadow elimination and blinding light suppression for interactive projected displays," IEEE Transactions on Visualization & Computer Graphics (TVCG), vol. 13, pp. 508–17, 2007.
Index Terms:
Index Terms—Spatiotemporal visualization, time series data, video visualization, sensor analytics, image/video analytics
Citation:
Mario Romero, Jay Summet, John Stasko, Gregory Abowd, "Viz-A-Vis: Toward Visualizing Video through Computer Vision," IEEE Transactions on Visualization and Computer Graphics, vol. 14, no. 6, pp. 1261-1268, Nov./Dec. 2008, doi:10.1109/TVCG.2008.185
