The Community for Technology Leaders

Multimedia Analytics

Nancy A. Chinchor, ChinchorEclectic
Michael G. Christel, Carnegie Mellon University
William Ribarsky, University of North Carolina at Charlotte

Pages: pp. 18-19

Last October at IEEE VisWeek 09, we held a workshop on video analytics. We wanted to bring together the visual-analytics and video analysis communities to determine the amount of synergy between the two fields. The workshop left everyone feeling that the two fields had much to offer each other. For this issue, we broadened the topic to multimedia analytics to include more researchers in multimedia analysis and visual analytics. The resulting articles reflect the range and quality of research that serves as a foundation for future multimedia analytics research.

"Spatial Navigation for Context-Aware Video Surveillance," by Gerwin de Haan and his colleagues, details a technique for following events in surveillance videos. The current practice in monitoring surveillance videos involves passive viewing of an array of video displays, maps, and indirect controls. As the number of cameras increases, it's necessary to improve context awareness because of information overload. Their "egocentric" technique provides this situational awareness. Live tracking of complex events along cameras requires rapid, accurate navigation decisions about which camera to follow. The authors' egocentric technique overcomes this by letting viewers directly navigate in the visible video, using the mouse, and providing updated visual feedback on the available camera transitions. This lets the observer be part of the action while following it. This new technique eliminates the boredom and inattention inherent in current passive technologies and makes the monitor's task much more efficient.

"Newdle: Interactive Visual Exploration of Large Online News Collections," by Jing Yang and her colleagues, presents a novel visual-analytic system for exploring tagged online news collections. Newdle automatically constructs underlying article and tag-article networks, clusters, and path analyses. It essentially adds a temporal aspect to Wordle. Users can grasp the content of a large news collection at a glance through a topic overview that meaningfully extracts and intuitively displays the semantics and temporal features of article clusters. Users can also conduct in-depth analyses on topics, tags, and articles of interest by drilling down through the initial displays and seeing more of those articles.

In "MediaTable: Interactive Categorization of Multimedia Collections," Ork de Rooij and his colleagues provide an aid for categorizing unknown collections of images or videos. Generally, multimedia collections have little useful metadata and are unannotated. Actually finding what you're looking for is impossible. The problem of determining what to instruct people to tag and how to be certain that they're tagging consistently across many videos is difficult and time-consuming. Many large collections are so spottily annotated that people can't be sure they're retrieving what they need. MediaTable helps improve categorization by providing a tabular interface that gives an overview of multimedia items, associated metadata, and a bucket list with which users can quickly categorize data. It uses familiar techniques for sorting, filtering, selecting, and visualizing. Results indicate that MediaTable yields efficient categorization and valuable insight into a collection. We feel that MediaTable is a large step toward enabling annotation of collections of images and videos.

At the IEEE VisWeek09 workshop, attendees also expressed great interest in further discussion of the foundational research for multimedia analytics and some examples. They thought a tutorial including the state of the art in video analysis and visual analytics would help them see the synergies between the two areas. To meet that request, this issue includes "Multimedia Analysis + Visual Analytics = Multimedia Analytics." This tutorial addresses more than what the VisWeek workshop participants requested, but it also provides a stronger foundation for the science.

These three articles and the tutorial illustrate the use of results from information visualization and the application of visual-analytics principles to exploit multiple types of media. Each article evaluates its presented techniques through studies of their utility for analysis. Furthermore, the systems' developers respond to pre-evaluation user comments to improve the technologies before rigorous testing. Testing includes both effectiveness and efficiency measures. The authors also demonstrate the systems on multiple datasets.

We're pleased to present such mature papers in this initial offering in the combination of multimedia analysis and visual analytics to form multimedia analytics. The future holds much promise if scientists maintain this quality of research while furthering the science of analysis by applying techniques to multiple types of media.

About the Authors

Nancy A. Chinchor is founder of ChinchorEclectic. Her research interests are analytics of visual media and multimedia. Chinchor has a PhD in linguistics from Brown University. Contact her at
Michael G. Christel is a research professor at Carnegie Mellon University's Entertainment Technology Center. His research interests are edutainment, digital libraries, human-computer interaction, and multimedia analytics. Christel has a PhD in computer science from Georgia Tech. Contact him at
William Ribarsky is the Bank of America Endowed Chair in Information Technology and the chair of the Computer Science Department at the University of North Carolina at Charlotte. His research interests are visual analytics, 3D multimodal interaction, bioinformatics visualization, virtual environments, visual reasoning, and interactive visualization of large-scale information spaces. Ribarsky has a PhD in physics from the University of Cincinnati. He's on the IEEE Computer Graphics and Applications editorial board. Contact him at
52 ms
(Ver 3.x)