Issue No.02 - April-June (2013 vol.20)
Published by the IEEE Computer Society
John R. Smith , IBM T.J. Watson Research Center
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/MMUL.2013.27
Growing multicamera, multiperspective image capture provides the opportunity to reconstruct and recount real-world events at a fine resolution.
When something happens today, anywhere in the world, it will probably be captured on camera. Very likely, it will be captured by many cameras. 1 Increasingly, it will be captured from many perspectives—by security cameras, overhead cameras, mobile users, and social media, as Figure 1 illustrates. This shift toward constant, multiperspective imaging will have a profound influence on how we chronicle and comprehend world events.
An event is a complex activity occurring within some spatio-temporal proximity involving interacting sets of people, actions, objects, locations, and subactivities that bring about some kind of transient or permanent change of state in the world. Examples include parades, sports games, concerts, and holiday celebrations. These types of events are cases when routine is broken, and the behavior of a group of people changes temporarily. Generally, events can occur at various space-time scales. For example, some specific newsworthy events, such as Occupy Wall Street, span many months over many locations, whereas Hurricane Sandy spanned a number of days over a more narrow area and the annual Times Square New Years' Eve celebration spans just hours at a specific location. Understanding events requires capturing the who, what, when, and where as well as determining what is happening over corresponding space and time scales.
Events involving a lot of people are becoming a new kind of social landmark that is dynamic and transient. As with traditional landmarks, everybody wants a picture, and association with a social landmark provides a nice memory or indicates status. And today, digital images are practically free to capture. In effect, citizens are acting as censors and contributors of large volumes of images and videos in social media. 2
Additionally, places where people gather can be of concern for safety and security reasons, which means growing numbers of surveillance cameras. Security is also moving beyond fixed ground-based cameras to include overhead capture from various platforms and moving cameras in vehicles or carried by personnel.
On a broad level, this offers us the possibility to capture and organize the world's events through imagery. We can observe patterns of life, recognize behaviors, and detect behavior changes. On a finer level, we can reconstruct and recount events with a resolution that is analogous to the synthesis of static real-world landmarks from photos. 3 If done live from thousands of cameras, it can provide a real-time operating picture of an event that can be used to drive alerts, such as for safety and security reasons, which can trigger specific immediate responses and actions, or provide an immersive experience for remote participants. Other the other hand, going back and analyzing reconstructed events for forensic purposes can help with investigations, provide education, or allow replay of collective social experiences.
The veil of mystery on events around the world is being lifted. No longer will we have to speculate what happens when a tree falls in the woods. More and more activities and events in life are being captured at high resolution from many cameras and from many perspectives. Privacy challenges and concerns will crop up for sure. But gone will be the shadows for criminals to lurk and act in the dark.
John R. Smith is a senior manager of Intelligent Information Management at IBM T.J. Watson Research Center. Contact him at firstname.lastname@example.org.