The Community for Technology Leaders

Labeling the World

Tim Kindberg, Matter 2 Media Ltd.
Thomas Pederson, IT University of Copenhagen
Rahul Sukthankar, Intel Labs Pittsburgh and Carnegie Mellon University

Pages: pp. 8-10

Approximately 10 years ago, the Cooltown project proposed an architecture for linking everyday physical objects to Web pages by tagging them with infrared beacons, RFID tags, and bar codes. Users carrying PDAs or other mobile devices could read those tags to view Web pages about the world around them. Recently, Google announced that it's dispatching approximately 200,000 stickers with bar codes for the windows of its "Favorite Places" in the US, so that people can use their smart phones to find out about them. The Japanese have been reading bar codes similar to these, called Quick Response (QR) codes, for more than five years. Besides such consumer-oriented uses, companies are attaching RFID tags to goods to help manage industrial supply chains.

In other words, developers are increasingly creating pervasive computing applications for the masses that aren't related to only GPS, navigation, and location. Smart phones with cameras and a variety of readers or sensors are in the hands of millions of people, and perhaps soon billions. Will there be billions—or trillions—of tags embedded in our everyday worlds before long? Will the tags employ bar codes, Near Field Communication, RFID, or some other technology? Who will embed them? To which content and service types will they link? Who will decide? And will image recognition soon replace physical tags so that people simply have to point their camera at an object instead of reading its tag?

Questions such as these—and the prospect of a world labeled to create a dynamic, exciting hybrid of the physical and the virtual—prompted us to put together this special issue of IEEE Pervasive Computing.

What's a Label?

A label is a bridge between the physical and virtual worlds that connects an object to services and applications. In most cases, labels are realized through tags—physical entities attached to or integrated with objects. For instance, an RFID tag lets you quickly and unambiguously distinguish a package of interest from hundreds that are virtually identical. An object's label simplifies its integration with applications and services and changes the pace of their evolution. For example, if an agent (human or artificial) wants to learn more about an object, a label can offer a shortcut to information that would otherwise be difficult or expensive to obtain at that instant. And increasingly, labels are used for play as well as for work—a label can just as easily facilitate a physical object's integration with an alternate-reality game.

Not all labels are realized through physical tags; some labeled objects can be algorithmically recognized. For example, if users' devices can accurately determine the position of a static object, such as a building, the object can be labeled by its position. Similarly, if image processing can unambiguously recognize the object, then no tag is required.

Labels aren't only for applications with a human in the loop. Machine-readable tags let computers better understand their physical environment while avoiding many challenges of noisy sensor-based perception. In particular, aspects of robot localization and object recognition become significantly easier when augmented by a few tags.

Labeling as a Process

When a label is associated with a physical object, the object owner, the label creator, the owner of the virtual resources to which that label links, and users who read the label are potential stakeholders. If the labeling process is open and anyone can link an object to information or content of his or her choosing, what tools are necessary? How can we create labels most conveniently, especially in situ? How best can different stakeholders' labels coexist on physical objects? Should people be able to delete or modify labels, including other people's labels? The labeling process gives rise to technological, cultural, social, and legal questions.

In This Issue

This special issue features new research in labeling the world, including enabling technology and innovative applications. These articles address topics ranging from improvements in tagging technology to robotics applications.

In "Memory Spot: A Labeling Technology," J.T. Edward McDonnell and his colleagues propose a new batteryless tagging technology that, unlike traditional RFID technologies, stores megabytes of information on the tag itself. Memory Spot can send or retrieve data at rates of up to 10 Mbits per second, enabling new usage models such as local video-content caching. Thus, Memory Spot complements existing labeling technology by augmenting links to Web content with a large amount of low-latency local storage.

In "Developing Mobile Workflow Support in the Internet of Things," Pau Giner and his colleagues propose a system for tailoring tagged objects' information to individuals' different interests. For example, when users read a library book's bar code, they obtain different results depending on whether they're librarians or borrowers—with those results based on the same underlying application.

Ann Light's article, "Bridging Global Divides with Tracking and Tracing Technology," also stresses stakeholders' different interests in a label-augmented value chain. However, she focuses on the sometimes problematic prerequisites for participating in these global business processes, especially for small producers. It becomes evident that political, market, and social factors can be at least as decisive as access to the labeling technology itself.

In "RFID-Guided Robots for Pervasive Automation," Travis Deyle and his colleagues describe RFID-guided robots for pervasive automation and propose using long-range tags to improve robots' interactions with people and objects. Although tagging every object of interest in the physical environment might be impractical in many settings, the proposed system could accelerate robot deployment in many real-world settings and complement other noisier forms of robot perception.

Finally, Scott Carter and his colleagues explore the design space of labeled paper documents in "Linking Digital Media to Physical Documents: Comparing Content- and Marker-Based Tags." In particular, they describe projects that use either explicit labels in tag form, such as bar codes, or implicit labels, such as text fragments that cameras can automatically recognize. They also discuss the two approaches' pros and cons.

Many more questions than answers exist about labeling the world, and the number of labels and communities using them is growing. To make matters more interesting, augmented reality (AR) is receiving renewed attention. AR is a much more difficult proposition than tagging because you must not only label physical objects but also ensure the content linked to them coincides with the objects in the visual plane in a manner that won't disappoint users' sensibilities. And yet, ideally AR should scale to as many objects as have labels. We look forward to the trillionth labeled object, with visuals layered precisely on top.

About the Authors

Bio Graphic
Tim Kindberg is founder and managing director of Matter 2 Media Ltd. His research interests are urban, pervasive, and distributed computing. Kindberg has a PhD in computer science from the University of Westminster. Contact him at
Bio Graphic
Thomas Pederson is an associate professor in the Innovative Communication Group at IT University of Copenhagen. His research interests include mobile mixed-reality interaction, tangible user interfaces, and pervasive computing. Pederson has a PhD in computing science from Umeå University. He's a member of IEEE and the ACM. Contact him at
Bio Graphic
Rahul Sukthankar is a senior principal researcher at Intel Labs Pittsburgh and an adjunct research professor at Carnegie Mellon University's Robotics Institute. His research interests include computer vision, machine learning, and information retrieval. Sukthankar has a PhD in robotics from Carnegie Mellon. He's a member of IEEE and the ACM. Contact him at
57 ms
(Ver 3.x)