The Community for Technology Leaders

Context-Aware Computing: Beyond Search and Location-Based Services

Pankaj Mehra, Whodini

Pages: pp. 12-16

Abstract—The holy grail of search is intent; of browsing, scent; and of classification, meaning. Each of these pillar technologies of the information age therefore depends critically on extracting and mining human context. Context awareness is the discipline that brings together technologies for extracting data about the larger human context and reasoning with that data at scale. It seeks to make the benefits of contextualization as pervasive in our lives as technology itself.

Keywords—computational and artificial intelligence, context awareness, pervasive computing, ubiquitous computing, context-aware services, information retrieval, information filtering, recommender systems

Context is the unstated actor in human communications, actions, and situations. It's the information halo that implicitly surrounds objects of interest, such as other people, places, events, things, information, and media (see Figure 1).


Figure 1   Context shapes human experience. As we increasingly perceive and begin to manipulate our world through a lens of information and information-based services, the quality of our interactions will depend crucially on the ability of our devices and services to extrapolate beyond our particular queries, utterances, or gestures to our broader intentions, needs, and commitment.

Context includes those pertinent supplementary facts, rules, or axioms whose consideration makes our communications efficient, our commands actionable, and our situations understandable by devices, people, or organizations seeking to provide us with content or services. Context data can be content-specific (provider-generated metadata), personal (user-generated preferences), or social (group-generated collaboration context). Any number of systems or devices along the path from provider to consumer — a user's GPS coordinates, search query logs, or an account's entire transaction history, for example — can produce context data.

Context of mention is derived mainly from content. Any entity mentioned in text can be surrounded by other words that provide context. For instance, the word "apple" might be mentioned in the context of either cooking or computing terms. Sometimes, therefore, we use context to disambiguate away polysemy, the existence of multiple word senses. Context of mention can likewise enable named entity resolution (NER). "The Sienna" can refer to a specific entity when used in my home context but to an entire family of cars when used in the automobile industry context. Likewise, one user's context might indicate it is a good car, even as another user's context indicates otherwise. In pulling together user reviews of the Sienna's features, characterizing and clustering such contexts of mention would save us from inappropriately averaging reliability and performance data, for instance, across mutually inconsistent data sources. But such uses of context data are trivial.

User context is everything that a user experiences, reads, and especially speaks or writes. It shapes how users perceive a message and interpret specific and often ambiguous words such as party or apple. It also determines how devices and services interpret user utterances and user-generated content. We can find interesting examples of such uses of context data in discourse and dialog engines, such as Apple's Siri, which uses the set of entities actually in the context (or via ontologies inferred to be in the context) to resolve anaphora — that is, to interpret context-denoting pronouns such as "her" or "that."

To work with user context, systems and services must go beyond exploiting location data from GPS coordinates or interpreting words in queries and documents. They must understand places. 1 They must understand what users know (such as their areas of interest), who they know, and who or what they trust (their social network and the social context within which they communicate and collaborate). These are what we call the dimensions of large context (see Figure 2).


Figure 2   Large context entities nest small context entities and drive preferences. An aura of latent and patent information surrounds the information objects that constitute a particular human situation or activity: Looking from right to left, the aura around an entity at first grows along the somewhat obvious dimensions of time and space, but then naturally extends through linkages provided in other dimensions of context by the social Web, the Semantic Web, and linked open data (see the article by Cory Hensen and colleagues in this issue).

Context-Aware Computing

Context-aware computing is an emerging system architecture trend targeting integrated — and subsequently interoperable — devices, databases, and services to comprehensively capture, represent, communicate, integrate, query, and further broker users' large context. Its advent is concomitant with the increased embedding of technology into our personal and social environments.

Context-aware computing offers billions of dollars per year in revenue opportunities for industries such as travel and retail. 2 Context awareness sharpens relevance when a device or service is responding to user-initiated actions (such as product search and support calls). It also enables proactive communications through pattern analytics over users' behavior and environment. Such proactive, contextualized delivery of information, alerts, and advertisements (including content and product recommendations) represents a significant commercial opportunity for all types of content and service providers. It stands to drive the experience of mobile Internet users well beyond simple searches for information, and even beyond current location-based services, which adapt to mere vestiges of our larger context.

Context Engines

Context is itself a type of data various operations can use to resolve ambiguous references or inconsistent facts, for example, or automate reasoning. In the mobile computing industry, GPS popularity has spawned many location-based services. Additionally, academia has a long tradition of treating context as a logical theory. Through big data technologies, these two disciplines are coming together to let us examine large context — namely, the billions of context clues provided by tens of millions of people in different situations and across different devices, over long periods of time, and in a large array of locations spanning vast distances, which are then integrated and brought to bear on a particular situation.

A context engine is an emerging class of computing platform designed to efficiently sift through trillions of context clues and generate, say, the fewer than 10 that it might pass on to a search engine or advertiser for situational advertising purposes, or to a content-recommendation service in order to achieve the greatest amount of situational appropriateness.

Challenges and Opportunities

Wouldn't you want to receive different restaurant recommendations on your wedding anniversary than on a family trip with young kids? Have you ever launched your PC vendor's automated help application only to have it start from the same script used for all customers, despite your extensive technical expertise? How would you feel about your shopping experience at Home Depot or Sears if they guided you to the right replacement filter for your refrigerator, knowing the brands you own? These scenarios underscore the importance of understanding the larger user context on the part of a mobile device, service, or business. Large context differs from much of the situational context capabilities being rolled out today in that it's a historically, socially, and semantically expanded model of a user's context encompassing product ownership, educational affiliation, sports team loyalties, locations visited and frequented, and so on. Leveraging large context not only enhances the relevance and personalization of delivered content, alerts, and advertisements, but also simplifies the interaction with users, thereby reducing time and costs.

Context's real power, and so far the area of its greatest unrealized potential, is in automated reasoning. We can view context as a theory — a collection of facts (entities and their interrelationships) and rules — describing the environment around a user or event. By lifting these facts and rules to a particular (sparsely described) situation, context enables inference. Inference is the foundation that makes devices and services look smart and capable of working with meager clues about users' intentions and preferences. Contextual inference will ultimately give us technology that can make even better recommendations, be they songs, product suggestions, or other search results.

The challenges we're facing and solving in realizing context-aware computing are concentrated around gathering, sharing, and obtaining data about large context. Apple and Google have implemented contextual computational advertising. Retailers are building context awareness into their online properties as well as retail kiosks. Hewlett-Packard, for instance, might be able to infer whether a user has arrived at in a support context or a shopping context — for the former, HP would recommend fixes and FAQs; for the latter, new models and discounted offers. Likewise, an eBay buyer who lost out on an auction could receive different recommendations than the person who won it — that is, information about other auctions in progress on related items versus recommendations for accessories for the item just purchased. Likewise, in-store retail kiosks are becoming place-aware. Place awareness also has other applications in mobile computing, be they pervasive intelligence built into our cars or in our handheld devices.

Context-aware computing broadly stands to revolutionize mobile experiences in which we increasingly look to communicate and accomplish a lot through interactions that have limited modalities. Context data enables parsimony in communication and interpretation; it supplements what's been said with what's been left unsaid. Place awareness might meaningfully disambiguate the terse utterance "upstairs;" large context awareness could interpret a command to "book the golf outing;" and awareness of work context and professional interests could yield recommendations regarding the most important person a user should meet this week, even before that user issues a search query or utters a command.

Context awareness in recommender systems has long been important to researchers and practitioners in personalized e-commerce, search engines, and marketing domains. However, existing approaches focus on query relevance and ignore context data, such as time, location, weather, or social dimensions. Ongoing research is aimed at modeling context data, user mapping and profiling, context query processing, and benchmarking of context-aware recommendation systems. One notable recent development in context-aware recommender systems is Facebook's Taste recommendation engine for its OpenGraph platform. This is the first example of an industry-scale context engine. It can handle billions of requests for context data per day from tens of millions of users and their hundreds of billions of context objects, through iterative self-guided query refinement, going from quick-and-dirty processing at large scale to deeper processing when recommending the final one to 10 data items.

In This Issue

Context-aware computing is moving from conceptual frameworks to systems and services that will soon be as pervasive as technology itself. The articles selected for this special issue show how context awareness is enabling Web and mobile services on the Internet to evolve beyond location-based search.

In "Augmenting Mobile Search Engines to Leverage Context Awareness," Elena Yndurain, Daniel Bernhardt, and Celeste Campo address the use of device context for enhancing mobile search queries. They introduce rule-based models of location awareness and context state inference (their approach to reducing the number of states in context data). They address contextualizing mobile search queries by adding context terms and present a simple but practical approach to determining whether the contextualization was beneficial, and ensuring that it did no harm if it was not.

"Semantic Perception: Converting Sensory Observations to Abstractions," by Cory Henson, Amit Sheth, and Krishnaprasad Thirunarayan, illustrates how to use abductive logic to derive context from observations. This article continues a long line of research from the late 1980s and early 1990s by R.V. Guha and John McCarthy, 35 and Fausto Giunchiglia and Luciano Serafini. 6,7 Here, the authors apply this research in the context of linked open data, a movement to build semantic linkages between items of information on the Internet. They focus on both sensed and derived context, covering techniques not unlike those used in early machine learning papers. This work is novel and timely in that contextual information is being socially sensed; this offers unique opportunities for setting up and validating symbolic hypotheses about what's happening at Internet scale, down to geographic and social locales. The authors use applications in real-time social sensing and contextualization of disease and weather events, and point to future benefits in real-time epidemiology and weather-emergency response.

Finally, "Bridging Communications and the Physical World," by Omer Boyaci, Victoria Beltran Martinez, and Henning Schulzrinne, describes a simple framework for writing contextualization rules using near-natural-language terminology of conditions and actions. The authors model multiple and diverse dimensions of context that are subject to real-time changes outside users' control, as well as location awareness and communication context. Sense Everything, Control Everything (SECE) is their Tcl-based framework aimed mainly at developers of context-aware applications for mobile phones.

Many commercial entities, ranging in scale from Rearden Commerce to Facebook and IBM, are beginning to make a business out of context awareness. It's clearly one of the most significant emerging applications of big data technologies. This issue gives a taste of the methods and applications of this field. Interested readers can find additional pointers about prior work in context-aware computing in my August 2010 guest editorial for IEEE Computing Now at


I thank Daniel Tunkelang of LinkedIn for his help in publicizing the call for papers and identifying reviewers for this special issue.


About the Authors

Pankaj Mehra is the CTO of Whodini. He was formerly a distinguished technologist at Hewlett-Packard. Contact him at
60 ms
(Ver 3.x)