by Kari Pulli

September 2008—Mobile Graphics

Kari Pulli

Mobile phones form a ubiquitous graphics platform; today, over half of the world's population uses mobile phones. New devices have significantly richer graphics, imaging, processing, interfacing, and networking capabilities than their antecedents, enabling many visual applications that used to be possible only on desktop computers. Designing handheld devices is challenging for several reasons. Owing to battery and heat dissipation concerns, power is limited, and form factor limitations constrain the physical size of input devices and graphical displays. However, the fact that the devices are always with you, always on, and always connected—and can find out so much of the user's history, preferences, and context—enables many new types of applications.

With graphics acceleration, the devices get more responsive user interfaces that are fun to use and can support action 3D games that users can play against other players when they have a free moment outside of home or office. Camera phones are more numerous than all traditional cameras put together. They're beginning to provide very high-quality images that can be annotated and uploaded immediately, and combined and processed directly in the device instead of later at a work station. Combined with downloadable maps, positioning through GPS and other sensors provides the best routes to the target location and information about what interesting places may be nearby. Combining all of the above makes the modern smart phone an ideal platform for augmented reality. The device knows where it's located and what it's looking at, and it annotates the camera image to provide a magic lens that combines the real world and the digital world.

This issue of Computing Now provides a selection of recent articles about mobile graphics from several IEEE Computer Society magazines. The articles highlight the technical challenges of providing real-time graphics on mobile devices, as well as the novel applications the new platform enables. Three of the articles come from the July–August 2008 special issue of IEEE Computer Graphics and Applications on Mobile Graphics (edited by Scott Klemmer and myself).

Mobile graphics is moving to the mainstream. It promises to revolutionize the accessibility of graphics to new users as the transition from high-end engineering graphics workstations to PC graphics did.

Kari Pulli is a Research Fellow at Nokia Research Center in Palo Alto, where he heads a team working on Visual Computing and Ubiquitous Imaging. Contact him at firstname dot lastname at nokia dot com.



Mobile 3D City Maps

By Antti Nurminen
From the July/August 2008 issue of IEEE Computer Graphics and Applications

Spatial information has been conveyed in different forms throughout human history. Map-like media varied from stories with geographic depictions to figurative paintings—until knowledge and technology advanced to the point where appropriate survey and positioning methods came into use, facilitating metrically accurate maps. While early maps were heavily influenced by the surrounding culture and religion, and suffered from both intentional and unintentional errors in scale, they usually contained rather accurately drawn landmarks. The more scientific approach for mapmaking in the Renaissance yielded increasingly accurate maps, without sacrificing their beauty, and facilitating exquisite detail. As an example, in the early 17th century the cartographer family Blaeu created a complete city catalog, Theater of Cities, which depicted Dutch cities from a bird's-eye view in 3D, in a realistic manner (see Figure 1). Later, the artistic side of cartography diminished, as surveying became the dominant part. Simultaneously, the readability of maps was increased by simplification and abstraction.

Toward Next-Gen Mobile AR Games

By Wolfgang Broll, Irma Lindt, Iris Herbst, Jan Ohlenburg, Anne-Kathrin Braun, and Richard Wetzel
From the July/August 2008 issue of IEEE Computer Graphics and Applications

Digital games employ digital technology as part of the game experience. In contrast to noncomputer games, digital games "facilitate the play of games that would be too complicated in a noncomputerized context." Game developers can create rich, interactive virtual worlds that real people can inhabit. Through digital technology and the possibility of representing complex behavior, computer games can deliver new and intense experiences to players.

The State of the Art in Mobile Graphics Research

By Tolga Capin, Kari Pulli, and Tomas Akenine-Möller
From the July/August 2008 issue of IEEE Computer Graphics and Applications

Mobile phones are virtually omnipresent. In 2008, 3.3 billion people—half the world population—use mobile phones, according to the International Telecommunications Union. By 2010, Nokia expects that there will be as many mobile phone users as toothbrush users (4 billion). Over the past 10 years, the phone has expanded from being just a phone to being a full multimedia unit, on which you can play games (see Figure 1), shoot photos, listen to music, watch television or video, send messages, and do videoconferencing.

Camera Phones: A Snapshot of Research and Applications

By Franklin Reynolds
From the April–June 2008 issue of IEEE Pervasive Computing

Once considered science fiction, mobile phones with digital cameras are now inexpensive, widely available, and very popular. Significantly more camera phones are sold each year than dedicated digital cameras. According to Strategy Analytics, it's likely that over one billion camera phones were sold last year (see Surprisingly, the tremendous popularity of camera phones has caused little controversy. A few companies and institutions have banned camera phones owing to security concerns, and there has been some abuse of personal privacy, but for the most part, there has been worldwide acceptance of this new technology.

Enabling Mobile Phones To Support Large-Scale Museum Guidance

By Erich Bruns, Benjamnin Brombach, Thomas Zeidler, and Oliver Bimber
From the April–June 2007 issue of IEEE MultiMedia

Mobile phones have the potential of becoming a future platform for personal museum guidance. They enable full multimedia presentations and—assuming that the visitors are using their own devices—will significantly reduce acquisition and maintenance costs for museum operators. However, several technological challenges must be mastered before this concept is successful. One is the question of how individual museum objects can be intuitively identified before presenting corresponding information.


What's New


High-Dimensionality Data Reduction with Java

By Elena Villalon
From the September/October 2008 issue of Computing in Science & Engineering

Exploratory data analysis is fundamental to many scientific disciplines, including bioinformatics, the social sciences, engineering, and physics. Accumulating massive amounts of data doesn't necessarily yield more information, as the inclusion of unnecessary features can make the analysis overly complex. Because researchers need to have an efficient way of discerning patterns or implicit regularities in the data, preprocessing high-dimensionality data can have a profound effect on subsequent higher-level analysis. In this article, I show that preprocessing raw data to decrease its dimensionality and size is essential for interpreting data and conveying useful information about its attributes. Graphically visualizing data sets with histograms or other graphs can help researchers detect structural features and properties for classifying similar data sets or making inferences about different data sets.

Practitioners or Academics: Which Are We?

By Sorel Reisman
From the September/October 2008 issue of IT Professional

Many people view the IEEE Computer Society's membership as if it were bifurcated—split between practitioners and academics. Yet, from the standpoint of our profession, there's really no such division. In truth, these are merely labels on a continuum. Viewing the profession in this way lets us more readily understand the products and services the Society might offer in order to help computing professionals around the world selfactualize in their careers.

Bringing VR and Spatial 3D Interaction to the Masses through Video Games

By Joseph J. LaViola Jr.
From the September/October 2008 issue of IEEE Computer Graphics and Applications

Virtual reality (VR) and 3D user interface (3D UI) research has been going on for more than 20 years, with limited success in being a part of people's everyday lives. Don't get me wrong; VR and 3D UIs have been useful in areas such as simulation and training, scientific visualization, medicine, and psychological rehabilitation. But to say that VR and 3D UI research has significantly affected mainstream society is undeniably false. Compared to other technologies such as the mouse and WIMP (windows, icons, menus, point and click) interfaces, VR and 3D UIs have had little impact.

How Do We Build Trust into E-commerce Web Sites?

By Ejike Ofuonye, Patricia Beatty, Ian Reay, Scott Dick, and James Miller
From the September/October 2008 issue of IEEE Software

No doubt, lack of trust is a fundamental reason why many users won't purchase goods or services from "faceless" e-commerce Web sites. Building trust into—and maintaining trust in—an e-commerce system is difficult because so many different elements, and their interactions, can impact user trust. Because trust is a subjective, user-centric, context-dependent concept, it's elusive, and we're unlikely to ever realize a unique, universally acceptable definition. "Trust" means different things to different people, reflecting personal goals, predispositions, and experiences. On the Internet, several factors make trust more difficult to build, perhaps explaining why some successful brick-and-mortar retail chains have been unable to translate their reputation to the virtual platform the Web offers.

Learning to Tag and Tagging to Learn: A Case Study on Wikipedia

By Peter Mika, Massimiliano Ciaramita, Hugo Zaragoza, and Jordi Atserias
From the September/October 2008 issue of IEEE Intelligent Sytems

Natural language technologies will play an important role in the Web's future. Recent Web developments, such as the huge success of Web 2.0, demonstrate annotated data's great potential. However, when it comes to annotating documents even at the most primitive levels, human effort alone can't scale to the Web. Recently, the focus in the Semantic Web has shifted from text to user-supplied explicit annotations. We believe, though, that the Semantic Web community should develop these two visions in parallel. The Web 2.0 phenomenon brought renewed energy to annotating Web content, as large-scale tagging, adopting microformats, and introducing a data structuring mechanism in textual sources such as Wikipedia exemplified. At the same time, although databases generate most Web pages (as much as 80 percent, according to estimates), many of those databases store significant amounts of text devoid of machine-processable semantics. This quality is particularly true for one of the most exciting fractions of Web content: user-generated content. These text contributions populate blogs, wikis, social networks, and social media sites such as Yahoo Answers, YouTube, and Flickr.

ISP Traffic Management: Will Innovation or Regulation Ensure Fairness?

By Greg Goth
From the September 2008 issue of IEEE Distributed Systems Online

Two North American telecommunications regulatory agencies are proactively entering into some gritty details of network management. The results might announce a new era of strengthened regulatory capabilities to enforce Net neutrality. Concurrently, the Internet Engineering Task Force (IETF) has recently convened two Birds of a Feather (BoF) groups to address traffic-management complexities exemplified by rising peer-to-peer (P2P ) application use that some ISPs claim is choking off bandwidth for other users.

Video Browsing on Handheld Devices—Interface Designs for the Next Generation of Mobile Video Players

By Qibin Sun and Wolfgang Hürst
From the July–September 2008 issue of IEEE MultiMedia

Recent studies confirm that mobile video use on handheld devices is quite different from watching TV in your living room. For instance, assume you are standing at the bus stop, waiting for the 10-minute bus ride that will take you home from work. While waiting, you want to quickly view the latest evening news show, which you downloaded to your mobile device just before you left your office. Once you're on the bus, you want to quickly go to the one news report that seems the most interesting to you, because you don't have enough time to watch the whole news show. Shortly before you arrive, you want to take a quick look at the weather forecast. Because you don't have much time left, you just want to find a frame in the video showing the map of your state with the temperature values. When comparing such a situation to watching this news show at home, we can see that in the mobile scenario, much more interaction and navigation occurs. First, you have to quickly skim the video content to get an overview. Then you have to set replay around the beginning of the particular news clip you want to watch. Finally, you need detailed navigation—that is, the ability to move to an exact position or a particular frame within the video.

Hacking is Pervasive

By Joseph A. Paradiso, John Heidemann, and Thomas G. Zimmerman
From the June–September 2008 issue of IEEE Pervasive Computing

These days, "hacking" and "hackers" are often viewed negatively. Many people associate these labels with somebody breaking into a computer system with malicious or criminal intent. However, hacking's older definitions are quite different: interacting with a computer or any other technology-infused system in a playful or exploratory way, or modifying an existing system (hardware, mechanical, or software) to improve performance or create an application that differs from the device's original purpose. These definitions are much broader, significantly more positive, and even (in many circles) laudatory.

Toward Web-Scale Workflows for Film Production

By Chun Ouyang, Marcello La Rosa, Arthur H.M. ter Hofstede, Marlon Dumas, and Katherine Shortland
From the September/October 2008 issue of IEEE Internet Computing

The screen business is characterized by business processes with high demands for creativity and flexibility. These processes span a value chain consisting of four major phases: development, preproduction, production, and post-production. The production phase involves many stakeholders and is usually the most expensive. Most cast and crew are contracted during production, for example, and additional costs are associated with renting equipment, such as cameras, cranes, and action vehicles.

The Future Is Virtually Here

By Michael van Lent
From the August 2008 issue of Computer

Increasingly popular for more than a decade, massively multiplayer online role-playing games now generate remarkable revenues. The current king of MMORPGs, World of Warcraft, boasts 10 million active subscribers as of early 2008. Developed by Blizzard, WoW hosts a virtual population larger than the real-world populations of countries such as Sweden and Hungary.

Insiders Behaving Badly

By Joel Predd, Shari Lawrence Pfleeger, Jeffrey Hunker and Carla Bulford
From the July/August 2008 issue of IEEE Security & Privacy

Often, we worry about outsiders attacking our systems and networks, breaking through the perimeter defenses we've established to keep bad actors out. However, we must also worry about "insider threats": people with legitimate access who behave in ways that put our data, systems, organizations, and even our businesses' viability at risk.

Workloads of the Future

By Jan M. Rabaey, Daniel Burke, Ken Lutz, and John Wawrzynek
From the July/August 2008 issue of IEEE Design & Test of Computers

Over the past decade, we have witnessed far-reaching changes in the IT field. Semiconductor sales for consumer and communication devices now surpass those for traditional computation. The IT infrastructure is moving away from the desktop and laptop model to centralized servers, communicating with ubiquitously distributed (and often mobile) access devices. Sensor networks and distributed information-capture devices are fundamentally changing the nature of the Internet from download centric to upload rich (see Figure 1). Whereas today a billion mobile phones are sold per year, in the near future perhaps upwards of a trillion sensory nodes per year will be sold and deployed, with the majority of these connected wirelessly. User interfaces and human-machine interactions could become responsible for a large percentage of the computational needs.

Parallel Computing Experiences with CUDA

By Michael Garland, Scott Le Grand, John Nickolls, Joshua Anderson, Jim Hardwick, Scott Morton, Everett Phillips, Yao Zhang, and Vasily Volkov
From the July/August 2008 issue of IEEE Micro

With the transition from singlecore to multicore processors essentially complete, virtually all commodity CPUs are now parallel processors. Increasing parallelism, rather than increasing clock rate, has become the primary engine of processor performance growth, and this trend is likely to continue. This raises many important questions about how to productively develop efficient parallel programs that will scale well across increasingly parallel processors.