The Community for Technology Leaders

In the News

Pages: pp. 3-6

These Computers Know What You're Thinking

Danna Voth

What if you could play a video game or drive your car with your thoughts? Researchers are developing brain-computer interfaces that could be used to operate prosthetic devices, play games, and help astronauts perform repairs. BCI research has also led to the discovery of "brain fingerprints," biometrics that could help identify individuals and allow authenticated access to computers, cars, and buildings.

Typically, BCIs feed electroencephalography (EEG) readings into a computer to direct an action, such as moving a computer cursor or operating a wheelchair or mechanism such as a prosthetic or robotic arm. EEG signals represent the brain's electrical activity; they're usually collected through electrodes that touch the scalp and are distributed in a cap resting on a layer of gel on the head. As a person thinks or performs an action, the EEGs are collected, amplified, digitized, and analyzed.

However, Jacques Vidal, a University of California, Los Angeles, computer science professor, notes that getting EEG signals is a "relatively awkward process" and the "technology for gathering brain waves at this time is cumbersome and expensive." Isolating information from the EEGs is also difficult because the signals include a lot of noise or input from competing sources. For example, Vidal explains, "eye movement is one of the major disturbances that competes with a brain signal at the surface." But, he says that "BCI technologies will become commonplace in the future. The prosthetic branch is definitely feasible at the moment."

Improving BCIs

Training people to use BCI devices usually takes several sessions because EEG patterns change over time and the system thus needs updating, a process called discontinuous adaptation. Carmen Vidaurre, a postdoctoral researcher at the Public University of Navarre in Spain, has developed a BCI that both adapts to changes in EEGs and lets users adapt to the system through feedback. Her system uses low-noise signal amplifiers to capture the EEG signal. Then it extracts feature information from the signal's frequency spectrum by filtering in one or more of the frequency bands and applying adaptive autoregressive parameters that change in time with the EEG patterns.

Vidauure tested her continuously adaptive BCI system on people inexperienced in using BCIs. The subjects, connected to EEG monitors, sat in front of a screen showing a red and a green basket. When a ball appeared at the top of the screen, the subjects tried to guide it toward the green basket by imagining left- and right-hand movements. The visual stimulus of the ball served as feedback; the subjects could see how successfully they were moving the ball toward the target. Test results were encouraging: Vidauure says most subjects could control the BCI system after a few hours, and their skill improved noticeably over time. "Comparison with a discontinuously (traditional) adaptive system showed the advantage of the continuously adaptive one," she says.

Decoding signals

Decoding brain signals to isolate information about specific thoughts is an important BCI research area. Yukiyasu Kamitani and his colleagues at the Advanced Telecommunications Research Institute International Neuroscience Laboratories in Kyoto, along with researchers at the Honda Research Institute in Saitama, have been working to decode brain signals associated with body movements. While subjects made "rock, paper, scissors" movements with their hands during a 20-minute functional magnetic-resonance-imaging scan, Kamitani and his team collected brain signals for analysis. The fMRI signals measure hemodynamic responses in the brain—changes in the blood's amount of hemoglobin—that occur as a subject performs actions. Kamitani and his colleagues used a pattern recognition algorithm to analyze brain activity patterns and decode signals related to the specific hand movements. Next, they fed the information to a robotic hand, which mimicked the particular movements with 85 percent accuracy.

Graphic: Electrode caps are commonly used for recording EEGs. Unfortunately, the caps are impractical because they require the application of gel to bridge the gap between each electrode and the scalp. (photo courtesy of James [Hyung Cheol] Park)

Figure    Electrode caps are commonly used for recording EEGs. Unfortunately, the caps are impractical because they require the application of gel to bridge the gap between each electrode and the scalp. (photo courtesy of James [Hyung Cheol] Park)

"This technology is potentially applicable to other types of noninvasive brain measurements such as the brain's electric and magnetic fields and optical signals," Kamitani says. He likes working with fMRI because that technology provides better spatial resolution than EEGs or optical tomography. But, he says, "An fMRI system is huge." He adds that "the device should become more compact in the future." Kamitani thinks a BCI project using this technology is still far from becoming a working commercial system.

Brain fingerprints

BCI research is also leading to methods of using brain waves to identify individuals. Vidal says security identification is definitely penetrating the BCI field. "I see an explosion in the general framework of bionics," he says. Bionics is the science of utilizing biometric readings, or biological measurements, such as iris or fingerprint patterns or brain waves collected after stimulus, which might be useful in the security field as means of identification and authentication. Vidal says, "Evoked potential responses, which are short-wave forms collected following a brief stimulus, are relatively easy to collect."

While analyzing EEG data from alcoholics and a control population in a study of alcoholism's electrophysiological impairments, Ramaswamy Palaniappan, lecturer in the University of Essex Computer Science Department, made a discovery. "I stumbled upon the fact that the EEG waveform in the gamma band was distinct between the subjects," he says. Exploring further, Palaniappan realized that gamma band energy was different between subjects but was constant within the subjects, and therefore could serve as a biometric.

To test this, Palaniappan showed subjects black-and-white line pictures and recorded 61 channels of EEG data, gathered from many locations on the subjects' scalps. He removed eyeblink artifact disturbances through preprocessing and reduced noise through principal component analysis. The captured EEG signals in the gamma range were filtered and computed as features. A neural network classified the features into a category representing the subject, creating a "brain fingerprint." Palaniappan says the EEG biometric could identify or authenticate individuals by using their brain's electrical response to visual perception and recognition. "It has the advantage of confidentiality as brain activity is not easily seen," he says, "and it has the advantage of fraud resistance as it is unlikely to be stolen, because brain activity is difficult to mimic."

Changeable pass-thoughts

Julie Thorpe at Carleton University in Ottawa is working on using brain wave patterns to create what she and her colleagues call "pass-thoughts." If the brain waves created when an individual thinks about something are unique, they could be used much like a password to access computers, cars, and buildings—anything that uses authentication keys now.

Thorpe is studying the individual brain wave patterns that are evoked both actively, as when a person is shown some stimulus such as a picture), and passively, as when a person imagines something (such as a pet's name and color). "We are trying to see which sort of stimulus will give us the best innate response and then determine whether or not different stimuli for the same person are different enough statistically that you could actually make use of this," Thorpe says.

Furthermore, she wants to discover whether a set of stimuli that creates one set of brain waves for one person creates a different set of brain waves for other people.

In other words, once you establish that two different people create different brain wave patterns from a single stimulus, Thorpe wants to know whether the two people will exhibit uniquely different brain wave patterns when the stimulus changes. If that turns out to be the case, then pass-thoughts could become a better security method than other biometrics. For example, imagine that an airport has stored a template of your fingerprint for security purposes and that you also use that template to access your computer. If someone at the airport who has access to your file steals it, that person could spoof your fingerprint to both log on to your computer and enter airport security areas. But if you could simply change your pass-thought, its theft wouldn't leave you vulnerable. You could create a new pass-thought simply by changing the stimulus you're using to evoke a response.


BCI research might bring more flexible security and authentication technologies, as well as technologies to direct a variety of computerized and intelligent machines. But the methods for capturing those handy brain waves are still costly, time consuming, and cumbersome. However, Vidal says, "it need not remain so." Palaniappan concurs, predicting that owing to the tremendous interest and research in BCI, "we will likely see working systems in the market in a decade or so."

Microsoft Tries to Kick-Start the Age of Smart Robots

Jan Krikke

Could Microsoft's new Robotics Studio turn out to be the "killer platform" for robotics? With MRS, developers can simulate, develop, and manage robot applications, and third-party developers can build extensions to share or sell. The robot industry is highly fragmented, and the lack of a standard platform hampers robot development. MRS could unify robotics behind a common code base, freeing developers to focus on higher-level tasks. Not all developers might want to become dependent on Microsoft tools, but MRS will likely raise the profile of robotics, especially in the educational market.

"I think Microsoft's entry into the field of robotics is very exciting," says Hung Pham, cochair of the Object Management Group's Robotics Domain Task Force and principal applications engineer at Real-Time Innovations, a company that develops middleware for real-time systems. "It will raise visibility across the entire field and will make the practice of robotics more accessible to a wider audience. This can only help to increase participation and create more opportunities for innovation." Wolfram Burgard, a computer science professor at Germany's Freiburg University and head of the university's Autonomous Intelligent Systems research laboratory, calls MRS's launch a major step for robotics. "Earlier robotics systems mostly targeted children or students, and it was hard to imagine extensions that are closer to products. A complex programming system might change this entirely and bring robots closer to people of all ages."

MRS runtime

MRS, available for download at, has three main components:

  • MRS Runtime, a runtime library for distributed processors,
  • a simulation environment to test robotics programs, and
  • a visual-programming tool.

MRS Runtime is arguably MRS's most significant feature. It provides what Microsoft describes as "a service-oriented architecture which combines key aspects of traditional Web-based architecture with pieces from Web Services to provide a highly flexible and lightweight, distributed application model."

MRS Runtime supports such varied applications as observing sensory input, drive-by-wire, and autonomous-robot cooperation. Developers can apply it to robots connected to a PC, robots with an onboard PC, or robot simulations.

Pham says that the MRS Runtime framework can significantly improve development of these complex systems. "A framework will separate a component's 'business logic' from its execution or deployment concerns," he explains. "This will allow the component developer (for example, the algorithm designer) to focus on his or her high-value contribution, rather than being caught up in mundane tasks, such as managing the component's life cycle." He adds that the runtime framework will simplify integrating many components from multiple contributors. "This will allow the system integrator to segment the problem into manageable pieces and have the implementation of these pieces be carried out by different providers. The consequence is that much more complex problems can be tackled and efficiently solved by large teams."

Commercial adoption

Several commercial robot developers have already embraced MRS. Lloyd Spencer, the president of CoroWare (, says his company has been interacting with Microsoft Research on robotics-related matters for well over a year and a half. He says that work with MRS has contributed to the utility and functionality of the CoroWare Surveyor 3000, the company's latest mobile service robot.

The European company Robosoft ( has switched all its software development to .NET/MRS. "Our proprietary robotic SDK [software development kit] was limiting our capabilities for improved development and cost reduction," says Robosoft's CEO Vincent Dupourque. "This limited our ability to target the growing market of service robotics. We settled on Microsoft's solutions because of their overall quality, completeness in terms of authoring and management tools, and the availability of skilled and experienced programmers." The key benefits of MRS, says Dupourque, are the possibility of having a single SDK for onboard and off-board software, the introduction of time management solutions through the Concurrency and Coordination Runtime, and a simple way to deploy heterogeneous distributed applications.

MRS and education

The Institute for Personal Robots in Education, cofounded by the Georgia Institute of Technology and Bryn Mawr College, will use MRS to introduce robotics into the computer science core curriculum. The IPRE will create a new version of Pyro, a robotics platform developed to make robotics more accessible to undergraduates. The new version of Pyro, tentatively dubbed "Myro," will be integrated with MRS.

Bryn Mawr Computer Science Chair Deepak Kumar said in a press release that robotics has much to gain from incorporating a wider range of perspectives. "Bryn Mawr's involvement in this partnership introduces the ideas and problems in artificial intelligence and robotics to a very different set of students from the traditional engineering types that have worked on those problems over the past 50 years. As a result, I think we will see some very different and amazing solutions to these kinds of problems."

Carnegie Mellon University's newly founded Center for Innovative Robotics ( will also use MRS. The center aims to make robotics accessible to a broader range of individuals and businesses. Like the IPRE, it's partly funded by Microsoft.

The down side?

MRS isn't without its critics. One possible concern is security. Users can control robots using Windows Explorer; Mobile-Robots president Jeanne Dietch, quoted in PC Magazine ( article2/0,1759,1979616,00.asp), suggested that users might want "something a little more difficult to hack."

However, Dupourque points out that using the browser to control robots is only one aspect of MRS. "The use of the browser is currently highlighted by Microsoft through their tutorials and demos because they now target hobbyist and limited robotic-skilled developers. Using the Web browser and JScript is a way to quickly and easily move robots, but we strongly doubt that professional robotic applications can be developed using this solution."

Dupourque adds that Microsoft's push toward Web browser applications "might also be due to the robots they are working with, which are not professional-grade robots, and also their limited experience in service and professional robotics. But this is exactly the reason why they decided to create a group of early adopters, including Robosoft." Dupourque stressed that that Robosoft will not use browsers to control robots. "The only security concern would be the hacking of wireless transmissions between robots and their central supervision system. But this is managed using cross-validations and coherence analysis of the received orders and, in any case, is not due to the use of MRS but of wireless communications."

Another possible problem, according to Pham, is the proprietary technology that MRS uses. "Users will be necessarily locked into Windows, which may not provide the necessary support that some robotics developer may need—for example, hard real time, small footprint, and so on. Also, the issue of having to go through a managed code layer may be very problematic, especially if one has strict performance requirements or if one has to deal with legacy issues."


Robot developers compare the state of robotics to the PC industry in the 1970s. The PC market was held back by fragmented systems until the industry standardized on the IBM PC and the Windows OS, which launched the PC industry's spectacular growth. By providing a common software foundation, MRS could similarly bootstrap the robot industry. Perhaps the days of a robot in every home are closer than you might think.

63 ms
(Ver 3.x)