The Community for Technology Leaders

In the News

Pages: pp. 5-9

Abstract—This News feature explores how AI techniques are being implemented to improve gaming products. It also reports on advances in autonomous, AI-augmented sailing vessels and electronic discovery (e-dicovery) in legal cases involving litigation, regulatory inquiries, and internal investigations.

Keywords—Intelligent systems, e-discovery, robotic boats, sailboats, AI in games

Using AI to Make Games Smarter


Computer gamers typically look for the next great thing, such as improved graphics and more realistic action. Game researchers have long used AI to improve their gaming products, but the practice has recently taken off in several interesting directions because of improvements in AI and enabling technologies.

Game makers even want to use the technology to improve the development process.

"There has been an almost tectonic shift in the way AI is viewed in the industry," noted Dave Mark, owner, president, and lead designer of Intrinsic Algorithm, a developer of entertainment and recreation software and services.

"Ten years ago," he explained, "AI in games was largely an afterthought. Since that time, most [game makers] have dramatically increased their focus on AI…. It is becoming not only a more important component of games but also a framework on which to hang entire portions of game design."

Games also provide a testbed for developing tools for AI research in general, said Adi Botea, senior researcher at both the Australian National University and National Information and Communications Technology Australia (NICTA).

AI in Games

Three areas have been particular targets of new AI research for games, said Botea.

Searching for the optimal route among numerous alternatives is used in pathfinding, which determines how game characters get from one point to another. This is one of the most common AI problems in games. Learning also lets AI nonplayer characters tune their behavior based on their own experience, examples that an expert provides, or an opponent's actions. Lastly, planning adapts the decision-making process to a game's environment, rules, and current state of play. It also computes plans in which actions must make sense within the game's present conditions.

Challenges to Address

Some industry observers say the days of graphics being the driving force behind the demand for games are over. Instead, they say, players want games that are more sophisticated and unpredictable, which will require improved AI. This will necessitate more intelligent behavior, including better decision making.

"In game AI, we have plenty of [hard-coded] techniques for decision making," said Intrinsic Algorithm's Mark. Those techniques include neural networks and fuzzy decision making.

However, Mark noted, this approach becomes more complex and less effective as game worlds become larger and more varied. Sophisticated players will not be interested in games in which characters display artificial, nonadaptive hard-coded behavior, such as repeating the same mistake many times or not adapting to simple environmental changes.

Addressing this, he added, requires techniques that can elegantly handle a near-infinite number of potential situations without the tedious handcrafting of one-off rules. These techniques will require AI, according to Botea.

Meeting the Challenges

AI techniques used in characters' decision making have matured over the last few years, using techniques inspired by robotics, said Alex Champandard, editor of, a website covering AI and game development.

Robotics research has advanced search, learning, and planning algorithms to upgrade a robot's autonomous decision making. This can improve the way nonplayer characters in games behave. Experience management uses learning algorithms to quantify the elements that relate to the player's game experience, such as how fast someone is progressing on various challenges. This technique can tune the nonplayer characters and various variables to keep a player engaged.

An AI director uses planning algorithms to make decisions about a game's composition and flow. AI techniques involving player profiling and agent research also help game developers.

Intelligent Trends

The increased use of AI in games is due in part to the development of better tools for managing the use and application of AI, said Max Dyckhoff, a senior engineer at game-developer Blizzard Entertainment.

Intrinsic Algorithm's Mark said the main obstacle to using more AI in games has been that the tools can be difficult to use. He expressed hope that better, easier-to-use tools will do work that can free up developers to focus more on the quality of game play.

With such tools, several areas are particularly promising for AI use in games: character animation, production, difficulty adjustment, and intercharacter communication.

Improved Character Animation

The demand for high-quality animation in games has grown dramatically over the last decade, and AI is helping to meet these expectations, said Bill Merrill, senior programmer at Double Helix Games. Determining the best paths characters should take and having them follow those paths precisely is a critical issue, he noted.

Current game-development libraries force designers to think in terms of units of code with simple functionality, such as specifying an individual walking technique. A more expressive AI-behavior language could allow higher-order abstractions that could, for instance, enable characters to dynamically avoid obstacles. This would keep developers from having to manually link separate libraries for the various tasks involved in obstacle avoidance.

Dynamic Difficulty Adjustment

A key challenge of keeping games engaging is adjusting their difficulty to an individual player's abilities. Games that are too easy or hard will bore players, explained Chulalongkorn University's assistant professor Vishnu Kotrajaras, who specializes in the use of AI in games.

He said this problem is something that experience-management and planning algorithms could help solve.

Researchers are also looking at machine-learning and neural-network approaches that let nonplayer characters adapt to a human player's strategy, thereby keeping a game interesting. Additional research involves genetic algorithms, which use an evolutionary process to determine the best way for games to respond to players' actions.

Improving the Production Process

Another challenge in game development is making sure the development process doesn't keep the product from being delivered on time and on budget.

An emerging area of research in this area, called procedural content generation—creating material algorithmically rather than manually—is looking at how to use AI to automate the development process.

In much the same way, the pharmaceutical and other industries have used AI techniques such as machine learning to automate, accelerate, and improve the product-development process, according to Simon Colton, an Imperial College London reader in computational creativity.

Intercharacter Communication

Gaming is limited by the dearth of good approaches for creating characters that can work together and communicate dynamically, said Jeff Orkin, a doctoral student in the MIT Media Lab.

The MIT lab is working on AI-based ways to plan characters' social behavior in games by recognizing approaches that have been successful under similar circumstances.

Inspiring AI Use in Other Settings

The improvement of AI techniques in games is designed to create more believable characters, more engaging play, and more efficient production processes. However, these efforts could have benefits in other research areas. For example, improvements in dynamic difficulty adjustment could improve training tools for stroke patients, said Chulalongkorn University's Kotrajaras. Each patient's recovery speed differs, so being able to adjust rehabilitative exercises' difficulty is crucial to maintaining user motivation and the program's effectiveness, he explained.

Meanwhile, AI used to improve game characters could help with military and search-and-rescue training.

Also, Botea said, pathfinding techniques in games could be applied to robot navigation and road-map route planning.

Robots Sail the Ocean Blue


Robots walk, climb, drive cars, and even fly as drone aircraft, but so far they have been unable to function as boats that can effectively sail over long distances.

That is changing, however, as numerous researchers are working on sailbots

that can hit the open water on their own.

AI and the Challenges of Robotic Boats

Sailing introduces new challenges to the AI used to manage autonomous vehicles. These challenges relate to the long duration of sailing trips and the rapidly changing air and water conditions that affect movement.

A sailing drone's most difficult challenge is traveling long distances accurately, noted Technical University of Denmark doctoral student Qiuyang Zhou, who is also head of engineering for the Protei sailbot project. AI can make decisions based on weather forecasts, information from sensors that monitor a boat's position, wind speeds, and so on, said Zhou. The vessels also must be able to understand and sense sea conditions, which are highly complex, he explained.

Using AI can help a system solve problems by working with data gleaned from the experience of sailors over the years. Many parameters of automated sailing require this capability, which comes with experience, noted maritime engineer Etienne Gernez, the academic coordinator for the Protei project. The robots can learn via their own experience and through simulations, she said.

The key to robotic boats' success is the AI controllers that handle route calculation and course maintenance. Algorithms determine the most efficient route to a destination based on wind speed and direction. The AI controller is a system of hardware, such as motors, and software that uses neural networks to determine if the boat is on course. Course-maintenance algorithms use data from GPS equipment and sensors that track how far the vessel moves off course. The system can adjust the rudder to correct for problems.

Sailbot Competitions

The first major robotic-sailing competition was the 2006 MicroTransat Challenge, which involved attempts to sail boats no more than 4-meters long across the Atlantic Ocean. Subsequent events were held in 2007 and 2010, with a fourth scheduled for this year.

So far, none of the participants has gone more than 30 kilometers, but researchers say they have learned much along the way.

The MicroTransat Challenge helped inspire a shorter-distance competition called the World Robotic Sailing Challenge. The event, which takes place over a race course, has been held every year in different parts of the world since 2008.

Roboat, designed by researchers at the UK's University of Aberystwyth, has proven successful in shorter competitions. The boat is 3.75 meters long and weighs 300 kg. Its AI system uses data from various sensors to control the rudder, sails, tacks, and jibes.

The University of Aberystwyth researchers have used their experience with Roboat to build the Pinta, a smaller, less expensive, oceangoing version. They made the Pinta smaller and less costly so that it wouldn't be such a hardship if they lost the boat at sea.

The open source Protei project is pursuing an alternative design for sailboats (see Figure 1). The design differs from that of the Roboat and Pinta primarily in that the new vessel wouldn't have a fixed hull shape. The new boat's AI controller could change the shape of the inflatable hull to optimize performance.


Figure 1   Protei robotic boat. The Protei project is designing a robotic vessel that uses AI to process and act on information collected by sensors. The boat's AI controller changes the inflatable hull's shape to optimize performance. (Courtesy of Protei)

Sailing Ahead

More research is needed on some of the challenges related to using AI to react to storms, sudden wind-direction changes, and strong currents, Gernez said. Better AI tools, using techniques such as machine learning, are required to identify and adjust to changing events.

Also, he added, "A good robot should be able to repair itself or continue functioning if one part breaks down."

Gernez envisions sailbot applications such as collecting plastic in the Great Pacific Garbage Patch, a floating area of plastic debris in the Pacific Ocean; monitoring areas in which marine fauna and flora are protected; and keeping track of coastal water quality.

The Protei project is designing an open source hardware and software system for sailbots that could clean up maritime oil spills, in the hope that someone will be interested in building the vessels.

Software Uses AI in Place of Lawyers


Complex court cases can entail teams of lawyers and paralegals, months of work, and large expenditures, much of it for the analysis of reams of documents. To help tackle this volume of work, electronic-discovery software is using AI to analyze documents, saving considerable time and money.

These programs are helping to find documents relevant to legal proceedings and cull outdated documents that attorneys don't need to retain. Moreover, proponents say, e-discovery software can be more precise at document analysis in some cases than people. After long periods of analysis, some people get too bored or tired to be very accurate, said Katey Wood, analyst with the Enterprise Strategy Group, a market research firm.

E-discovery is a prime example of a new trend in which intelligent software can save time and money doing the same work that multiple people would need a long time to complete.

E-discovery Technology

Electronic discovery is critical in legal cases involving litigation, regulatory inquiries, and internal investigations, said Kurt Leafstrand, director of product management at Symantec, which has a line of e-discovery products. These cases entail considerable document analysis, which can be expensive if done manually, he explained.

The most common types of e-discovery software use natural language processing and other linguistic techniques to search material, identify concepts, and classify the importance of documents. E-discovery is starting to utilize various AI techniques.

Predictive coding uses machine learning algorithms to train an AI system to recognize relevant and irrelevant documents that are part of a larger set, such as a company's email database. Lawyers and paralegals could then analyze the relevant documents more carefully.

Other techniques include concept foldering, which links documents that contain a particular item of interest, such as a person's name. Early case assessment identifies documents that are likely to be either relevant or irrelevant to a case, thereby reducing the paperwork that a human must analyze and that a law firm must retain. And network analysis uses graph-analysis algorithms popularized by social networking to map communications between people of interest in a legal proceeding.

Newer approaches, such as Cataphora's actor-role-analysis tool, use network analysis to identify patterns in email, telephone, and other communications relevant to a legal issue, such as an employment dispute.

Discovering What's on the Horizon

Large vendors are starting to build portfolios of e-discovery tools that they can integrate into existing enterprise-application suites. Notable acquisitions in this area include Cataphora by the Ernst & Young professional-services company, Clearwell Systems by security vendor Symantec, and Iron Mountain's e-discovery business by data-analysis-software vendor Autonomy.

Nonetheless, e-discovery faces challenges. "Developing a process and workflow able to provide a quantifiable measure of confidence in the machine's decisions has been challenging, and systems that provide true enterprise-class functionality that can be utilized by the average lawyer are only just emerging," explained Leafstrand.

Also, e-discovery might cost some legal personnel their jobs and reduce law firms' billable hours.

Currently, the technology is being used mostly in high-profile cases or in the pharmaceutical, tobacco, and other industries that face regular legal proceedings. Figure 2 shows its current and predicted future usage based on a survey of law firms by Enterprise Strategy Group.


Figure 2   Lawyers currently are employing electronic-discovery technology in relatively small numbers. However, based on a survey of law firms, the market research firm Enterprise Strategy Group expects usage to increase significantly in the next two years. (Courtesy of Enterprise Strategy Group)

Blackstone Discovery president and CEO John Kelly explained, "Even though e-discovery has been exploding, lawyers are nervous about investing in it. When it comes to the right case, people are willing to pay for the best technology. But other times, you just need meat and potatoes."

66 ms
(Ver 3.x)