Issue No. 08 - August (2004 vol. 37)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/MC.2004.72
Today, both the Internet and corporate data warehouses are full of all types of digital information, from simple text documents to complex applications. One type of information gaining in prominence is the three-dimensional object. From computer-aided design (CAD) drawings of complex engineering parts to digital representations of proteins and complex molecules, an increasing amount of 3D information is making its way onto the Web and into corporate databases.
Because of this, users need ways to store, index, and search this information. Typical Web-searching approaches, such as Google's, can't do this. Even for 2D images, they generally search only the textual parts of a file, noted Greg Notess, editor of the online Search Engine Showdown newsletter.
However, researchers at universities such as Purdue and Princeton have begun developing search engines that can mine catalogs of 3D objects, such as airplane parts, by looking for physical, not textual, attributes. Users formulate a query by using a drawing application to sketch what they are looking for or by selecting a similar object from a catalog of images. The search engine then finds the items they want.
Susan Feldman, research vice president of content technologies at IDC, a market research firm, said 3D image recognition and searching would be important for intelligence agencies examining photos, corporate marketing departments looking for one of many product images, or any individual or organization trying to find an item in a collection of objects.
3D search engines could also help big companies quickly find whether they have certain parts in their inventories. For example, many big engineering and manufacturing companies have huge databases of parts they search for components to use in new products. Frequently, engineers spend a lot of time looking for such parts either manually or by older automated methods. If a part exists but engineers can't find it, the company must make it again, wasting valuable time and money.
3D search engines could eliminate these problems and also let users mine for other important parts-related information, such as cost or manufacturing method.
However, the technology faces several technical and marketplace hurdles before it can become popular and commercially successful.
Searching in Three Dimensions
Advances in computing power combined with interactive modeling software, which lets users create images as queries for searches, have made 3D-search technology possible.
3D searching has several important elements.
The key to the way computer programs look for 3D objects is the voxel (volume pixel). A voxel is a set of graphical data—such as position, color, and density—that defines the smallest cube-shaped building block of a 3D image.
Computers can display 3D images only in two dimensions. To do this, 3D rendering software takes an object and slices it into 2D cross sections. The cross sections consist of pixels (picture elements), which are single points in a 2D image.
To render the 3D image on a 2D screen, the computer determines how to display the 2D cross sections stacked on top of each other, using the applicable interpixel and interslice distances to position them properly. The computer interpolates data to fill in interslice gaps and create a solid image.
True 3D search systems offer two principal ways to formulate a query: Users can select objects from a catalog of images based on product groupings, such as gears or sofas; or they can utilize a drawing program to create a picture of the object they are looking for. For example, Princeton's 3D search engine uses an application to let users draw a 2D or 3D representation of the object they want to find.
The 3D search process
The 3D-search system uses algorithms to convert the selected or drawn image-based query into a mathematical model that describes the features of the object being sought. This converts drawings and objects into a form that computers can work with.
The search system then compares the mathematical description of the drawn or selected object to those of 3D objects stored in a database, looking for similarities in the described features.
3D SEARCH ENGINES
Purdue and Princeton researchers have developed two major examples of 3D search engines.
Scientists at the Purdue Research and Education Center for Information Systems in Engineering, led by Professor Karthik Ramani, created a 3D shape search technology called 3DESS (3D Engineering Search System; http://tools.ecn.purdue.edu/~cise/dess.html). The engine is designed primarily to find computer-designed industrial parts.
For example, said Ramani, the system would be useful for any design and manufacturing company that deals with many CAD models.
3DESS gives users three options for searching industry databases: using a drawing application to sketch a part from scratch, penciling in modifications to an existing part to create a new search object, or selecting a part from a catalog of choices.
3DESS begins by converting query drawings into voxels via an algorithm that uses mathematical representations describing the solid shape.
Another algorithm uses thinning, which extracts only the voxels that represent the most important parts of the shape, to create a skeleton of the object. This is a reduced graphical representation of the object that displays its basic outline and topology, as Figure 1 shows.
3DESS then develops a skeletal graph, using an algorithm that analyzes and renders the skeleton in terms of three common topological constructs: loops; edges; and nodes, which are connecting points between loops and/or edges. Expressing an object in terms of these common constructs, plotted on a graph, reduces the overall amount of data in the object representation and makes it easier to store and index descriptions in the database.
According to Ramani, 3DESS also describes objects in terms of feature vectors, a set of mathematical representations of various aspects of the item's shape, such as its volume, surface area, and the number of loops, edges, and nodes.
The system can analyze a query by comparing either its feature-vector set or its skeletal graph with those of objects stored in a database.
When the system retrieves models in response to a query, users can input which models more closely resemble the object they're seeking. 3DESS then uses neural-network technology to analyze users' feedback, learn more about what they want, and fine-tune both the current search and future similar searches.
Professor Thomas Funkhouser and his colleagues at the Princeton Shape Retrieval and Analysis Group ( www.cs.princeton.edu/gfx/proj/shape) developed their Web-based 3D Model Search Engine. The engine works with a Java-based application that lets users produce, via their mouse, 2D renderings of objects to search for in a database.
Funkhouser and his team were not available for comment. However, Computer was able to speak with Michael M. Kahzdan, an assistant professor at Johns Hopkins University who, while a Princeton PhD student, worked on the 3D search project with Funkhouser.
The Princeton search engine, like its Purdue counterpart, uses mathematical representations to store the salient characteristics of 3D shapes in its database and to describe drawings that users submit as queries. The system compares queries' shape representations with those stored in the database to find the ones that are most similar.
Princeton's 3D search engine first renders a 3D object or drawing as a voxel grid and then maps this to a sphere, yielding spherical-based mathematical functions. The system then breaks the spherical functions into component parts that make up the ultimate shape descriptor, a graphlike abstraction that represents the item's overall shape.
An advantage of the spherical approach, Kahzdan explained, is that it lets the search engine work with shape descriptors regardless of the object's position. Thus, the system will match a query with the appropriate result even if the query drawing is positioned differently than the object in the database.
Searchers can refine their queries by adding a text description.
The Web-based 3D Model Search Engine has not yet been released commercially, according to Kahzdan.
3D search technology faces several potential obstacles. For example, researchers are trying to improve accuracy so that search results more closely match queries.
In addition, researchers must improve search speed, said Nainesh Rathod, president and CEO of software vendor Imaginestics, which is licensing Purdue's engine for use in its products this fall. He noted that researchers are currently trying to improve speed by developing new search algorithms.
Ease of use is another important issue. For 3D search engines, ease of use will boil down to the manner in which people can enter queries, according to Search Engine Showdown's Notess. Drawing shapes is more difficult than entering text queries. Thus, Notess said, 3D search engines will be most helpful for users with computer-aided drawing skills, such as those who work with CAD and graphic design.
Also, he noted, the quality of models drawn by users can be a factor if drawing is the main or only query approach. This would not be such a problem if users can select predrawn objects from catalogs, he added.
"Advances in systems allowing users to efficiently and effectively generate models would play an important role," Johns Hopkins' Kahzdan said.
Currently, said Kahzdan, 3D search focuses on submitting an image of an entire object as a query and receiving an entire object as a matching result. He said it would be important if, for example, researchers could give 3D search engines the capability to take a query that is a part of an object or scene and either return the entire object or scene as a result or find the part within a larger object or scene.
Developing this approach, he noted, is a much more challenging technical problem but could be useful for such purposes as recognizing health problems in the results of 3D medical imaging. For example, a physician could search a hospital's database of already-diagnosed magnetic-resonance scans to accurately diagnose a vexing abnormality found in a patient's scan.
Most of today's individual and corporate computer users don't conduct the type of searches that would justify paying for and learning to use sophisticated 3D search engines. Thus, said IDC's Feldman, the technology is likely to find only small, niche markets.
However, Notess said, within those markets, 3D search could prove useful and successful.
Sixto Ortiz Jr. is a freelance technology writer based in Spring, Texas. Contact him at firstname.lastname@example.org.