This special section was planned in honor of the memory of late Professor King-Sun Fu, who passed away in 1985. The year 2005 is the 20th anniversary of this unfortunate event. Professor K.S. Fu was the founder of the IEEE Transactions on Pattern Analysis and Machine Intelligence ( TPAMI) and served as its first Editor-in-Chief. This special section is a tribute to his memory. Dr. King-Sun Fu is widely recognized for his paramount contributions in the field of pattern recognition, especially in the area of syntactic and structural pattern recognition. This is a moment of reflection to gauge the progress of the field in the last 20 years and list new efforts that are being made at present.
2. Problemsof Interestin Syntacticand Structural Pattern Recognitionandin Related Areas
The fields of syntactic and structural pattern recognition experienced explosive growth during the 1960s and 1970s, as evidenced by the number of papers published in this area. The International Association for Pattern Recognition (IAPR) was established in 1978, with Professor Fu as its first president. In the following year, the first issue of TPAMI was published. Since then, the field of syntactic and structural pattern recognition has evolved with time to keep up with the demand of real-world problems. Parallel to the developments in syntactic and structural pattern recognition, other important paradigms emerged and were further developed, such as statistical, neural, and fuzzy approaches.
3. Syntactic Pattern Recognition
The syntactic approach takes the view that a pattern is composed of simpler subpatterns which, in turn, are built from even simpler subpatterns [
^{1} ], [
^{2} ], [
^{3} ]. The most elementary subpatterns are known as primitives. A complex pattern is then expressed in terms of the relationships among its primitives. An analogy between the structures of patterns and the theory of formal languages is used to establish the foundation for syntactic pattern recognition. The patterns represent the sentences in a language, while the primitives constitute the alphabet of the language. A grammar for a language generates and identifies sentences belonging to that language by employing its rules. The idea that a potentially large set of related complex patterns can be described by a finite number of primitives and grammatical rules makes this approach appealing. Moreover, in addition to answering questions regarding membership, the rules of the underlying grammar provide a description about how to assemble the patterns/objects from their basic building blocks, the primitives. The issues in developing syntactic approach to a pattern recognition problem are:
• selection of type of grammar—deterministic, stochastic, fuzzy, or a hybrid grammar,
• selection of level of descriptive complexity—string, tree, or graph grammar,
• selection of an optimal set of primitives—too many may lead to an unmanageably large set of rules, while too few may lack enough descriptive power,
• grammatical inference (GI)—learning the rules of the grammar for a pattern class from sample patterns, and
• parsing—finding out if a given sentence is a member of the language generated by a certain grammar.
4. Structural Pattern Recognition
There are many applications where patterns can be described in terms of primitives and their relations, but where a grammar is not suitable for pattern class description because the patterns under consideration lack regularities and can't be defined by rules. In such a case, the structural approach to pattern recognition can be adopted. In structural pattern recognition, we use symbolic data structures, such as strings, trees, and graphs, for the representation of individual patterns, similarly to the syntactic approach. However, rather than using a grammar, we represent pattern classes through a number of prototypes. Consequently, the recognition problem turns into a pattern-matching problem. For example, given a database of strings, each representing a sample pattern from our learning set, class membership of an unknown pattern is determined by matching the unknown string with all (or a suitable subset) of the prototypes in the database. The final goal is to assign the unknown pattern to the class of the most similar prototype from the database.
Similar to the syntactic approach, there is a tradeoff between representational power and computational complexity in structural pattern recognition. String matching, i.e., measuring the degree of similarity of a pair of strings, has a complexity that is only quadric in the lengths of the two strings under consideration. But, the representational power of strings is limited. (Strings are adequate to model one-dimensional signals and objects, but encounter serious limitations when it comes to the description of two or higher-dimensional patterns.) On the other hand, graphs are a universal modeling tool, yet many operations on graphs are very costly from the computational complexity point of view. For example, testing if one graph is a subgraph of another is exponential in the number of nodes involved. Nevertheless, in many applications heuristics and constraints can be found that cut down the complexity to a manageable size. Another alternative to cope with the high-computational complexity of graph matching is to resort to suboptimal methods based on neural networks, simulated annealing, genetic algorithms, etc.
The call for papers in January 2004 resulted in the submission of 40 manuscripts. After a careful review process, 11 papers were selected for publication. There are five papers in each area and the last one combines semantic, syntactic, and structural approaches. These papers provide a wide range of viewpoints and address a variety of key issues. Import of techniques/ideas from other disciplines and new applications have changed the landscape of pattern recognition research. It is hoped that this special section will provide a useful snapshot of the current problems and research efforts in the area.
The five papers in the syntactic pattern recognition area cover two of the most challenging issues—1) grammatical inference (GI) and 2) parsing. Four of these papers deal with stochastic grammars and the fifth paper with deterministic grammar. The bias toward stochastic grammar reflects the trend in the current research endeavors. Development of algorithms for GI has evolved over the years from dealing with only positive training samples to more fundamental efforts that try to circumvent the lack of negative samples. This idea is pursued in stochastic grammars and languages, which attempt to overcome absence of negative samples by gathering statistical information from available positive samples [
^{2} ]. Stochastic grammars are robust and possess better generative power than corresponding underlying deterministic grammars. Of the four papers on stochastic grammars, three are on string grammars and one is on two-dimensional tree grammar. Again, this manifests the complexity of higher order grammars.
The two-part paper "Probabilistic Finite-State Machines" by Enrique Vidal, Frank Thollard, Colin de la Higuera, Francisco Casacuberta, and R.C. Carrasco studies various properties of a finite state machine and its relation to other generative models of similar stature. Probabilistic finite state machines represent a class of syntactic objects, e.g., probabilistic finite state automata, hidden Markov models, stochastic regular grammars, probabilistic suffix trees, n-grams, weighted automata, etc. Researchers have brought in different terminologies/definitions depending on their background/training areas of specialties. This paper takes the first step toward proposing a common language, interrelating these models and disseminating important theoretical results. It covers both parsing and learning issues.
Jose L. Verdú-Mas, Rafael C. Carrasco, and Jorge Calera-Rubio introduce a family of stochastic k-testable tree languages (in the case of strings, this would correspond to k-gram models) in the paper titled "Parsing with Probabilistic Strictly Locally Testable Tree Languages." The k-testable tree language is a proper subclass of the class of languages recognized by finite-state tree automata. Higher order grammars are very attractive because of their generative (representation) power. Thus, they are better suited for real-world pattern recognition problems than string grammars. Unfortunately, the associated inference and parsing issues become computationally difficult. Here, the authors propose an efficient technique to build a probabilistic k-testable tree automaton from a deterministic finite-state tree automaton (similar in spirit to the procedures used to obtain a k-gram model from a context free grammar in the case of strings). The authors provide a comparison of parsing performance with that of unannotated and parent-annotated PCFGs.
Yasubumi Sakakibara addresses the application of grammatical techniques in the emerging area of bioinformatics in the paper titled "Grammatical Inference in Bioinformatics." This is an area that is experiencing tremendous growth. Biological knowledge is being accumulated at a fast rate (for example, through genome sequencing). To make sense of this information is a challenge. A major part of the challenge is to organize, classify, and parse the richness of the sequence data. This is much more than an abstract task of string parsing [
^{3} ]. There are intricate patterns of information hidden in these sequences of bases or amino acids. What is very interesting here is the characteristics of the sequence data and some of the associated research issues. Sequences are long chains composed from a few symbols (four in the case of DNA and RNA, 20 in the case of protein). The author delineates seven tasks in biological sequence analysis and discusses the suitability of different stochastic grammars for the given tasks.
Simon M. Lucas and T. Jeff Reynolds discuss a new learning algorithm for deterministic finite automata from a set of labeled strings in their paper "Learning Deterministic Finite Automata with a Smart State Labeling Evolutionary Algorithm." The learning scheme is drawn from the area of evolutionary computation. Comparisons with other DFA learning algorithms are provided. The authors present results with benchmark noise-free and noisy data sets.
The next paper combines structural, syntactic, and semantic approaches. In "Structural Semantic Interconnections: A Knowledge-Based Approach to Word Sense Disambiguation," Roberto Navigli and Paola Velardi discuss the problem of word sense disambiguation, which is an important issue in computational linguistics with significant impact on many Web-based applications. The authors use directed labeled graphs (where nodes and edges code concepts and semantic relations between two concepts, respectively) to represent word sense. A context-free weighted grammar that encodes the meaningful semantic patterns among graphs is used to find the most likely interpretation of a word. The proposed process is iterative. The paper contains experimental results on standard and domain specific test data.
The next five papers in the area of structural pattern recognition focus on two important issues: 1) matching/clustering procedures for trees and graphs and 2) pattern representation using graphs. In the case of matching, attempts are made to find optimal solutions for the matching of symbolic data structures, or to cast the symbolic matching problem into some other framework, for example, a probabilistic one. The notion of edit distance is implicit in the issue of matching/clustering. The edit distance of two symbolic data structures is a very important concept in structural pattern recognition [
^{4} ], [
^{5} ], [
^{6} ]. It is based on the idea of measuring the similarity between two patterns in terms of elementary edit operations (for example, substitution, deletion, and insertion of nodes and edges in a graph) required to make the two structures identical with each other. Complexity is usually high as one deals with graph structures. One of the papers explores the possibility of converting graph structures to equivalent pattern vectors. Two other papers investigate the issue of pattern representation from two different aspects.
The paper titled "Polynomial-Time Metrics for Attributed Trees" by Andrea Torsello, Džena Hidović-Rowe, and Marcello Pelillo proposes general distance measures on trees with symbolic or continuous-valued attributes. The measures are computable in polynomial time. The authors use proposed measures and present experimental results for matching shapes.
The next paper "Exact and Approximate Graph Matching Using Random Walks" by Marco Gori, Marco Maggini, and Lorenzo Sarti, proposes a polynomial time algorithm for the graph isomorphism problem for a special class of graphs—Markovian spectrally distinguishable graphs. The work here opens new research directions for the general subgraph matching problem. The authors use their matching algorithm for an image retrieval task and present experimental results on standard data sets.
To avoid dealing with graph structures that are computation intensive, Richard C. Wilson, Edwin R. Hancock, and Bin Luo. propose converting graphs into pattern vectors by utilizing the spectral decomposition of the Laplacian matrix and basis sets for symmetric polynomials in "Pattern Vectors from Algebraic Graph Theory." The authors study various alternatives to embed these vectors in a low-dimensional space for clustering purpose. They show that this results in well-defined clusters of original graph structures.
A similar attempt is made in "Indexing Hierarchical Structures Using Graph Spectra" by Ali Shokoufandeh, Diego Macrini, Sven Dickinson, Kaleem Siddiqi, and Steven W. Zucker to sidestep the issue of computational complexity. Here, the authors describe a framework that embeds the topological structure of a directed acyclic graph into a low-dimensional vector space. The vector encoding of graph structure allows the use of nearest neighbor search to retrieve similar graph structures. The encoding algorithm can handle minor perturbations due to noise, occlusion, and node split/merge. The paper contains a series of indexing experiments in the domain of view-based 3D object recognition using shock graphs.
The last paper, titled "Generic Model Abstraction from Examples," by Yokov Keselman and Sven Dickinson addresses the problem of acquiring a generic 2D view-based class model from a set of images. The problem is cast in the form of a graph theoretic problem. The authors exploit certain aspects of their representation method to reduce the complexity of the involved search process. They present experimental results on real images.
We are grateful to the authors and reviewers who have made this a successful endeavor. In addition, we would like to thank Dr. Rama Chellappa, the former EIC of TPAMI, for the opportunity to bring this body of work to a large-scale readership and the editorial staff for their help in managing the submission and review process.
Mitra Basu Horst Bunke Alberto Del Bimbo Guest Editors
• M. Basu is with the CISE/CCF, National Science Foundation, 4201 Wilson Blvd., Arlington, VA 22230. E-mail: mbasu@nsf.gov.
• H. Bunke is with the Institute of Informatics and Applied Mathematics, University of Bern, Neubrueckstr. 10, CH-3012 Bern, Switzerland.
E-mail: bunke@iam.unibe.ch.
• A. Del Bimbo is with the Dipartimento di Sistemi e Informatica, Universitá degli Studi di Firenze, Via S. Marta 3-50139 Firenze, Italy.
E-mail: delbimbo@dsi.unifi.it.
For information on obtaining reprints of this article, please send e-mail to: tpami@computer.org.
Mitra Basu received the PhD degree in electrical and computer engineering from Purdue University. Dr. Basu was one of the last doctoral students of late Professor King-Sun Fu. Since then, she has been with the Electrical Engineering Department at the City College of the City University of New York (CUNY). She holds joint appointments at the New York Center for Biomedical Engineering and the Doctoral Faculty of Computer Science at CUNY. She works and publishes in the areas of pattern recognition, learning systems, computing in biological organisms, and bioinspired computing. She is the guest editor of a special issue on grammatical inference techniques and applications that will appear in
Pattern Recognition Journal in 2005. She is editing a book titled
Data Complexity in Pattern Recognition (Springer Verlag), also in 2005. Since 2002, she has been on loan to the US National Science Foundation (NSF). At the NSF, she directs Emerging Models and Technologies in Computation, a new funding program on bio, nano, and quantum computing that she initiated in the Computer, Information Science, and Engineering directorate.
Horst Bunke received the MS and PhD degrees in computer science from the University of Erlangen, Germany. In 1984, he joined the University of Bern, Switzerland, where he is a professor in the Computer Science Department. He was department chairman from 1992-1996, dean of the faculty of science from 1997 to 1998, and a member of the executive committee of the faculty of science from 2001 to 2003. From 1998 to 2000, he served as first vice president of the International Association for Pattern Recognition (IAPR). In 2000, he also was acting president of this organization. He is a fellow of the IAPR, former editor-in-charge of the
International Journal of Pattern Recognition and Artificial Intelligence, editor-in-chief of the journal
Electronic Letters of Computer Vision and Image Analysis, editor-in-chief of the book series on Machine Perception and Artificial Intelligence by World Scientific, an associate editor of
Acta Cybernetica, the
International Journal of Document Analysis and Recognition, and
Pattern Analysis and Applications. He has held visiting positions at the IBM Los Angeles Scientific Center (1989), the University of Szeged, Hungary (1991), the University of South Florida at Tampa (1991, 1996, 1998-2004), the University of Nevada at Las Vegas (1994), Kagawa University, Takamatsu, Japan (1995), and Curtin University, Perth, Australia (1999). He served as a cochair of the Fourth International Conference on Document Analysis and Recognition held in Ulm, Germany, 1997 and as a track cochair of the 16th and 17th International Conference on Pattern Recognition held in Quebec City, Canada and Cambridge, United Kingdom, in 2002 and 2004, respectively. Dr. Bunke has been on the program and organization committee of many other conferences and served as a referee for numerous journals and scientific organizations. He has published more than 450 publications, including 31 authored, coauthored, edited, or coedited books and special editions of journals.
Alberto Del Bimbo is a full professor of computer engineering at the University of Florence, Italy. From 1997 to 2000, he was the director of the Department of Systems and informatics at the University of Florence. From 1998 to the present, he has been the director of the Master in Multimedia program, and from 2000 to the present has been the deputy rector of the University of Florence, in charge of Research and Innovation Transfer. His scientific interests are pattern recognition, image databases, and multimedia. He has delved into object recognition and image sequence analysis, image and video database content-based retrieval, and advanced man-machine interaction. Professor Del Bimbo is the author of more than 180 publications that have appeared in the most distinguished international journals and conference proceedings. He is the author of the monograph
Visual Information Retrieval on content-based retrieval from image and video databases. He has also been the guest editor of many special issues on image databases in many highly respected journals. He was the general chairman of the Ninth IAPR International Conference Image Analysis and Processing, ICIAP '97, Florence 1997, and of the Sixth IEEE International Conference Multimedia Computing and Systems, ICMCS '99, Florence 1999. He was the president of IAPR from 1996 to 2000, member-at-large of IEEE Pubs Board from 1999 to 2001, and was appointed a fellow of the IAPR, the International Association for Pattern Recognition, in 2000. Professor Del Bimbo is presently an associate editor of
Pattern Recognition,
Journal of Visual Languages and Computing,
Multimedia Tools and Applications Journal,
Pattern Analysis and Applications,
IEEE Transactions on Multimedia, and
IEEE Transactions on Pattern Analysis and Machine Intelligence.