This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
The Application of Semantic Classification Trees to Natural Language Understanding
May 1995 (vol. 17 no. 5)
pp. 449-460

Abstract—This article describes a new method for building a natural language understanding (NLU) system, in which the system’s rules are learnt automatically from training data. The method has been applied to design of a speech understanding (SU) system. Designers of such systems rely increasingly on robust matchers to perform the task of extracting meaning from one or several word sequence hypotheses generated by a speech recognizer; a robust matcher processes semantically important islands of words and constituents rather than attempting to parse the entire word sequence. We describe a new data structure, the Semantic Classification Tree (SCT), that learns semantic rules from training data and can be a building block for robust matchers for NLU tasks. By reducing the need for handcoding and debugging a large number of rules, this approach facilitates rapid construction of an NLU system. In the case of an SU system, the rules learned by an SCT are highly resistant to errors by the speaker or by the speech recognizer because they depend on a small number of words in each utterance. Our work shows that semantic rules can be learned automatically from training data, yielding successful NLU for a realistic application.

[1] L. Bahl,P. Brown,P. De Souza,, and R. Mercer,“A tree-based statistical language model for natural language speech recognition,” Readings in Speech Recognition, A. Waibel and K.-F. Lee, eds., Morgan-Kaufmann, pp. 507-514, 1990.
[2] E. Black,F. Jelinek,J. Lafferty,D. Magerman,R. Mercer,, and S. Roukos,“Towards history-based grammars: Using richer models for probabilistic parsing,” Proc. 31st Ann. Meeting Assoc. Computational Linguistics, pp. 31-37, June 1993.
[3] E. Black,R. Garside,, and G. Leech,Statistically Driven Computer Grammars of English: The IBM/Lancaster Approach, Rodopi Press, Amsterdam-Atlanta, 1993.
[4] E. Black,F. Jelinek,J. Lafferty,R. Mercer,, and S. Roukos,“Decision tree models applied to the labeling of text with parts of speech,” Proc. 1992 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, pp. 117-121, Feb.23-26, 1992.
[5] L. Breiman,J. Friedman,R. Olshen,, and C. Stone,Classification and Regression Trees,Wadsworth, 1984.
[6] P. Chou,“Optimal partitioning for classification and regression trees,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 4, pp. 340-354, Apr. 1991.
[7] S. Crawford,R. Fung,L. Appelbaum,, and R. Tong,“Classification trees for information retrieval,” Proc. 8th Int’l Workshop Machine Learning, NorthwesternUniv., Evanston, Ill., 1991.
[8] Proc. 1990 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, June 1990.
[9] Proc. 1991 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, Feb.19-22, 1991.
[10] Proc. 1992 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, Feb.23-26, 1992.
[11] K.S. Fu and T. Booth,“Grammatical inference: Introduction and survey,” IEEE Trans. Pattern Analysis and MachineIntelligence, vol. 8, no. 3, pp. 343-375, May 1986.
[12] K.S. Fu,“A step towards unification of syntactic and statistical pattern recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 3, pp. 398-404, May 1986.
[13] S. Gelfand,C. Ravishankar,, and E. Delp,“An iterative growing and pruning algorithm for classification tree design,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 2, pp. 163-174, Feb. 1991.
[14] L. Hirschman,“Multi-site data collection for a spoken language corpus,” Proc. 1992 Int’l Conf. Spoken Language Processing, vol. 2, pp. 903-906, Oct. 1992.
[15] E. Jackson,D. Appelt,J. Bear,R. Moore,, and A. Podlozny,“A template matcher for robust NL interpretation,” Proc. 1991 DARPA Speech and Natural Language Workshop, Morgan Kaufmann Inc., pp. 190-194, Feb.19-22, 1991.
[16] F. Jelinek,“Self-organized language modeling for speech recognition,” in Readings in Speech Recognition, A. Waibel, and K.-F. Lee, eds., Morgan Kaufmann, pp. 450-506, 1990.
[17] R. Kuhn, and R. De Mori,“Learning speech semantics with keyword classification trees,” Proc. ICASSP 93, vol. II, pp. 55-58, Apr. 1993.
[18] R. Kuhn,Keyword Classification Trees for Speech Understanding Systems, PhD thesis, McGill Univ., June 1993.
[19] C. Ling,S. Cherwenka,, and M. Marinov,“A symbolic model for learning the past tenses of English verbs,” IJCAI 93, Aug.28- Sept.3, 1993, vol. 2, pp. 1143-1149.
[20] S. Miller,R. Schwartz,R. Bobrow,, and R. Ingria,“Statistical language processing using hidden understanding models,” Spoken Language Systems Technology Workshop,Princeton, N.J., Mar. 1994.
[21] E. Millien and R. Kuhn,“A robust analyzer for spoken language understanding,” Eurospeech 93, vol. II, pp. 1331-1334,Berlin, Sept. 1993.
[22] R. Pieraccini,E. Levin,, and C.-H. Lee,“Stochastic representation of conceptual structure in the ATIS task,” Proc. 1991 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, pp. 121-124, Feb.19-22, 1991.
[23] R. Pieraccini,E. Tzoukermann,Z. Gorelov,J.-L. Gauvain,E. Levin,C.-H. Lee,, and J. Wilpon,“A speech understanding system based on statistical representation of semantics,” Proc. ICASSP 92, pp. I-193-I-196, Mar. 1992.
[24] R. Pieraccini and E. Levin,“A Learning Approach to Natural Language Understanding,” to be published in New Advances and Trends in Speech Recognition and Coding (NATO ASI), A. Rubio, ed. Springer Verlag.
[25] P. Price,“Evaluation of spoken language systems: The ATIS domain,” Proc. 1990 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, pp. 91-95, June 1990.
[26] L. Rabiner,“A tutorial on hidden Markov models and selected applications in speech recognition,” Readings in Speech Recognition, A. Waibel and K.-F. Lee, eds., Morgan Kaufmann, pp. 267-296, 1990.
[27] S. Seneff,“A relaxation method for understanding spontaneous speech utterances,” Proc. 1992 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, pp. 299-304, Feb.23-26, 1992.
[28] Readings in Speech Recognition, A. Waibel and K.-F. Lee, eds., Morgan Kaufmann, 1990.
[29] W. Ward,“Evaluation of the CMU ATIS system,” , Proc. 1991 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, pp. 101-105, Feb.19-22, 1991.
[30] W. Ward,S. Issar,X. Huang,H. Hon,M. Hwang,S.R. Young,M. Matessa,R. Stern,, and F. Liu,“Speech understanding in open tasks,” Proc. 1992 DARPA Speech and Natural Language Workshop, Morgan Kaufmann, pp. 78-83, Feb.23-26, 1992.
[31] W. Ward and S. Young,“Flexible use of semantic constraints in speech recognition,” Proc. ICASSP 93, vol. II, pp. 49-50,Minneapolis, Apr. 1993.

Index Terms:
Speech understanding, semantic classification tree, SCT, machine learning, natural language, decision tree.
Citation:
Roland Kuhn, Renato De Mori, "The Application of Semantic Classification Trees to Natural Language Understanding," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 5, pp. 449-460, May 1995, doi:10.1109/34.391397
Usage of this product signifies your acceptance of the Terms of Use.