The Community for Technology Leaders
RSS Icon
Issue No.01 - January-March (2006 vol.28)
pp: 46-55
Ralph Weischedel , BBN Technologies
Natural-language understanding has evolved from its earliest days at Bolt Beranek and Newman, in which scientists use an early approach to parsing, to more sophisticated techniques that enable systems to extract information from open-domain text sources to fill data bases automatically.
natural language processing, question answering, semantic, networks, parsing, knowledge representation
Ralph Weischedel, "Natural-Language Understanding at BBN", IEEE Annals of the History of Computing, vol.28, no. 1, pp. 46-55, January-March 2006, doi:10.1109/MAHC.2006.17
1. M.Mitchell Waldrop, The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal, Viking Press, 2001.
2. L. Beranek, "BBN's Earliest Days: Founding a Culture of Engineering Creativity," IEEE Annals of the History of Computing, vol. 27, no. 2, 2005, pp. 6-14.
3. J. Swets, "The ABC's of BBN: From Acoustics to Behavioral Sciences to Computers," IEEE Annals of the History of Computing, vol. 27, no. 2, 2005, pp. 15-29.
4. T. Marill, Statistical Recognition Function and the Design of Pattern Recognizers, tech. report 847, BBN, 1 Feb. 1961.
5. D. Bobrow, Syntactic Analysis of English by Computer—A Survey, tech. report 1055, BBN, 8 Aug. 1963.
6. F. Black, A Question-Answering System: QAS-5, tech. report 1063, BBN, 1 Oct. 1963.
7. Bobrow's PhD research with its Student system also had a natural-language component; see D.G. Bobrow, "Natural Language Input for a Computer Problem-Solving System," Semantic Information Processing, M. Minsky ed. MIT Press, 1968, pp. 133-215.
8. D. Bobrow, J. Fraser, and M.R. Quillian, Survey of Automated Language Processing, tech. report 1490, BBN, 1 Apr. 1967; D. Bobrow and J. Fraser, A Phonological Rule Tester, tech. report 1589, BBN, 1 Jan. 1968.
9. M.R. Quillian, "Semantic Memory," Semantic Information Processing, M. Minsky, ed., MIT Press, 1968, p. 262.
10. A.M. Collins and M.R. Quillian, "Retrieval Time From Semantic Memory," J. Verbal Learning and Verbal Behavior, vol. 8, 1969, pp. 240-247; A.M. Collins and M.R. Quillian, "How to Make a Language User," Organization and Memory, E. Tulving and W. Donaldson, eds., Academic Press, 1972, pp. 310-354.
11. W.A. Woods, "Transition Network Grammars for Natural Language Analysis," Comm. ACM, vol. 3, no. 10, 1970, pp. 591-606; reprinted in Readings in Natural Language Processing, B.J. Grosz, K. Sparck-Jones, and B.L. Webber, eds., Morgan Kaufmann, 1986, pp. 71-88.
12. J. Earley, "An Efficient Context-Free Parsing Algorithm," Comm. ACM, vol. 13, no. 2, 1970, pp. 94-102.
13. Such as Backus-Naur notation that computer language designers frequently use to describe their languages.
14. W.A. Woods, R.M. Kaplan, and B. Nash-Webber, The Lunar Sciences Natural Language Information System: Final Report,1 June 1972; W.A. Woods, "Semantics and Quantification in Natural Language Question Answering," Advances in Computers, vol. 17, M. Yovits, ed., Academic Press, 1978, pp. 2-64.
15. J.J. Wolf and W.A. Woods, "The HWIM Speech Understanding System," Trends in Speech Recognition, W.A. Lea, ed., Prentice Hall, 1980, pp. 316-339.
16. R.R. Burton and J.S. Brown, "Toward a Natural-Language Capability for Computer-Aided Instruction," Procedure for Instructional Systems Development, H. O'Neil, ed., Academic Press, 1979, pp. 273-313.
17. W.A. Woods, "What's in a Link: Foundations for Semantic Networks," Representation and Understanding: Studies in Cognitive Science, D.G. Bobrow and A.M. Collins, eds., Academic Press, 1975, pp. 35-82.
18. R.J. Brachman, A Structural Paradigm for Representing Knowledge, doctoral dissertation, Applied Mathematics Dept., Harvard Univ., 1977.
19. Rusty Bobrow was a major participant—more about him later.
20. R.J. Brachman and J. Schmolze, "An Overview of the KL-ONE Knowledge Representation System," Cognitive Science, vol. 9, no. 2, 1985, pp. 171-216.
21. F. Baader et al., eds., The Description Logic Handbook: Theory Implementation, and Applications, Cambridge Univ. press, 2003.
22. Later when BBN's speech and natural-language processing groups were joined together Bates was appointed deputy department manager for the natural language part of the combined department.
23. In time Bates Bobrow, and others received a patent relating to this work: L. Bates R. Bobrow Systems and Methods for Providing User Assistance in Retrieving Data From a Relational Database, US patent 6,023,697 Patent and Trademark Office 2000.
24. Rusty had not named his program. It was first called RUS for Render Unto Syntax by Dick Burton who was writing a paper. Burton presumably was thinking of Rusty's penchant for saying "Render unto syntax that which is syntax, and render unto semantics that which is semantics." No doubt Burton also saw the pun on Rusty's name (related during a 26 April 2004 phone conversation with Rusty Bobrow).
25. I joined BBN in 1984 leaving a faculty position at the University of Delaware where among other things I was working to augment ATNs with more semantics (R.M. Weischedel, "A New Semantic Computation While Parsing: Presupposition and Entailment," Syntax and Semantics II: Presupposition, C. Oh and D. Dineen eds. Academic Press 1979 pp. 155-182). I was introduced to BBN by Rusty Bobrow and hired into the AI department by the then department manager Walter Reitman. In short order I was involved in the natural-language understanding group's efforts particularly with regard to DARPA's Strategic Computing Program.
26. R.M. Weischedel, "A Hybrid Approach to Representation in the Janus Natural Language Processor," Proc. 27th Ann. Meeting of the Assoc. for Computational Linguistics, Assoc. for Computational Linguistics, 1989, pp. 193-202.
27. D.M. Ayuso, V. Shaked, and R.M. Weischedel, "An Environment for Acquiring Semantic Information," Proc. 25th Ann. Meeting of the Assoc. for Computational Linguistics, Assoc. for Computational Linguistics, 1987, pp. 32-40.
28. R. Bobrow, P. Resnik, and R. Weischedel, "Multiple Underlying Systems: Translating User Requests Into Programs to Produce Answers," Proc. 28th Ann. Meeting of the Assoc. for Computational Linguistics, Assoc. for Computational Linguistics, 1990, pp. 227-234.
29. S. Levy, "The History of Technology Transfer at BBN," IEEE Annals of the History of Computing, vol. 27, no. 2, 2005, pp. 30-38.
30. B.J. Grosz and C.L. Sidner, "Attention, Intentions, and the Structure of Discourse," Computational Linguistics, vol. 12, no. 3, 1986, pp. 175-204.
31. C. Sidner, "Focusing in the Comprehension of Definite Anaphora," Computational Models of Discourse, M. Brady and R. Berwick, eds., MIT Press, 1983, pp. 267-330.
32. And sponsored by BBN's Science Development Program described by D. Walden, A Culture of Innovation: Insider Accounts of Computing and Life at BBN, D. Walden and R. Nickerson eds. to be published.
33. M. Bates and R. Weischedel eds. Challenges in Natural Language Processing, Cambridge Univ. Press, 1993.
34. Also about this time I became leader of BBN's natural-language R&D group.
35. B. Santorini, Annotation Manual for the Penn TREEBANK Project, tech. report, CIS Dept., Univ. of Pennsylvania, 1990.
36. R. Weischedel et al., "Coping with Ambiguity and Unknown Words through Probabilistic Models," Computational Linguistics, June 1993, pp. 359-382.
37. Miller provides another example of the benefits of BBN's ongoing connections with universities. For example, John Makhoul has, as an adjunct professor at Northeastern University, set up a path that made it relatively easy to provide research funding to graduate students at Northeastern. One such student, Scott Miller, was interested in applying statistical learning to natural-language understanding, and I provided initial research funding for him to do work in this area. He wrote his PhD thesis on statistical language models for spoken dialogue interfaces under John Makhoul and Rich Schwartz, then rejoined the research on text understanding as an employee.
38. An earlier draft of this article covered continuing innovative work done in the late 1990s and in the current decade. That work, not yet historical enough, was elided from this article.
59 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool