This Article 
 Bibliographic References 
 Add to: 
Put: Language-Based Interactive Manipulation of Objects
March 1996 (vol. 16 no. 2)
pp. 31-39
Describing a scene to a computer is an inherent task of computer graphics applications. Modeled scenes are typically built with direct placement techniques or specialized scripting languages. The scene description task could be greatly eased if natural language were an interactive control option. However, natural language understanding is notoriously difficult for computers. This difficulty is exacerbated in the case of computer graphics by the need for geometric output, not just "conceptual understanding" or high-level inferencing. General text-understanding techniques have not been successfully applied to scene generation. Typically, a few task-specific commands, such as "walk," are implemented as an ad-hoc collection of procedures. Our approach aims to separate the expressive power of fundamental natural concepts from the difficult task of text understanding. We are developing a 3D object placement system based on a combination of natural commands and interactive techniques. Guided by research in cognitive linguistics, we use basic spatial relationships--such as in, on, and at--and fundamental scene parameters--such as viewer location and object dimensionality--to identify regions of placement for objects in a scene. These natural commands can be used to quickly prototype a complex scene and constrain object placement.

1. N. Badler, Bonnie L. Webber, Jugal Kelita, Jeffrey Esakov, "Animation from Instructions," in Making Them Move, N. Badler, B. Barsky, and D. Zeltzer, eds., Morgan Kaufmann San Francisco, Calif., 1991, pp. 51-98.
2. G. Lakoff, Women, Fire, and Dangerous Things: What CategoriesReveal About the Mind, Univ. of Chicago Press, Chicago, 1987.
3. M. Gleicher, "Integrating Constraints and Direct Manipulation," Proc. 1992 ACM Symp. Interactive 3D Graphics, ACM, New York, 1992, pp. 171-174.
4. J. Wilhelms and R. Skinner, "A Notion for Interactive Behavior Animation Control," IEEE CG&A, Vol. 10, No. 3, May 1990, pp. 14-22.
5. D. Zeltzer, "Task-level Graphical Simulation: Abstraction, Representation, andControl," in Making Them Move, Morgan Kaufmann San Francisco Calif., 1991, pp. 3-34.
6. N.I. Badler, C.B. Phillips, and B.L. Webber, Simulating Humans: Computer Graphics Animation and Control, Oxford Univ. Press, New York, 1993.
7. C. Schmandt, M. Ackerman,, and D. Hindus,"Augmenting a Window System with Speech Input," Computer, vol. 23, no. 8, Aug. 1990, pp. 50-56.
8. L. Talmy, "How Language Structures Space," in Spatial Orientation: Theory, Research, and Applications, H. Picks and L. Aeredolo, eds., Plenum Press, New York, 1983, pp. 225-232.
9. R.W. Langacker, Foundations of Cognitive Grammar: Theoretical Prerequisites, Vol. I, Stanford Univ. Press, Stanford, Calif., 1987.
10. B. Landau and R. Jackendoff, "What and Where in Spatial Language and Spatial Cognition," Behavior and Brain Sciences, Vol. 16, No. 2, 1991, pp. 217-265.
11. B. Tversky, "Spatial-Mental Models," The Psychology of Learning and Motivation, Vol. 27, 1991, pp. 109-145.

Index Terms:
interactive 3D graphics, linguistic interfaces to computers, scene building, object placement
Sharon Rose Clay, Jane Wilhelms, "Put: Language-Based Interactive Manipulation of Objects," IEEE Computer Graphics and Applications, vol. 16, no. 2, pp. 31-39, March 1996, doi:10.1109/38.486678
Usage of this product signifies your acceptance of the Terms of Use.