This Article 
 Bibliographic References 
 Add to: 
Perceptual Organization for Scene Segmentation and Description
June 1992 (vol. 14 no. 6)
pp. 616-635

A data-driven system for segmenting scenes into objects and their components is presented. This segmentation system generates hierarchies of features that correspond to structural elements such as boundaries and surfaces of objects. The technique is based on perceptual organization, implemented as a mechanism for exploiting geometrical regularities in the shapes of objects as projected on images. Edges are recursively grouped on geometrical relationships into a description hierarchy ranging from edges to the visible surfaces of objects. These edge groupings, which are termed collated features, are abstract descriptors encoding structural information. The geometrical relationships employed are quasi-invariant over 2-D projections and are common to structures of most objects. Thus, collations have a high likelihood of corresponding to parts of objects. Collations serve as intermediate and high-level features for various visual processes. Applications of collations to stereo correspondence, object-level segmentation, and shape description are illustrated.

[1] G. J. Agin and T. O. Binford, "Computer description of curved objects,"IEEE Trans. Comput., vol. C-25, Apr. 1976.
[2] F. Attneave, "Some informational aspects of visual perception,"Psycholog. Rev., vol. 61, pp. 183-193, 1954.
[3] D. H. Ballard, G. E. Hinton, and T. J. Sejnowski, "Parallel visual computation,"Nature, vol. 306, pp. 21-26, Nov. 1983.
[4] T. O. Binford, "Visual perception by computer," inProc. IEEE Conf. syst. Contr. (Miami), Dec. 1971.
[5] H. Blum, "Biological shape and visual science (Part 1),"J. Theoritical Biol., vol. 38, pp. 205-287, 1973.
[6] M. Boldt, R. Weiss, and E. Riseman, "Token based extraction of straight lines,"IEEE Syst. Man Cybern, vol. 19, no. 6, pp. 1581-1594, Dec. 1989.
[7] R. M. Bolle, A. Califano, R. Kjeldsen, and R. W. Taylor, "Visual recognition using concurrent and layered parameter networks," IBM Tech. Rep., Nov. 1988; IEEE Conf. Computer Vision and Pattern Recognition, 1989.
[8] M. Brady, "Criteria for representation of shapes," inHuman and Machine Vision(J. Beck, B. Hope, and A. Rosenfeld, Eds.). New York: Academic, 1983, pp. 39-84.
[9] M. Brady and H. Asada, "Smoothed local symmetries and their implementation,"Int. J. Robotics Res., vol. 3, pp. 36-61, Fall 1984.
[10] R. A. Brooks, "Goal-directed edge linking and ribbon finding," inProc. DARPA Image Understanding Workshop, Apr. 1979, pp. 72-78.
[11] R. A. Brooks, "Model-based three-dimensional interpretation of two-dimensional images,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-5, no. 2, pp. 140-150, Mar. 1983.
[12] J. F. Canny, "A computational approach to edge detection,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, pp. 679-697, 1986.
[13] J. H. Connell and M. Brady, "Generating and generalizing models of visual objects," Tech. Rep. AIM-823, Mass. Inst. Technol., July 1985.
[14] J. Dolan and R. Weiss, "Perceptual grouping of curved lines," inProc. DARPA Image Understanding Workshop(Palo Alto, CA), 1989, pp. 1135-1145.
[15] S. E. Fahlman and G. E. Hinton, "Connectionist architectures for artificial intelligence,"IEEE Comput. Mag., vol. 20, no. 1, pp. 100-109, 1987.
[16] S. E. Fahlman, G. E. Hinton, and T. J. Sejnowski, "Massively parallel architectures for AI: NETL, Thistle, and Boltzman machines," inProc. Nat. Conf. AI(Menlo Park, CA), 1983.
[17] T.-J. Fan, "Describing and recognizing 3-D objects using surface properties," Ph.D dissertation, University of Southern California, Tech. Rep. IRIS-237, Aug. 1988.
[18] J. A. Feldman and D. H. Ballard, "Connectionist models and their properties,"Cognitive Sci., pp. 205-254, 1982.
[19] M.A. Fischler and R. C. Bolles, "Perceptual organization and curve partitioning,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, no. 1, pp. 100-105, 1986.
[20] W. R. Garner,The Processing of Information and Structure. Potomac, MD: Erlbaum, 1974.
[21] W. R. Garner and D. E. Clement, "Goodness of pattern and pattern uncertainity,"J. Verbal Learning Verbal Behavior, vol. 2, pp. 446-452, 1963.
[22] S. L. Gazit and G. Medioni, "Multi-scale contour matching in a motion sequence," inProc. DARPA Image Understanding Workshop, May 1989.
[23] S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-6, no. 6, pp. 721-741, Nov. 1984.
[24] W. E. L. Grimson, "The combinatorics of heuristic search termination for object recognition in cluttered environment," inProc. First Euro. Conf. Comput. Vision, 1990, pp. 552-556.
[25] J. E. Hochberg and E. McAllister, "A quantitative approach to figural 'goodness,'"J. Experimental Psychol., vol. 46, pp. 361-364, 1953.
[26] J. J. Hopfield, "Neurons with graded response have collective computational properties like those of two-state neurons,"Proc. Nat. Acad. Sci. vol. 81, pp. 3088-3092, May 1984.
[27] J. J. Hopfield and D. W. Tank, "Neural networks and physical systems with emergent collective computational abilities,"Proc. Nat. Acad. Sci., vol. 79, pp. 2554-2558, Apr. 1982.
[28] J. J. Hopfield and D. W. Tank, "'Neural' computation of descisions in optimization problems,"Biol. Cybern., vol. 52, pp. 141-152, 1985.
[29] J. J. Hopfield and D. W. Tank, "Computing with neural circuits: A model,"Sci., vol. 233, pp. 625-633, 1986.
[30] R. A. Hummel and S. W. Zucker, "On the foundations of relaxation labeling process,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-5, no. 3, pp. 267-287, May 1983.
[31] D. J. Jacobs, "GROPER: A grouping based recognition system for two dimensional objects, " inProc. IEEE Comput. Soc. Workshop Comput. Vision(Miami Beach, FL), Dec. 1987.
[32] B. Julesz, "Figure and ground perception in briefly presented isodipole textures," inPerceptual Organization(Kubovy and Pomerantz, Eds.). Hillsdale, NJ: Lawrence Erlbaum, 1981, pp. 27-54.
[33] T. Kanade, "Recovery of the three-dimensional shape of an object from a single view," Tech. Rep. CMU-CS-79-153, Carnegie-Mellon Univ., Pittsburgh, PA, Oct. 1979.
[34] T. Kanade and J. R. Kender, "Mapping image properties into shape constraints: Skewed symmetry, affine-transformable patterns, and the shape-from-texture paradigm," inHuman and Machine Vision(J. Beck, B. Hope, and A. Rosenfeld, Eds.). New York: Academic, 1983, pp. 237-268.
[35] G. Kanisza,Organization in Vision. New York: Praeger, 1979.
[36] D. Katz,Gestalt Psychology: Its Nature and Significance. New York: Ronald, 1950.
[37] R. E. Kelly, P. R. M. McConnell, and S. J. Mildenberger, "The gestalt photomapping system,"J. Photogrammetric Eng. Remote Sensing, 1977.
[38] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, "Optimization by simulated annealing,"Sci., vol. 220, pp. 671-680, 1983.
[39] Kubovy and Pomerantz, Eds.,Perceptual Organization. Hillsdale, NJ: Lawrence Erlbaum, 1981.
[40] K. I. Laws, "Textured image segmentation," Ph.D. dissertation, Univ. Southern Calif., 1980.
[41] E. L. J. Leewburg, "A perceptual coding language for visual and auditory patterns,"Amer. J. Psychol., vol. 84, pp. 307-350, 1971.
[42] H. S. Lim and T. O. Binford, "Stereo correspondence: A hierarchical approach," inProc. DARPA Image Understanding Workshop(Los Angeles), Feb. 1987, pp. 234-241.
[43] D. Lowe, Perceptual Organization And Visual Recognition. Boston: Kluwer, 1985.
[44] D. G. Lowe and T. O. Binford, "Perceptual organization as a basis for visual recognition," inProc. AAAI-83(Washington, DC), Aug. 1983.
[45] J. Malik, "Interpreting line drawings of curved objects,"Int. J. Comput. Vision, vol. 1, no. 1, pp. 73-103, 1987.
[46] D. Marr,Vision. San Francisco: W. H. Freeman, 1982.
[47] G. Medioni and R. Nevatia, " Segment-based stereo matching,"Comput. Vision Graphics Image Processing, vol. 31, pp. 2-18, 1985.
[48] R. Mohan, "Application of neural constraint satisfaction networks to vision," inProc. Int. Joint Conf. Neural Networks(San Diego), 1989.
[49] R. Mohan, "Perceptual organization for computer vision," Ph.D. Thesis, IRIS Rep., Univ. Southern Calif., 1989.
[50] R. Mohan and R. Nevatia, "Segmentation and description based on perceptual organization," inProc. IEEE Conf. Comput. Vision Patt. Recogn. (San Diego, CA), June 1989.
[51] R. Mohan and R. Nevatia, "Using perceptual organization to extract 3-D structures,"IEEE Trans. Patt. Anal. Machine Intell., vol. 11, no. 10, Oct. 1989.
[52] L. R. Nackman and S. M. Pizer, "Three-dimensional shape description using the symmetric axis transform,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-7, pp. 187-201, Mar. 1985.
[53] R. Nevatia,Computer Analysis of Scenes of 3-D Curved Objects. Basel, Switzerland: Birkhauser-Verlag, 1976.
[54] R. Nevatia, K. Price, and G. Medioni, "USC image understanding research 1989-1990," inProc. DARPA Image Understanding Workshop(Pittsburgh), Sept. 1990.
[55] R. Nevatia and T. O. Binford, "Description and recognition of complexcurved objects,"Artificial Intell., vol. 8, pp. 77-98, 1977.
[56] S. E. Palmer, "The psychology of perceptual organization: A transformational approach," inHuman and Machine Vision(J. Beck, B. Hope, and A. Rosenfeld, Ed.). New York: Academic, 1983, pp. 269-339.
[57] J. Ponce, D. Chelberg, and W. Mann, "Analytical properties of generalized cylinders and their projections," inProc. DARPA Image Understanding Workshop(Los Angeles, CA), Feb. 1987.
[58] L. Quan, R. Mohr, and E. Thirion, "Generating the initial hypothesis using perspective invariants for a 2D image and 3D model matching." inProc. Int. Conf. Comput. Vision(Tarpon Springs, FL), Dec. 1988.
[59] K. Rao, "Shape description from sparse and imperfect data," Ph.D. Thesis, Tech. Rep. IRIS-250, Inst. Robotics Intell. Syst., Univ. Southern Calif., Dec. 1988.
[60] G. Reynolds and J. R. Beveridge, "Searching for geometric structure in images of natural scenes," inProc. DARPA Image Understanding Workshop(Los Angeles, CA), Feb. 1987.
[61] A. Rosenfeld, "Axial representation of shape,"Comput. Vision Graphics Image Processing, vol. 33, pp. 156-173, 1986.
[62] D. E. Rumelhartet al., Parallel Distributed Processing: Explorations in the Microstructures of Computing. Cambridge, MA: MIT Press, 1986.
[63] P. Saint-Marc and G. Medioni, "Adaptive smoothing for feature extraction," inProc. DARPA Image Understanding Workshop(Boston, MA), Apr. 1988.
[64] K. A. Stevens, "Computation of locally parallel structure,"Biol. Cybern., vol. 29, pp. 19-28, 1981.
[65] K. A. Stevens and A. Brookes, "Detecting structures by symbolic constructions on tokens,"Comput. Vision Graphics Image Processing, vol. 37, pp. 238-260, 1987.
[66] A. Triesman, "Perceptual grouping and attention in visual search for features and objects,"J. Experimental Psychol: Human Perception Performance, vol. 8, no. 2, pp. 194-214, 1982.
[67] M. Tuceryan and N. Ahuja, "Perceptual segmentation of nonhomogeneous dot patterns," inProc. Comput. Vision Patt. Recogn. Conf. (Washington, DC), June 1983, pp. 47-52.
[68] F. Ulupinar and R. Nevatia, "Using symmetries for analysis of shape from contour," inProc. 2nd ICCV, Tarpon Springs, FL, 1988, pp. 414-426.
[69] D. L. Waltz, "Understanding line drawings in scenes with shadows," inThe Psychology of Computer Vision(P. H. Winston, Ed.). New York: McGraw-Hill, 1975, pp. 19-91.
[70] M. Wertheimer, "Principles of perceptual organization," inReadings in Perception(D. Beardslee and M. Wertheimer, Eds.). Princeton, NJ: Princeton University Press, 1958, pp. 115-135.
[71] A. Witkin, "Scale-space filtering," inProc. Seventh Int. Joint Conf. Artificial Intell.(Kalsruhe, West Germany), 1983.
[72] A. P. Witkin and J. M. Tenenbaum, "On the role of structure in vision," inHuman and Machine Vision(J. Beck, B. Hope, and A. Rosenfeld, Eds.). New York: Academic, 1983, pp. 481-543.
[73] S. W. Zucker, "Computational and psychophysical experiments in grouping: Early orientation selection," inHuman and Machine Vision(J. Beck, B. Hope, and A. Rosenfeld, Eds.). New York: Academic, 1983, pp. 545-567.
[74] S. W. Zucker, "The diversity of perceptual grouping," inVision, Brain, and Cooperative Computation(M.A. Arbib and A.R. Hanson, Eds.). Cambridge, MA: MIT Press, 1987, pp. 231-262.

Index Terms:
computerised picture processing; computerised pattern recognition; scene segmentation; data-driven system; boundaries; surfaces; perceptual organization; geometrical regularities; edge groupings; collated features; abstract descriptors; 2-D projections; stereo correspondence; shape description; computerised pattern recognition; computerised picture processing
R. Mohan, R. Nevatia, "Perceptual Organization for Scene Segmentation and Description," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 6, pp. 616-635, June 1992, doi:10.1109/34.141553
Usage of this product signifies your acceptance of the Terms of Use.