• Publication
  • 1994
  • Issue No. 1 - January
  • Abstract - Part II: 3-D Object Recognition and Shape Estimation from Image Contours Using B-splines, Shape Invariant Matching, and Neural Network
 This Article 
 Bibliographic References 
 Add to: 
Part II: 3-D Object Recognition and Shape Estimation from Image Contours Using B-splines, Shape Invariant Matching, and Neural Network
January 1994 (vol. 16 no. 1)
pp. 13-23

This paper is the second part of a 3-D object recognition and shape estimation system that identifies particular objects by recognizing the special markings (text, symbols, drawings, etc.) on their surfaces. The shape of the object is identified from the image curves using B-spline curve modeling as described in Part I, as well as a binocular stereo imaging system. This is achieved by first estimating the 3-D control points from the corresponding curves in each image in the stereo imaging system. From the 3-D control points, the 3-D object curves are generated, and these are subsequently used for estimating the 3-D surface parameters. A Bayesian framework is used for classifying the image into one of c possible surfaces based on the extracted 3-D object curves. This is complemented by a neural network (NN) that recognizes the surface as a particular object (e.g., a Pepsi can versus a peanut butter jar), by reading the text/markings on the surface. To reduce the amount of training the NN has to undergo for recognition, the object curves are "unwarped" into planar curves before the matching process. This eliminates the need for templates that are surface shape dependent and results in a planar curve that might be a rotated, translated, and scaled version of the template. Hence, for the matching process we need to use measures that are invariant to these transformations. One such measure is the Fourier descriptors (FD) derived from the control points associated with the unwarped parent curves. The approach is tried on a variety of images of real objects and appears to hold great promise.

[1] B. Julesz,Foundations of Cyclopean Perception. Chicago: Univ. of Chicago Press, 1971.
[2] J. M. Brady, "Preface--The changing shape of computer vision,"Artificial Intell., vol. 17, pp. 1-15, Aug. 1981.
[3] D. Marr,Vision. San Francisco: W. H. Freeman, 1982.
[4] H. K. Nishihara, "Representation of the spatial organization of three-dimensional shapes for visual recognition," Ph.D. Dissertation, Massachusetts Inst. of Technol., Cambridge, MA, 1978.
[5] T. Poggioet al., "The MIT vision machine," inProc. Image Understanding Workshop, 1988, pp. 177-198.
[6] J. Aloimonos and D. Shulman,Integration of Visual Modules: An Extension of the Marr Paradigm. Boston, MA: Academic, 1989.
[7] M. L. Moerdler, "Multiple shape-from-texture into texture analysis and surface segmentation," inProc. IEEE Int. Conf. Computer Vision, 1988.
[8] Jean-Yves Herve and J. Aloimonus, "Shading into texture and texture into shading: An active approach," inProc. 1st European Conf. Computer Vision, Antibes, France, Apr. 1990.
[9] B. K. P. Horn, "Obtaining shape from shading information," inPsygology of Computer Vision, P. H. Winston Ed. New York: MacGraw-Hill, 1975, Chap. 4.
[10] B. K. P. Horn and R. W. Sjoberg, "Calculating the reflectance map,"Appl. Opt., vol. 18, June 1979.
[11] A. P. Pentland, "Finding the illuminant direction,"J. Opt. Soc. Amer., vol. 72, Apr. 1982.
[12] M. J. Brooks and B. K. P. Horn, "Shape and source from shading," inProc. Int. Joint Conf. on Artificial Intelligence, Los Angeles, CA, Aug. 1985.
[13] R. M. Belle and D. B. Cooper, "Bayesian recognition of local 3-D shape by approximating image intensity functions with quadric polynomials,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-6, July 1984.
[14] B. Cernushi-Frias and D. B. Cooper, "Estimation of location and orientation of 3-D surfaces using a single 2-D image,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-6, July 1984.
[15] D. Marr, "Analysis of occluding contour,"Proc. Roy. Soc., Ser. B, vol. 197, 1977.
[16] K. A. Stevens, "The visual interpretation of visual contours,"Artificial Intell., vol. 17, Aug. 1981.
[17] R. D. Rimey and F. S. Cohen, "A maximum-likelihood approach for segmenting range data,"IEEE Trans. Robotics Automat., vol. 4, no. 3, June 1988.
[18] J. J. Gibson,The Perception of the Visual World. Boston: Houghton Mifflin, 1950.
[19] K. Bajcsy and L. Lieberman, "Texture gradients as a depth cue,"Comp. Graphics and Image Processing 5, 1976.
[20] A. P. Witkin, "Recovering surface shape and orientation from texture,"Artificial Intell., vol. 17, 1981.
[21] K. Ikeuchi, "Shape from regular patterns,"Artificial Intell., vol. 22, 1984.
[22] J. R. Kender, "Shape from texture: A computational paradigm," inProc. Image Understanding Workshop, May 1979.
[23] Y. Ohta, K. Maenobu, and T. Sakai, "Obtaining surface orientation from texels under perspective projection," inProc. IJCAI, 1980.
[24] F. S. Cohen and M. S. Patel, "Modeling and synthesis of images of 3-D textured surfaces,"Computer Vision, Graphics and Image Processing, Sept. 1991.
[25] F. S. Cohen and M. S. Patel, "Shape from texture using Gaussian Markov random fields," inTheory and Applications of Markov Random Fields for Image Processing, R. Chellappa and A. K. Jain, Eds. New York: Academic Press, to appear.
[26] L. L. Struto and J. B. Lehman "Robot vision," Draper Lab., Int. Rep. R-635, 1973.
[27] H. H. Baker and T. O. Binford, "Depth from edge and intensity based stereo," inProc. 7th Int. Joint Conf. on Artificial Intelligence, Vancouver, Canada, 1981.
[28] D. Marr and T. Poggio, "A theory of human stereo vision,"Proc. Roy. Soc., vol. B-204, 1979.
[29] W. E. Grimson, "A computer implementation of a theory of human stereo vision,"Philosophical Trans. of the Roy. Soc., vol. B-292, 1981.
[30] B. Cernushi-Frias, D. B. Cooper, Y. P. Yung, and P. N. Belhumeur, "Toward a model-based Bayesian theory for estimating and recognizing parametrized 3-D objects using two or more images taken from different positions,"IEEE Trans. Patt. Anal. Machine Intell., vol. 11, no. 10, pp. 1028-1052, Oct. 1989.
[31] N. Kehtarnavaz and R. J. P. DeFigueiredo, "A framework for surface reconstruction from 3-D contours,"Computer Vision, Graphics and Image Processing, no. 42, pp. 32-47, 1988.
[32] I. Weiss, "3-D shape representation by contours,"Computer Vision, Graphics and Image Processing, no. 41, pp. 80-100, 1988.
[33] S. K. Yuen, "Shape from contour using symmetry," inProc. 1st European Conf. Computer Vision, Antibes, France, April 1990.
[34] M. Brady and A. Yuille, "An extremum principle for shape from contour,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-6, no. 3, pp. 288-301, May 1984.
[35] C. de Boor, "On calculation with B-splines,"J. Approx. Theory, vol. 6, pp. 50-62, 1972.
[36] C. de Boor,A Practical Guide to Splines. New York: Springer, 1978.
[37] D. F. Rogers and J. A. Adams,Mathemational Elements for Computer Graphics, 2nd ed. New York: McGraw-Hill, 1990.
[38] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representation by error propagation,"Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vols. 1 and 2. Cambridge, MA: MIT Press, 1986.
[39] R. Hecht-Nielsen, "Theory of backpropagation neural network," inProc. IEEE-IJCNN89(Washinghton DC), 1989, pp. 593-605, vol. I.
[40] R. O. Duda and P. E. Hart,Pattern Classification and Scene Analysis. New York: Wiley, 1973.
[41] P. D. Sampson, "Fitting conic sections to 'very scattered' data: An iterative refinement of the Bookstein algorithm,"Computer Graphics and Image Processing, vol. 18, 1982.
[42] E. Persoon and K. Fu, "Shape discrimination using Fourier descriptors,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-8, no. 3, pp. 388-397, May 1986.
[43] C. T. Zahn and R. Z. Roskies, "Fourier descriptors for plane closed curves,"IEEE Trans. Comput., vol. C-21, pp. 269-281, March 1972.
[44] R. P. Lippman, "An introduction to computing with neural nets,"IEEE ASSP Msg., vol. 4, pp. 4-22, 1987.
[45] Y. H. Pao,Adaptive Pattern Recognition and Neutral Networks, Reading, MA: Addison-Wesley, 1989.
[46] D.B. Cooper, Y.P. Hung, and G. Taubinl, "A new model-based stereo approach for 3D surface reconstruction using contours on the surfaces pattern, inProc. Second Int. Conf. Comput. Vision, Dec. 1988.
[47] R. S. Millman and G. D. Parher,Elements Differential Geometry. Englewood Cliffs, NJ: Prentice-Hall, 1977.

Index Terms:
splines (mathematics); neural nets; image recognition; Bayes methods; stereo image processing; image sequences; 3-D object recognition; shape estimation; image contours; B-splines; shape invariant matching; neural network; image curves; curve modeling; binocular stereo imaging system; Bayesian framework; training; matching; Fourier descriptors; unwarped parent curves
F.S. Cohen, Jin-Yinn Wang, "Part II: 3-D Object Recognition and Shape Estimation from Image Contours Using B-splines, Shape Invariant Matching, and Neural Network," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 1, pp. 13-23, Jan. 1994, doi:10.1109/34.273720
Usage of this product signifies your acceptance of the Terms of Use.