This Article 
 Bibliographic References 
 Add to: 
Viewpoint Invariant Recovery of Visual Surfaces from Sparse Data
September 1992 (vol. 14 no. 9)
pp. 897-909

An algorithm for the reconstruction of visual surfaces from sparse data is proposed. An important aspect of this algorithm is that the surface estimated from the sparse data is approximately invariant with respect to rigid transformation of the surface in 3D space. The algorithm is based on casting the problem as an ill-posed inverse problem that must be stabilized using a priori information related to the image and constraint formation. To form a surface estimate that is approximately invariant with respect to viewpoint, the stabilizing information is based on invariant surface characteristics. With appropriate approximations, this results in a convex functional to minimize, which is then solved using finite element analysis. The relationship of this algorithm to several previously proposed reconstruction algorithms is discussed, and several examples that demonstrate its effectiveness in reconstructing viewpoint-invariant surface estimates are given.

[1] W. E. L. Grimson,From Images to Surfaces: A Computational Study of the Human Early visual System. Cambridge, MA: MIT Press, 1981.
[2] B. Julesz, "Binocular depth perception of computer-generated patterns,"Bell Syst. Techn. J., vol. 39, pp. 1125-1162, 1960.
[3] P. Allen,Object Recognition Using Vision and Touch. Philadelphia: Univ. of Pennsylvania, 1985.
[4] A. Blake and A. Zisserman,Visual Reconstruction. Cambridge, MA: MIT Press, 1987.
[5] T. E. Boult and J. R. Kender, "Visual surface reconstruction using sparse depth data," inProc. 1986 IEEE Comput. Vision Patt. Recogn. Conf. (Miami), June 22-26, 1986, pp. 68-76.
[6] J. R. Kender, D. Lee, and T. Boult, "Information based complexity applied to optimal recovery of the 2 1/2 D sketch," inProc. Third IEEE Workshop Comput. Vision: Representation Contr.(Bellaire, MI), Oct. 13-16, 1985, pp. 157-167.
[7] J. Mayhew, "The interpretation of stereo information: The computation of surface orientation and depth,"Perception, vol. 11, pp. 387-403, 1982.
[8] S. S. Sinha and B. G. Schunck, "Discontinuity-preserving surface reconstruction in vision processing," inProc. Conf. Comput. Vision Patt. Recogn., 1989.
[9] D. Terzopoulos, "Multilevel computational processes for visual surface reconstruction,"Comput. Vision Graphics Image Processing, vol. 24, no. 1, pp. 52-96, Oct. 1983.
[10] D. Terzopoulos, "Multilevel reconstruction of visual surfaces: Variational principles and finite-element representations," inMultiresolution Image Processing and Analysis(A. Rosenfeld, Ed.). ___New York, NY: Springer-Verlag, 1984, pp. 237-310.
[11] D. Terzopoulos, "Regularization of inverse visual problems involving discontinuities,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, no. 4, pp. 413-424, July 1986.
[12] D. Terzopoulos, "The computation of visible-surface representations,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-10, no. 4, pp. 417-438, July 1988.
[13] R. Szeliski, "Fast surface interpolation using hierarchical basis functions," inProc. 1989 Conf. Comput. Vision Patt. Recogn. (San Diego, CA), June 4-8, 1989, pp. 222-228.
[14] J. Marroquin, S. Mitter, and T. Poggio, "Probabilistic solution of ill-posed problems in computational vision,"J. Amer. Stat. Assoc., vol. 82, no. 397, pp. 76-89, Mar. 1987.
[15] J. G. Harris, "A new approach to surface reconstruction: The coupled depth/slope model," inProc. First Int. Conf. Comput. Vision(London, England), June 8-11, 1987, pp. 277-283.
[16] C. Chu and A. C. Bovik, "Visual surface reconstruction using minimax approximation,"Patt. Recogn., vol. 21, no. 4, pp. 303-312, 1988.
[17] J. Y. Jou and A. C. Bovik, "Improving visible-surface reconstruction," inProc. IEEE Conf. Computer vision and Pattern Recognition, June 1988, pp. 138-143.
[18] J. Jou and A. C. Bovik, "Improved initial approximation and intensity-guided discontinuity detection in visible-surface reconstruction,"Comput. Vision Graphics Image Processing, vol. 47, no. 3, pp. 292-325, Sept. 1989.
[19] A. Blake and A. Zisserman, "Invariant surface reconstruction using weak continuity constraints," inProc. 1986 IEEE Comput. Vision Patt. Recogn. Conf.(Miami, FL), June 22-26, 1986, pp. 62-67.
[20] D. Lee and T. Pavlidis, "One-dimensional regularization with discontinuities,"IEEE Trans. Patt. Anal. Machine Intell., vol. 10, no. 6, pp. 822-829, Nov. 1988.
[21] G. Wahba, "Three topics in ill-posed problems," inInverse and Ill-Posed Problems(H. W. Engl, C. W. Engl, and C. W. Groetsch, Eds.). Orlando, FL: Academic, 1987, pp. 37-52, vol. 4.
[22] R. L. Stevenson and E. J. Delp, "Invariant recovery of curves inm- dimensional space from sparse data,"J. Opt. Soc. Amer. A, Mar. 1990.
[23] R. L. Stevenson and E. J. Delp, "Invariant reconstruction of visual surfaces," inProc. IEEE Workshop Interpretation 3D Scenes(Austin, TX), Nov. 27-29, 1989, pp. 131-137.
[24] T. Poggio, V. Torre, and C. Koch, "Computational vision and regularization theory,"Nature, vol. 317, pp. 314-319, 1985.
[25] A. N. Tikhonov and V. Y. Arsenin,Solutions of Ill-Posed Problems. Washington, DC: V. H. Winston, 1977.
[26] A. N. Tikhonov and A. V. Goncharsky, Eds.,Ill-Posed Problems in the Natural Sciences. Moscow: MIR, 1987.
[27] P. Besl and R. Jain, "Invariant surface characteristics for 3-D object recognition in range images,"Comput. Vision Graphics Image Processing, 1986, pp. 33-80, vol. 33.
[28] H. W. Guggenheimer,Differential Geometry. New York: Dover, 1977.
[29] M. M. Lipschutz,Differential Geometry. New York, NY: McGraw-Hill, 1969.
[30] P. T. Sander, "On reliably inferring differential structure from three-dimensional images," Ph.D. thesis, Dept. of Elect. Eng., McGill Univ., Montreal, Canada, 1988.
[31] P. T. Sander and S. W. Zucker, "Inferring differential structure from three-dimensional images: Smooth cross sections of fiber bundles,"IEEE Trans. Patt. Anal. Machine Intell., to be published.
[32] R. Courant and D. Hilbert,Methods of Mathematical Physics. New York: Interscience, 1953, vol. I.
[33] G. Bachman and L. Narici,Functional Analysis. New York: Academic, 1966.
[34] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, "Optimization by simulated annealing,"Sci., vol. 220, no. 4598, pp. 671-680, May 13, 1983.
[35] L. Davis,Genetic Algorithms and Simulated Annealing. London: Pitman, 1987.
[36] A. Blake, "Comparison of the efficiency of deterministic and stochastic algorithms for visual reconstruction,"IEEE Trans. Patt. Anal. Machine Intell., vol. 11, no. 1, pp. 2-12, Jan. 1989.
[37] J. Duchon, "Splines minimizing rotation-invariant semi-norms in Sobolev spaces,"Constructive Theory of Functions of Several Variables, pp. 85-100, 1976.
[38] J. Meinguet, "Multivariate interpolation at arbitrary points made simple,"J. Applied Math. Phys., vol. 30, pp. 292-304, 1979.
[39] G. Wahba, "Spline bases, regularization and generalized cross validation for solving approximation problems with large quantities of noisy data,"Approximation Theory III(W. Cheney, Ed.). Academic, 1980, pp. 905-912.
[40] G. Wahba, "Variational methods for multidimensional inverse problems,"Advances in Remote Sensing Retrieval Methods, pp. 385-407, 1985.
[41] A. K. Cline and R. L. Renka, "A storage-efficient method for construction of a Thiessen triangulation,"Rocky Mountain J. Math., vol. 14, no. 1, pp. 119-139, Winter 1984.
[42] R. J. Renka, "A Storage-efficient method for construction of a Thiessen triangulation," Tech. Rep. ORNL/CSD-101, Oak Ridge Nat. Lab., 1982.
[43] F. Ferrie, J. Lagarde, and P. Whaite, "Darboux frames, snakes, and super-quadrics: Geometry from the bottom-up," inProc. Workshop Interpretation 3D Scenes, 1989, pp. 170-176.
[44] P. J. Besl and R. C. Jain, "Segmentation through symbolic surface descriptions," inProc. 1986 Comput. Vision Patt. Recogn. Conf.(Miami, FL), June 22-26, 1986, pp. 77-85.
[45] T. J. Fan, G. Medioni, and R. Nevatia, "Description of surfaces from range data using curvature properties," inProc. 1986 IEEE Comput. Vision Patt. Recogn. Conf.(Miami, FL), June 22-26, 1986, pp. 86-91.
[46] I. Faux and M. Pratt,Computational Geometry for Design and Manufacture. Ellis Horwood, 1979.
[47] P. Parent and S. W. Zucker, "Curvature consistency and curve detection,"J. Opt. Soc. Amer. A, vol. 2, no. 13, pp. 5, 1985.

Index Terms:
viewpoint invariant recovery; picture processing; pattern recognition; visual surfaces; sparse data; 3D space; ill-posed inverse problem; invariant surface characteristics; finite element analysis; finite element analysis; pattern recognition; picture processing
R.L. Stevenson, E.J. Delp, "Viewpoint Invariant Recovery of Visual Surfaces from Sparse Data," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 9, pp. 897-909, Sept. 1992, doi:10.1109/34.161349
Usage of this product signifies your acceptance of the Terms of Use.