This Article 
 Bibliographic References 
 Add to: 
Recovering the Position and Orientation of Free-Form Objects from Image Contours Using 3D Distance Maps
April 1995 (vol. 17 no. 4)
pp. 378-390

Abstract—The accurate matching of 3D anatomical surfaces with sensory data such as 2D X-ray projections is a basic problem in Computer and Robot Assisted Surgery. In model-based vision, this problem can be formulated as the estimation of the spatial pose (position and orientation) of a 3D smooth object from 2D video images. We present a new method for determining the rigid body transformation that describes this match. Our method performs a least squares minimization of the energy necessary to bring the set of the camera-contour projection lines tangent to the surface. To correctly deal with projection lines that penetrate the surface, we consider the minimum signed distance to the surface along each line (i.e., distances inside the object are negative). To quickly and accurately compute distances to the surface, we introduce a precomputed distance map represented using an octree spline whose resolution increases near the surface. This octree structure allows us to quickly find the minimum distance along each line using best-first search. Experimental results for 3D surface to 2D projection matching are presented for both simulated and real data. The combination of our problem formulation in 3D, our computation of line to surface distances with the octree-spline distance map, and our simple minimization technique based on the Levenberg-Marquardt algorithm results in a method that solves the 3D/2D matching problem for arbitrary smooth shapes accurately and quickly.

[1] S. Lavallée,“Registration for Computer Integrated Surgery: Methodology, state of the art,” R.H. Taylor, S. Lavallée, G.C. Burdea, and R.W. Mosges, eds., Computer Integrated Surgery,Cambridge, MA: MIT Press, 1995.
[2] R.H. Taylor, S. Lavallée, G.C. Burdea, and R.W. Mosges, eds., Computer Integrated Surgery,Cambridge, MA: MIT Press, 1995.
[3] G. Champleboux,S. Lavallée,P. Sautot,, and P. Cinquin,“Accurate calibration of cameras and range imaging sensors, the NPBS method,” IEEE Int’l Conf. on Robotics and Automation, pp. 1552-1558,Nice France, May 1992.
[4] P. Sautot,P. Cinquin,S. Lavallée,, and J. Troccaz,“Computer assisted spine surgery: A first step towards clinical application in orthopaedics,” 14th IEEE Eng. in Medicine and Biology Conf., pp. 1071-1072,Paris, Nov. 1992.
[5] F. Leitner, I. Marque, S. Lavallée, and P. Cinquin, "Dynamic Segmentation: Finding the Edge with Snake-Splines," Proc. Int'l Conf. Curves and Surfaces, pp. 1-4,Chamonix, France, June 1990. Academic Press.
[6] I. Marque,Segmentation d’Images Medicales Tridimensionnelles Basée sur une Modelisation Continue du Volume, PhD thesis, Grenoble University, France, Dec. 1990.
[7] O. Monga, R. Deriche, G. Malandain, and J-P. Cocquerez, "Recursive Filtering and Edge Closing: Two Primary Tools for 3D Edge Detection," Proc. First European Conf. Computer Vision (ECCV), Antibes, France, Apr. 1990.
[8] S. Lavallée,R. Szeliski,, and L. Brunie,“Matching 3D smooth surfaces with their 2D projections using 3D distance maps,” In SPIE Vol. 1570, Geometric Methods in Computer Vision, pp. 322-336,San Diego, CA, July 1991.
[9] G. Champleboux,S. Lavallée,R. Szeliski,, and L. Brunie,“From accurate range imaging sensor calibration to accurate model-based 3D object localization,” IEEE CS Conf. on Comput. Vision and Pattern Recognition (CVPR’92),Champaign, Ill., June 1992.
[10] B.A. Kall,P.J. Kelly,, and S.J. Goerss,“Comprehensive computer-assisted data collection treatment planning and interactive surgery,” SPIE, Medical Imaging, vol. 767, pp. 27-35, 1987.
[11] C. Schiers,U. Tiede,, and K.H. Hohne,“Interactive 3D registration of image volumes from different sources,” H.U. Lemke, ed., Computer Assisted Radiology,Berlin: Springer-Verlag, pp. 667-669, June 1989.
[12] C.A. Pelizzari,G.T.Y. Chen,D.R. Spelbring,R.R. Weichselbaum,, and C-T. Chen,“Accurate 3D registration of CT, PET, and-or MR images of the brain,” J. Computer Assisted Tomography, 13(1):20-26, 1989.
[13] A. Gueziec and N. Ayache,“Smoothing and matching of 3D space curves,” Second European Conf. Computer Vision, pp. 620-629,Santa Margherita Ligure, Italy, Springer-Verlag, 1992.
[14] D. Terzopolous, A. Witkin, and M. Kass, "Constraints on deformable models: Recovering 3D shape and nonrigid motion, AI, no. 36, pp. 91-123, 1988.
[15] R. Bajscy and S. Kovacic, "Multiresolution Elastic Matching," Computer Vision, Graphics&Image Processing, vol. 46, no. 1, pp. 1-21, 1989.
[16] R. Szeliski and S. Lavallée,“Matching 3D anatomical surfaces with non-rigid deformations using octree-splines,” IEEE Workshop on Biomedical Image Analysis,Seattle, WA: pp. 144-153, June 1994.
[17] Y. Demazeau,Niveaux de représentation pour la vision par ordinateur; indices d’image et indices de scène, PhD thesis, INPG, Grenoble University, 1986.
[18] D.G. Lowe, Perceptual Organization and Visual Recognition. Boston: Kluwer Academic, 1985.
[19] M. Dhome, M. Richetin, J.T. Lapreste, and G. Rives, “Determination of the Attitude of 3D Objects from a Single Perspective View,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 12, pp. 1,265-1,278, Dec. 1989.
[20] C. Granger,Reconnaissance d’objets par mise en correspondance en vision par ordinateur, Nice University, France, 1985.
[21] D.G. Lowe, "Fitting Parameterized Three-Dimensional Models to Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 5, pp. 441-450, May 1991.
[22] P.J. Besl and N.D. McKay, "A Method for Registration of 3D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[23] D.J. Kriegman and J. Ponce, "On Recognizing and Positioning Curve 3-D Objects From Image Contours," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, pp. 1,127-1,137, Dec. 1990.
[24] R. Glachet,M. Dhome,, and J. T. Lapreste,“Finding the pose of an object of revolution,” G. Sandini, ed., Second European Conference on Computer Vision,Santa Margherita, Italy,New York: Springer Verlag, LNCS Series Vol. 588, pp. 681-686, May 1992.
[25] R. Brooks,“Symbolic reasoning among 3D models and 2D images,” Artificial Intelligence, 17:285-348, 1981.
[26] S. Sullivan, L. Sandford, and J. Ponce, "Using Geometric Distance Fits for 3D Object Modeling and Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 12, pp. 1,183-1,196, 1994.
[27] H. G. Barrow,J. M. Tenenbaum,R. C. Bolles,, and H. C. Wolf,“Parameteric correspondence and chamfer matching: Two new techniques for image matching,” Fifth International Joint Conference on Artificial Intelligence,Cambridge, MA: pp. 659-663, August 1977.
[28] N. Ayache, Artificial Vision for Mobile Robots—Stereo-Vision and Multisensory Perception. MIT Press, 1991.
[29] R. Y. Tsai,“Synopsis of recent progress on camera calibration for 3D machine vision,” The Robotics Review,Cambridge, MA: MIT Press, pp. 147-160, 1989.
[30] H. A. Martins,J. R. Birk,, and R. B. Kelley,“Camera models based on data from two calibration planes,” Computer Graphics and Image Processing, 17:173-179, 1981.
[31] G. Borgefors,“Distance transformations in arbitrary dimensions,” Computer Vision, Graphics, and Image Processing, 27:321-345, 1984.
[32] H. Samet, The Design and Analysis of Spatial Data Structures. Addison-Wesley, 1990.
[33] G. Garcia,Contribution a la modelisation d’objets et a la détection de collisions en robotique a l’aide d’arbres octaux, PhD thesis, Nantes University, September 1989.
[34] B.V. Herzen and A.H. Barr,“Accurate triangulations of deformed, intersecting surfaces,” Computer Graphics, 21(4):103-110, 1987.
[35] O. Monga,N. Ayache,, and P. T. Sander,“From voxel to curvature,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition,Maui, HI, pp. 644-649, June 1991.
[36] W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes, the Art of Scientific Computing. Cambridge, Mass.: Cambridge Univ. Press, 1986.
[37] P. J. Huber,Robust Statistics,New York: John Wiley∧Sons, 1981.
[38] W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C. Cambridge Univ. Press, 1992.
[39] W.E.L. Grimson, T. Lozano-Perez, S.J. White, W.M. Wells III, R. Kikinis, and G.J. Ettinger, “An Automatic Registration Method for Frameless Stereotaxy, Image Guided Surgery, and Enhanced Reality Visualization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 430-436, June 1994.
[40] H.J. Wolfson, "On Curve Matching," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 5, pp. 483-489, May 1990.
[41] J. Barnes and P. Hut,“A hierarchical O(N log N) force-calculation algorithm,” Nature, 324:446-449, December4, 1986.
[42] D. Terzopoulos and M. Vasilescu, "Sampling and Reconstruction With Adaptive Meshes," Proc. IEEE Computer Vision and Pattern Recognition, pp. 70-75,Lahaina, Maui, Haw., June 1991.
[43] D.R. Forsey and R.H. Bartels, “Hierarchical B-Spline Refinement,” Computer Graphics (SIGGRAPH '88 Proc.), vol. 22, no. 4, pp. 205-212, Aug. 1988.
[44] R. Szeliski, "Fast Surface Interpolation Using Hierarchical Basis Functions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 6, pp. 513-528, June 1990.
[45] L. Brunie,“Accurate 3D modelization of the scoliosis by matching 3D images (MR, CT) with 2D X-rays,” J. Dansereau, ed., Int. Symp. on 3D Scoliotic Deformities,Montreal, Canada: Gustav Fischer Verlag, pp. 11-17, June 1992.

Index Terms:
Registration, 3D/2D matching, octree spline, line-to-surface distance, X-ray projections, non-linear least squares minimization.
Stéphane Lavallée, Richard Szeliski, "Recovering the Position and Orientation of Free-Form Objects from Image Contours Using 3D Distance Maps," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 4, pp. 378-390, April 1995, doi:10.1109/34.385980
Usage of this product signifies your acceptance of the Terms of Use.