This Article 
 Bibliographic References 
 Add to: 
Modal Matching for Correspondence and Recognition
June 1995 (vol. 17 no. 6)
pp. 545-561

Abstract—Modal matching is a new method for establishing correspondences and computing canonical descriptions. The method is based on the idea of describing objects in terms of generalized symmetries, as defined by each object’s eigenmodes. The resulting modal description is used for object recognition and categorization, where shape similarities are expressed as the amounts of modal deformation energy needed to align the two objects. In general, modes provide a global-to-local ordering of shape deformation and thus allow for selecting which types of deformations are used in object alignment and comparison. In contrast to previous techniques, which required correspondence to be computed with an initial or prototype shape, modal matching utilizes a new type of finite element formulation that allows for an object’s eigenmodes to be computed directly from available image information. This improved formulation provides greater generality and accuracy, and is applicable to data of any dimensionality. Correspondence results with 2D contour and point feature data are shown, and recognition experiments with 2D images of hand tools and airplanes are described.

[1] D.H. Ballard and C.M. Brown, Computer Vision, Prentice Hall, Upper Saddle River, N.J., 1982.
[2] K. Bathe,Finite Element Procedures in Engineering Analysis.Englewood Cliffs, N.J.: Prentice Hall, 1982.
[3] A. Baumberg and D. Hogg, “Learning Flexible Models from Image Sequences,” Computer Vision-ECCV '94, J.-O. Eklundh, ed., Springer-Verlag, vol. 1, pp. 299-308, 1994.
[4] T. Beier and S. Neely, "Feature Based Image Metamorphosis," Computer Graphics(Proc. Siggraph 91), ACM Press, New York, 1992, pp. 35-42.
[5] A. Blake, R. Curwen, and A.A. Zisserman, “A Framework for Spatiotemporal Control in the Tracking of Visual Contours,” Int'l J. Computer Vision, vol. 11, no. 2, pp. 127-145, Oct. 1993.
[6] L. Bookstein, "Principal Warps: Thin-Plate Splines and the Decomposition of Deformations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 6, pp. 567-585, June 1989.
[7] I. Cohen, N. Ayache, and P. Sulger, “Tracking Points on Deformable Objects Using Curvature Information,” Proc. European Conf. Computer Vision, pp. 458-466, 1992.
[8] T.F. Cootes, D.H. Cooper, C.J. Taylor, and J. Graham, “A Trainable Method of Parametric Shape Description,” Image and Vision Computing, vol. 10, pp. 289-294, June 1992.
[9] J. Duncan, R.L. Owen, L.H. Staib, and P. Anandan, “Measurement of Non-Rigid Motion Using Contour Shape Descriptors,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 318-324, 1991.
[10] P. Soille, “From Binary to Grey Scale Convex Hulls,” Fundamenta Informaticae, vol. 41, nos. 1-2, pp. 131-146, Jan. 2000.
[11] P.W. Hallinan,“A low-dimensional representation of human faces for arbitrary lighting conditions,” Technical Report 93-6, Harvard Robotics Laboratory, Cambridge, Mass., Dec. 1993.
[12] B. Horn,“Closed-form solution of absolute orientation using unit quarternions,” J. Optical Soc. America A, vol. 4, pp. 629-642, 1987.
[13] T. Kanade,“Geometrical aspects of interpreting images as a three-dimensional scene,” Proc. IEEE, vol. 71, no. 7, pp. 789-802, 1983.
[14] M. Kass,A. Witkin,, and D. Terzopoulos,“Snakes: Active contour models,” Int’l J. Computer Vision, vol. 1, pp. 321-331, 1987.
[15] M. Kirby and L. Sirovich,“Application of Karhunen-Loève procedure for the characterization of human faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 103-108, Jan. 1990.
[16] H. Murase and S.K. Nayar, "Learning and Recognition of 3D Objects from Appearance," Proc. IEEE Qualitative Vision Workshop, New York, pp. 39-49, 1993.
[17] A. Pentland,“Perceptual organization and the representation of natural form,” Artificial Intelligence, vol. 28, pp. 293-331, 1986.
[18] A.P. Pentland,“Automatic extraction of deformable part models,” Int’l J. Computer Vision, vol. 4, pp. 107-126, 1990.
[19] A.P. Pentland, “Computational Complexity versus Simulated Environments,” Computer Graphics, (Symp. Interactive 3D Graphics), Vol. 24, No. 2, Mar. 1990, pp. 185-192.
[20] A. Pentland and B. Horowitz, “Recovery of Non-Rigid Motion and Structure,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 730-742, July 1991.
[21] A. Pentland, B. Moghaddam, and Starner, "View-Based and Modular Eigenspaces for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994, pp. 84-91.
[22] A. Pentland and S. Sclaroff, "Closed-Form Solutions for Physically-Based Shape Modeling and Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 715-729, July 1991.
[23] T. Poggio and F. Girosi, "A Theory of Networks for Approximation and Learning," MIT AI Lab TR-1140, 1989.
[24] M. Powell,“Radial basis functions for multivariate interpolation: A review.” Technical Report DAMPT 1985/NA12, Cambridge Univ., Cambridge, England, 1985.
[25] W. Richards and D. Hoffman,“Codon constraints on closed 2D shapes,” Computer Vision, Graphics, and Image Processing, vol. 31, pp. 265-281, 1985.
[26] A. Samal and P. Iyengar,“Natural shape detection based on principle components analysis,” SPIE J. Electronic Imaging, vol. 2, no. 3, pp. 253-263, July 1993.
[27] S. Sclaroff and A.P. Pentland, "A Model Framework for Correspondence and Description," Proc. 4th Int'l Conf. Computer Vision, pp. 308-313, 1993.
[28] S. Sclaroff,“Modal matching: A method for describing, comparing, and manipulating digital signals,” PhD thesis, Massachusetts Inst. of Tech nology, Cambridge, Mass., February 1995. Also appears as Technical Report, MIT Media Laboratory Vision and Modeling TR-311, January 1995.
[29] S. Sclaroff and A. Pentland,“Object recognition and categorization using modal matching,” Proc. Second CAD-Based Vision Workshop, pp. 258-265, Feb. 1994.
[30] G. Scott and H. Longuet-Higgins,“An algorithm for associating the features of two images,” Proc. Royal Soc. of London, vol. B, no. 244, pp. 21-26, 1991.
[31] T.W. Sederberg and E. Greenwood, A Physically Based Approach to 2D Shape Blending Proc. SIGGRAPH '92, pp. 25-34, July 1992.
[32] L. Shapiro,“Towards a vision-based motion framework,” Technical Report, Oxford Univ., Oxford, England, 1991.
[33] L.S. Shapiro and J.M. Brady, “Feature-Based Correspondence: An Eigenvector Approach,” Image and Vision Computing, vol. 10, pp. 283-288, 1992.
[34] L.H. Staib and J.S. Duncan, “Parametrically Deformable Contour Model,“ Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 98-103, 1989.
[35] D. Terzopoulos, "The Computation of Visible Surface Representations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp. 417-438, Apr. 1988.
[36] D. Terzopoulos and D. Metaxas, “Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 703-714, July 1991.
[37] D. Terzopolous, A. Witkin, and M. Kass, "Constraints on deformable models: Recovering 3D shape and nonrigid motion, AI, no. 36, pp. 91-123, 1988.
[38] C. Thorpe,“Machine learning and human interface for the CMU navlab,” Proc. Computer Vision for Space Applications,Juan-les-Pins, France, Sept. 1993.
[39] M. Turk and A. Pentland., “Eigenfaces for recognition,” J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[40] S. Ullman and R. Basri, "Recognition by Linear Combinations of Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, pp. 992-1006, 1991.
[41] S. Umeyama, “An Eigendecomposition Approach to Weighted Graph Matching Problems,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 5, pp. 695-703, Sept. 1988.
[42] M.P. van Oeffelen and P.G. Vos,“An algorithm for pattern description on the level of relative proximity,” Pattern Recognition, vol. 16, no. 3, pp. 341-348, 1983.
[43] Z. Wang and A. Jepson,“A new closed-form solution of absolute orientation,” Proc. Computer Vision and Pattern Recognition, pp. 129-134, 1994
[44] G. Wolberg, Digital Image Warping, IEEE CS Press, 1990.
[45] D. Wood,Jane’s World Aircraft Recognition Handbook. London: Jane’s Publishing Co., 1979.

Index Terms:
Correspondence, shape description, shape invariants, object recognition, deformation, finite element methods, modal analysis, vibration modes, eigenmodes.
Stan Sclaroff, Alex P. Pentland, "Modal Matching for Correspondence and Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 6, pp. 545-561, June 1995, doi:10.1109/34.387502
Usage of this product signifies your acceptance of the Terms of Use.