This Article 
 Bibliographic References 
 Add to: 
Matching Hierarchical Structures Using Association Graphs
November 1999 (vol. 21 no. 11)
pp. 1105-1120

Abstract—It is well-known that the problem of matching two relational structures can be posed as an equivalent problem of finding a maximal clique in a (derived) “association graph.” However, it is not clear how to apply this approach to computer vision problems where the graphs are hierarchically organized, i.e., are trees, since maximal cliques are not constrained to preserve the partial order. Here, we provide a solution to the problem of matching two trees by constructing the association graph using the graph-theoretic concept of connectivity. We prove that, in the new formulation, there is a one-to-one correspondence between maximal cliques and maximal subtree isomorphisms. This allows us to cast the tree matching problem as an indefinite quadratic program using the Motzkin-Straus theorem, and we use “replicator” dynamical systems developed in theoretical biology to solve it. Such continuous solutions to discrete problems are attractive because they can motivate analog and biological implementations. The framework is also extended to the matching of attributed trees by using weighted association graphs. We illustrate the power of the approach by matching articulated and deformed shapes described by shock trees.

[1] A.P. Ambler, H.G. Barrow, C.M. Brown, R.M. Burstall, and R.J. Popplestone, “A Versatile Computer-Controlled Assembly System,” Proc. Int'l Joint Conf. Artificial Intelligence, pp. 298-307, Stanford, Calif., 1973.
[2] D.H. Ballard and C.M. Brown, Computer Vision, Prentice Hall, Upper Saddle River, N.J., 1982.
[3] H.G. Barrow and R.M. Burstall, “Subgraph Isomorphism, Matching Relational Structures, and Maximal Cliques,” Information Processing Letters, vol. 4, no. 4, pp. 83-84, 1976.
[4] M. Bartoli, M. Pelillo, K. Siddiqi, and S.W. Zucker, “Attributed Tree Homomorphism Using Association Graphs,” Technical Report CS-99-12, Dipartimento di Informatica, UniversitàCa' Foscari di Venezia, 1999.
[5] L.E. Baum and J.A. Eagon, “An Inequality with Applications to Statistical Estimation for Probabilistic Functions of Markov Processes and to a Model for Ecology,” Bulletin Am. Math. Soc., vol. 73, pp. 360-363, 1967.
[6] R.C. Bolles and R.A. Cain, “Recognizing and Locating Partially Visible Objects: The Locus-Feature-Focus Method,” Int'l J. Robotics Research, vol. 1, no. 3, pp. 57-82, 1982.
[7] I.M. Bomze, “Evolution Towards the Maximum Clique,” J. Global Optimization, vol. 10, pp. 143-164, 1997.
[8] I.M. Bomze, “On Standard Quadratic Optimization Problems,” J. Global Optimization, vol. 13, pp. 369-387, 1998.
[9] I.M. Bomze, M. Budinich, P.M. Pardalos, and M. Pelillo, “The Maximum Clique Problem,” Handbook of Combinatorial Optimization, D.-Z. Du and P.M. Pardalos, eds., vol. 4. Boston, Mass.: Kluwer Academic, 1999.
[10] I.M. Bomze, M. Pelillo, and R. Giacomini, “Evolutionary Approach to the Maximum Clique Problem: Empirical Evidence on a Larger Scale,” Developments in Global Optimization, I.M. Bomze, T. Csendes, R. Horst, and P.M. Pardalos, eds., pp. 95-108, Dordrecht, The Netherlands: Kluwer, 1997.
[11] I.M. Bomze, M. Pelillo, and V. Stix, “Approximating the Maximum Weight Clique Using Replicator Dynamics,” Technical Report CS-99-13, Dipartimento di Informatica, UniversitàCa' Foscari di Venezia, 1999.
[12] R.W. Brockett and P. Maragos, “Evolution Equations for Continuous-Scale Morphology,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 1-4, Mar. 1992.
[13] J.F. Crow and M. Kimura, An Introduction to Population Genetics Theory. New York: Harper&Row, 1970.
[14] R. Durbin and D. Willshaw, “An Analog Approach to the Travelling Salesman Problem Using an Elastic Net Method,” Nature, vol. 326, pp. 689-691, 1987.
[15] R.A. Fisher, The Genetical Theory of Natural Selection. London: Oxford Univ. Press, 1930.
[16] Y. Fu and P.W. Anderson, “Application of Statistical Mechanics to NP-Complete Problems in Combinatorial Optimization,” J. Physics A, vol. 19, pp. 1,605-1,620, 1986.
[17] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness.New York: W.H. Freeman, 1979.
[18] L.E. Gibbons, D.W. Hearn, and P.M. Pardalos, “A Continuous Based Heuristic for the Maximum Clique Problem,” Cliques, Coloring, and Satisfiability—Second DIMACS Implementation Challenge, D.S. Johnson and M. Trick, eds., pp. 103-124, 1996.
[19] L.E. Gibbons, D.W. Hearn, P.M. Pardalos, and M.V. Ramana, “Continuous Characterizations of the Maximum Clique Problem,” Math. Operations Research, vol. 22, pp. 754-768, 1997.
[20] S. Gold and A. Rangarajan, “A Graduated Assignment Algorithm for Graph Matching,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 4, pp. 377-388, Apr. 1996.
[21] M. Grötschel, L. Lovász, and A. Schrijver, Geometric Algorithms and Combinatorial Optimization. Berlin: Springer-Verlag, 1988.
[22] F. Harary, Graph Theory. Reading, Mass.: Addison-Wesley, 1969.
[23] J. Hofbauer and K. Sigmund, The Theory of Evolution and Dynamical Systems. Cambridge, U.K.: Cambridge Univ. Press, 1988/
[24] J.J. Hopfield and D.W. Tank, “Neural Computation of Decisions in Optimization Problems,” Biological Cybernetics, vol. 52, pp. 141-152, 1985.
[25] R. Horaud and T. Skordas, “Stereo Correspondence Through Feature Grouping and Maximal Cliques,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, pp. 1,168-1,180, 1989.
[26] A. Jagota, “Approximating the Maximum Clique with a Hopfield Network,” IEEE Trans. Neural Networks, vol. 6, pp. 724-735, 1995.
[27] B. B. Kimia,A. R. Tannenbaum,, and S. W. Zucker,“Toward a computational theory of shape: An overview,” in Proc. of ECCV,Antibes, France, 1990.
[28] B.B. Kimia, A.R. Tannenbaum, and S.W. Zucker, “Shape, Shocks and Deformations I: The Components of 2D Shape and the Reaction-Diffusion Space,” Int'l J. Computer Vision, vol. 15, pp. 189-224, 1995.
[29] J.J. Kosowsky and A.L. Yuille, "The Invisible Hand Algorithm: Solving the Assignment Problem with Statistical Physics," Neural Networks, vol. 7, pp. 477-490, 1994.
[30] P.D. Lax, “Shock Waves and Entropy,” Contributions to Nonlinear Functional Analysis, E.H. Zarantonello, ed., pp. 603-634, New York: Academic Press, 1971.
[31] J.T.L. Wang, K. Zhang, K. Jeong, and D. Shasha, “A System for Approximate Tree Matching,” IEEE Trans. Knowledge and Data Eng., vol. 6, no. 4, pp. 559-571, Aug. 1994.
[32] T. Liu, D. Geiger, and R. Kohn, “Representation and Self-Similarity of Shapes,” Proc. Int'l Conf. Computer Vision, Bombay, 1998.
[33] V. Losert and E. Akin, “Dynamics of Games and Genes: Discrete versus Continuous Time,” J. Math. Biology, vol. 17, pp. 241-251, 1983.
[34] S.Y. Lu, “A Tree-Matching Algorithm Based on Node Splitting and Merging,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, pp. 249-256, 1984.
[35] Y. Lyubich, G.D. Maistrowskii, and Y.G. Ol'khovskii, “Selection-Induced Convergence to Equilibrium in a Single-Locus Autosomal Population,” Problems of Information Transmission, vol. 16, pp. 66-75, 1980.
[36] D. Marr and K.H. Nishihara, “Representation and Recognition of the Spatial Organization of Three-Dimensional Shapes,” Proc. Royal Soc. London B, vol. 200, pp. 269-294, 1978.
[37] D.W. Matula, “An Algorithm for Subtree Identification,” SIAM Review, vol. 10, pp. 273-274, 1968.
[38] D. Miller and S.W. Zucker, “Efficient Simplex-Like Methods for Equilibria of Nonsymmetric Analog Networks,” Neural Computation, vol. 4, no. 2, pp. 167-190, 1992.
[39] D.A. Miller and S.W. Zucker, “Computing with Self-Excitatory Cliques: A Model and Application to Hyperacuity-Sclae Computation in Visual Cortex,” Neural Computation, vol. 11, pp. 21-66, 1999.
[40] T.S. Motzkin and E.G. Straus, “Maxima for Graphs and a New Proof of a Theorem of Turán,” Canadian J. Math., vol. 17, pp. 533-540, 1965.
[41] M. Neff, R. Byrd, and O. Rizk, “Creating and Querying Hierarchical Lexical Databases,” Proc. Conf. Applied Natural Language Processes, pp. 84-93, 1988.
[42] H. Ogawa, "Labeled Point Pattern Matching by Delaunay Triangulation and Maximal Cliques," Pattern Recognition, vol. 19, no. 1, pp. 35-40, 1986.
[43] M. Ohlsson, C. Peterson, and B. Söderberg, “Neural Networks for Optimization Problems with Inequality Constraints: The Knapsack Problem,” Neural Computation, vol. 5, pp. 331-339, 1993.
[44] P.M. Pardalos, “Continuous Approaches to Discrete Optimization Problems,” Nonlinear Optimization and Applications, G.D. Pillo and F. Giannessi, eds., pp. 313-328. Plenum Press, 1996.
[45] P.M. Pardalos and A.T. Phillips, “A Global Optimization Approach for Solving the Maximum Clique Problem,” Int'l J. Computer Math., vol. 33, pp. 209-216, 1990.
[46] M. Pelillo, “Relaxation Labeling Networks for the Maximum Clique Problem,” J. Artificial Neural Networks, vol. 2, pp. 313-328, 1995.
[47] M. Pelillo, “The Dynamics of Nonlinear Relaxation Labeling Processes,” J. Math. Imaging and Vision, vol. 7, pp. 309-323, 1997.
[48] M. Pelillo, “A Unifying Framework for Relational Structure Matching,” Proc. Int'l Conf. Pattern Recognition, pp. 1,316-1,319, Brisbane, Australia, 1998.
[49] M. Pelillo, “Replicator Equations, Maximal Cliques, and Graph Isomorphism,” Neural Computation, vol. 11, no. 8, pp. 2023-2045, 1999.
[50] M. Pelillo and A. Jagota, “Feasible and Infeasible Maxima in a Quadratic Program for Maximum Clique,” J. Artificial Neural Networks, vol. 2, pp. 411-420, 1995.
[51] F. Pla and J. Marchant, “Matching Feature Points in Image Sequences through a Region-Based Method,” Computer Vision and Image Understanding, vol. 66, no. 3, pp. 271-285, 1997.
[52] B. Radig, “Image Sequence Analysis Using Relational Structures,” Pattern Recognition, vol. 17, pp. 161-167, 1984.
[53] A. Rangarajan and E. Mjolsness, “A Lagrangian Relaxation Network for Graph Matching,” IEEE Trans. Neural Networks, vol. 7, no. 6, pp. 1,365-1,381, 1996.
[54] S.W. Reyner, “An Analysis of a Good Algorithm for the Subtree Problem,” SIAM J. Computing, vol. 6, pp. 730-732, 1977.
[55] H. Rom and G. Medioni, “Hierarchical Decomposition and Axial Shape Description,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, pp. 973-981, 1993.
[56] A. Rosenfeld, R.A. Hummel, and S.W. Zucker, “Scene Labeling by Relaxation Operations,” IEEE Trans. Systems, Man, and Cybernetics, vol. 6, pp. 420-433, 1976.
[57] H. Samet, The Design and Analysis of Spatial Data Structures. Addison-Wesley, 1990.
[58] B.A. Shapiro and K. Zhang, “Comparing Multiple RNA Secondary Structures Using Tree Comparisons,” Computer Applications Bioscience, vol. 6, pp. 309-318, 1990.
[59] D. Shasha, J. Wang, and K. Zhang, “Exact and Approximate Algorithm for Unordered Tree Matching,” IEEE Trans. Systems, Man, and Cybernetics, vol. 24, no. 4, pp. 668-678, 1994.
[60] K. Siddiqi and B.B. Kimia, A Shock Grammar for Recognition Proc. Conf. Computer Vision and Pattern Recognition, pp. 507-513, 1996.
[61] K. Siddiqi, B.B. Kimia, A. Tannenbaum, and S.W. Zucker, “Shapes, Shocks and Wiggles,” Image and Vision Computing, vol. 17, nos. 5-6, pp. 365-373, 1999.
[62] K. Siddiqi, A. Shokoufandeh, S. Dickinson, and S. Zucker, “Shock Graphs and Shape Matching,” Proc. Int'l Conf. Computer Vision, pp. 222-229, 1998.
[63] J.W. Weibull, Evolutionary Game Theory. Cambridge, Mass.: MIT Press, 1995.
[64] H.S. Wilf, “Spectral Bounds for the Clique and Independence Numbers of Graphs,” J. Combinatorial Theory, Series B, vol. 40, pp. 113-117, 1986.
[65] B. Yang, W.E. Snyder, and G.L. Bilbro, "Matching Oversegmented 3D Images to Models Using Associated Graphs," Image and Vision Computing, vol. 7, no. 2, pp. 135-143, 1989.
[66] S.C. Zhu and A.L. Yuille, “FORMS: A Flexible Object Recognition and Modeling System,” Int'l J. Computer Vision, vol. 20, no. 3, Dec. 1996

Index Terms:
Maximal subtree isomorphisms, association graphs, maximal cliques, replicator dynamical systems, shock trees, shape recognition.
Marcello Pelillo, Kaleem Siddiqi, Steven W. Zucker, "Matching Hierarchical Structures Using Association Graphs," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 11, pp. 1105-1120, Nov. 1999, doi:10.1109/34.809105
Usage of this product signifies your acceptance of the Terms of Use.