This Article 
 Bibliographic References 
 Add to: 
Similarity Searching in Medical Image Databases
May-June 1997 (vol. 9 no. 3)
pp. 435-447

Abstract—We propose a method to handle approximate searching by image content in medical image databases. Image content is represented by attributed relational graphs holding features of objects and relationships between objects. The method relies on the assumption that a fixed number of "labeled" or "expected" objects (e.g., "heart," "lungs," etc.) are common in all images of a given application domain in addition to a variable number of "unexpected" or "unlabeled" objects (e.g., "tumor," "hematoma," etc.). The method can answer queries by example, such as "find all X-rays that are similar to Smith's X-ray." The stored images are mapped to points in a multidimensional space and are indexed using state-of-the-art database methods (R-trees). The proposed method has several desirable properties:

  • Database search is approximate, so that all images up to a prespecified degree of similarity (tolerance) are retrieved.

  • It has no "false dismissals" (i.e., all images qualifying query selection criteria are retrieved).

  • It is much faster than sequential scanning for searching in the main memory and on the disk (i.e., by up to an order of magnitude), thus scaling-up well for large databases.

  • [1] S.C. Orphanoudakis, C. Chronaki, and S. Kostomanolakis, "${\rm I^2C}$: A System for the Indexing, Storage, and Retrieval of Medical Images by Content," J. Medical Informatics, vol. 19, no. 2, pp. 109-122, 1994.
    [2] J.R. Bach, S. Paul, and R. Jain, “A Visual Information Management System for the Interactive Retrieval of Faces,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 619-628, 1993.
    [3] D.H. Ballard and C.M. Brown, Computer Vision, Prentice Hall, Upper Saddle River, N.J., 1982.
    [4] S.-K. Chang, Principles of Pictorial Information Systems Design.Englewood Cliffs, N.J.: Prentice Hall Int'l Editions, 1989.
    [5] M.A. Fischler and R.A. Elschlager, "The Representation and Matching of Pictorial Structures," IEEE Trans. Computers, vol. 22, no. 1, pp. 67-92, 1973.
    [6] L.G. Shapiro and R.M. Haralick, "Structural Descriptions and Inexact Matching," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 5, pp. 504-519, 1981.
    [7] H. Tamura and N. Yokoya, "Image Database Systems: A Survey," Pattern Recognition, vol. 17, no. 1, pp. 29-49, 1984.
    [8] S.-K. Chang and A.D. Hsu, “Image-Information Systems—Where Do We Go from Here?“ IEEE Trans. Knowledge and Data Eng., vol. 4, no. 5, pp. 431-442, Oct. 1992.
    [9] R. Jain and W. Niblack, Proc. NSF Workshop on Visual Information Management, Feb. 1992.
    [10] S. Christodoulakis, M. Theodoridou, F. Ho, M. Papa, and A. Pathria, "Multimedia Document Presentation, Information Extraction, and Document Formation in MINOS: A Model and A System," ACM Trans. Office Information Systems, vol. 4, no. 4, pp. 345-383, Oct. 1986.
    [11] S.K. Chang, Q.Y. Shi, and C.W. Yan, “Iconic Indexing by 2-D Strings,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 3, pp. 413-427, July 1987.
    [12] E.G.M. Petrakis and S.C. Orphanoudakis, "Methodology for the Representation, Indexing, and Retrieval of Images by Content," Image and Vision Computing, vol. 11, no. 8, pp. 504-521, Oct. 1993.
    [13] E.G.M. Petrakis and S.C. Orphanoudakis, "A Generalized Approach for Image Indexing and Retrieval Based on 2-D Strings," Intelligent Image Database Systems, S.-K. Chang, E. Jungert, and G. Tortora, eds., pp. 197-218. World Scientific Pub. Co., 1996
    [14] S.Y. Lee and F.J. Hsu, “2D C-String: A New Spatial Knowledge Representation for Image Database Systems,” Pattern Recognition, vol. 23, pp. 1077-1088, Oct. 1990.
    [15] S.Y. Lee, M.K. Shan, and W.P. Yang, “Similarity Retrieval of Iconic Image Database,” Pattern Recognition, vol. 22, no. 6, pp. 675-682, 1989.
    [16] S.Y. Lee and F. Hsu, “Spatial Reasoning and Similarity Retrieval of Images using 2D C-Strings Knowledge Representation,” Pattern Recognition, vol. 25, no. 3, pp. 305-318, 1992.
    [17] H.V. Jagadish, “A Retrieval Technique for Similar Shapes,” Proc. ACM SIGMOD Int'l Conf. Management of Data, pp. 208-217, 1991.
    [18] Hou, Hsu, Liu, and Chiu, "A Content-Based Indexing Technique Using Relative Geometry Features," SPIE 92, vol. 1,662, pp. 59-68, 1992.
    [19] C. Faloutsos, R. Barber, M. Flicker, J. Hafner, W. Niblack, and W. Equitz, "Efficient and effective querying by image content," J. Intell. Information Systems," vol. 3, pp. 231-262, 1994.
    [20] J.K. Wu and A.D. Narasimhalu, "Identifying Faces Using Multiple Retrievals," IEEE Multimedia, pp. 27-38, Summer 1994.
    [21] W.I. Gorsky and R. Mehrotra, "Index-Based Object Recognition in Pictorial Data Management," Computer Vision, Graphics, and Image Processing, vol. 52, pp. 416-436, 1990.
    [22] F. Rabitti and P. Savino, “An Information Retrieval Approach for Image Databases,” Proc. 18th VLDB, pp. 574–584, 1992.
    [23] N. Roussopoulos, S. Kelley, and F. Vincent, “Nearest Neighbor Queries,” Proc. ACM SIGMOD Int'l Conf. Management of Data, pp. 71-79, 1995.
    [24] T. Brinkhoff, H.-P. Kriegel, R. Schneider, and B. Seeger, “Multi-Step Processing of Spatial Joins,” Proc. ACM SIGMOD Conf. Management of Data, 1994.
    [25] K. Hinrichs and J. Nievergelt, "The Grid-File: A Data Structure to Support Proximity Queries on Spatial Objects," Technical Report 54, Institut${\rm f\ddot ur}$Informatik, ETH, Zurich, July 1983.
    [26] J. Orenstein, “Spatial Query Processing in an Object-Oriented Database System,” Proc. Fifth ACM-SIGMOD Conf., pp. 326-336, 1986.
    [27] C. Faloutsos and S. Roseman, "Fractals for Secondary Key Retrival," Proc. Symp. Principles of Database Systems, SIGMOD-SIGACT PODS, 1989.
    [28] H.V. Jagadish, "Linear Clustering of Objects with Multiple Attributes," Proc. Int'l Conf. Management of Data, pp. 332-342, ACM SIGMOD, 1990.
    [29] J.L. Bentley, "Multidimensional Binary Search Trees Used for Associative Searching," Comm. ACM, vol. 18, no. 9, pp. 509-517, 1975.
    [30] A. Guttman, “R-Trees: A Dynamic Index Structure for Spatial Searching,” Proc. ACM SIGMOD Conf. Management of Data, 1984.
    [31] N. Roussopoulos and D. Leifker, “Direct Spatial Search on Pictorial Databases Using Packed R-trees,” Proc. ACM SIGMOD Conf. Management of Data, 1985.
    [32] T. Sellis, N. Roussopoulos, and C. Faloutsos, “The R+-Tree: A Dynamic Index for Multidimensional Objects,” Proc. 13th Int'l Conf. Very Large Data Bases (VLDB), 1987.
    [33] N. Beckmann, H.-P. Kriegel, R. Schneider, and B. Seeger, “The R*-Tree: An Efficient and Robust Access Method for Points and Rectangles,” Proc. ACM SIGMOD Conf. Management of Data, 1990.
    [34] I. Kamel and C. Faloutsos, "Hilbert R-Tree: An Improved R-Tree using Fractals," Proc. Int'l Conf. Very Large Data Bases, 1994.
    [35] M. Otterman, "Approximate Matching with High Dimensionality R-Trees," MSc scholarly paper, Dept. of Computer Science, Univ. of Maryland, College Park, Md., 1992.
    [36] I. Kapouleas, "Segmentation and Feature Extraction for Magnetic Resonance Brain Image Analysis," Proc. 10th Int'l Conf. Pattern Recognition, pp. 583-590, 1990.
    [37] S. Dellepiane, G. Venturi, and G. Vernazza, "Model Generation and Model Matching of Real Images by a Fuzzy Approach," Pattern Recognition, vol. 25, no. 2, pp. 115-137, 1992.
    [38] S.V. Raman, S. Sarkar, and K.L. Boyer, “Hypothesizing Structures in Edge-Focused Cerebral Magnetic Resonance Images Using Graph-Theoretic Cycle Enumeration,” Computer Vision, Graphics, and Image Processing, vol. 57, no. 1, pp. 81-98, 1993.
    [39] S.C. Orphanoudakis, E.G. Petrakis, and P. Kofakis, "A Medical Image Database System for Tomographic Images," Proc. Computer Assisted Radiology, CAR89, pp. 618-622,Berlin, June 1989.
    [40] E.G.M. Petrakis and C. Faloutsos, "Similarity Searching in Medical Image Databases," Technical Report 01, Multimedia Systems Institute of Crete, Chania, Crete, July 1994.

    Index Terms:
    Image database, image retrieval by content, query by example, image content representation, attributed relational graph, image indexing, R-tree, similarity searching.
    Euripides G.M. Petrakis, Christos Faloutsos, "Similarity Searching in Medical Image Databases," IEEE Transactions on Knowledge and Data Engineering, vol. 9, no. 3, pp. 435-447, May-June 1997, doi:10.1109/69.599932
    Usage of this product signifies your acceptance of the Terms of Use.