This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Boosting Framework for Visuality-Preserving Distance Metric Learning and Its Application to Medical Image Retrieval
January 2010 (vol. 32 no. 1)
pp. 30-44
ASCII Text
x 
 
Liu Yang, Rong Jin, Lily Mummert, Rahul Sukthankar, Adam Goode, Bin Zheng, Steven C.H. Hoi, Mahadev Satyanarayanan, "A Boosting Framework for Visuality-Preserving Distance Metric Learning and Its Application to Medical Image Retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 30-44, January, 2010.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2008.273,
author = {Liu Yang and Rong Jin and Lily Mummert and Rahul Sukthankar and Adam Goode and Bin Zheng and Steven C.H. Hoi and Mahadev Satyanarayanan},
title = {A Boosting Framework for Visuality-Preserving Distance Metric Learning and Its Application to Medical Image Retrieval},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {32},
number = {1},
issn = {0162-8828},
year = {2010},
pages = {30-44},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2008.273},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - A Boosting Framework for Visuality-Preserving Distance Metric Learning and Its Application to Medical Image Retrieval
IS - 1
SN - 0162-8828
SP30
EP44
EPD - 30-44
A1 - Liu Yang,
A1 - Rong Jin,
A1 - Lily Mummert,
A1 - Rahul Sukthankar,
A1 - Adam Goode,
A1 - Bin Zheng,
A1 - Steven C.H. Hoi,
A1 - Mahadev Satyanarayanan,
PY - 2010
KW - Machine learning
KW - image retrieval
KW - distance metric learning
KW - boosting.
VL - 32
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Liu Yang, Carnegie Mellon University, Pittsburgh
Rong Jin, Michigan State University, East Lansing
Lily Mummert, Intel Research, Pittsburgh
Rahul Sukthankar, Intel Research, Pittsburgh
Adam Goode, Carnegie Mellon University, Pittsburgh
Bin Zheng, University of Pittsburgh, Pittsburgh
Steven C.H. Hoi, Nanyang Technological University, Singapore
Mahadev Satyanarayanan, Carnegie Mellon University, Pittsburgh
Similarity measurement is a critical component in content-based image retrieval systems, and learning a good distance metric can significantly improve retrieval performance. However, despite extensive study, there are several major shortcomings with the existing approaches for distance metric learning that can significantly affect their application to medical image retrieval. In particular, “similarity” can mean very different things in image retrieval: resemblance in visual appearance (e.g., two images that look like one another) or similarity in semantic annotation (e.g., two images of tumors that look quite different yet are both malignant). Current approaches for distance metric learning typically address only one goal without consideration of the other. This is problematic for medical image retrieval where the goal is to assist doctors in decision making. In these applications, given a query image, the goal is to retrieve similar images from a reference library whose semantic annotations could provide the medical professional with greater insight into the possible interpretations of the query image. If the system were to retrieve images that did not look like the query, then users would be less likely to trust the system; on the other hand, retrieving images that appear superficially similar to the query but are semantically unrelated is undesirable because that could lead users toward an incorrect diagnosis. Hence, learning a distance metric that preserves both visual resemblance and semantic similarity is important. We emphasize that, although our study is focused on medical image retrieval, the problem addressed in this work is critical to many image retrieval systems. We present a boosting framework for distance metric learning that aims to preserve both visual and semantic similarities. The boosting framework first learns a binary representation using side information, in the form of labeled pairs, and then computes the distance as a weighted Hamming distance using the learned binary representation. A boosting algorithm is presented to efficiently learn the distance function. We evaluate the proposed algorithm on a mammographic image reference library with an Interactive Search-Assisted Decision Support (ISADS) system and on the medical image data set from ImageCLEF. Our results show that the boosting framework compares favorably to state-of-the-art approaches for distance metric learning in retrieval accuracy, with much lower computational cost. Additional evaluation with the COREL collection shows that our algorithm works well for regular image data sets.

[1] P. Croskerry, “The Theory and Practice of Clinical Decision-Making,” Canadian J. Anesthesia, vol. 52, no. 6, pp. R1-R8, 2005.
[2] L. Yang, R. Jin, R. Sukthankar, B. Zheng, L. Mummert, M. Satyanarayanan, M. Chen, and D. Jukic, “Learning Distance Metrics for Interactive Search-Assisted Diagnosis of Mammograms,” Proc. SPIE Conf. Medical Imaging, 2007.
[3] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-Based Image Retrieval at the End of the Early Years,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 22, no. 12, pp.1349-1380, Dec. 2000.
[4] H. Kim, P. Howland, and H. Park, “Dimension Reduction in Text Classification with Support Vector Machines,” J. Machine Learning Research, vol. 6, pp. 37-53, 2005.
[5] L. Vandenberghe and S. Boyd, “Semidefinite Programming,” SIAM Rev., vol. 38, no. 1, pp. 49-95, 1996.
[6] K. Weinberger, J. Blitzer, and L. Saul, “Distance Metric Learning for Large Margin Nearest Neighbor Classification,” Advances in Neural Information Processing Systems, MIT Press, http://www.seas.upenn.edu/kilianwlmnn, 2006.
[7] D. Gur, J.H. Sumkin, L.A. Hardesty, and H.E. Rockette, “Computer-Aided Detection of Breast Cancer: Has Promise Outstripped Performance?” J. Nat'l Cancer Inst., vol. 96, pp. 717-718, 2004.
[8] R.M. Nishikawa and M. Kallergi, “Computer-Aided Detection in Its Present Form Is Not an Effective Aid for Screening Mammography,” Medical Physics, vol. 33, pp. 811-814, 2006.
[9] T.M. Freer and M.J. Ulissey, “Screening Mammography with Computer-Aided Detection: Prospective Study of 12,860 Patients in a Community Breast Center,” Radiology, vol. 220, pp. 781-786, 2001.
[10] L.A. Khoo, P. Taylor, and R.M. Given-Wilson, “Computer-Aided Detection in the United Kingdom National Breast Screening Programme: Prospective Study,” Radiology, vol. 237, pp. 444-449, 2005.
[11] J.M. Ko, M.J. Nicholas, J.B. Mendel, and P.J. Slanetz, “Prospective Assessment of Computer-Aided Detection in Interpretation of Screening Mammograms,” Am. J. Roentgenology, vol. 187, pp. 1483-1491, 2006.
[12] M.L. Giger, Z. Huo, C.J. Vyborny, L. Lam, I. Bonta, K. Horsch, R.M. Nishikawa, and I. Rosenbourgh, “Intelligent CAD, Workstation for Breast Imaging Using Similarity to Known Lesions and Multiple Visual Prompt Aides,” Proc. SPIE Conf. Medical Imaging '02: Image Processing, pp. 768-773, 2002.
[13] I. El-Naga, Y. Yang, N.P. Galatsanos, R.M. Nishikawa, and M.N. Wernick, “A Similarity Learning Approach to Content-Based Image Retrieval: Application to Digital Mammography,” IEEE Trans. Medical Imaging, vol. 23, no. 10, pp. 1233-1244, Oct. 2004.
[14] C. Wei, C. Li, and R. Wilson, “A General Framework for Content-Based Medical Image Retrival with Its Application to Mammograms,” Proc. SPIE Conf. Medical Imaging '05: PACS and Imaging Informatics, pp. 134-143, 2005.
[15] H. Alto, R.M. Rangayyan, and J.E. Desautels, “Content-Based Retrieval and Analysis of Mammographic Masses,” J. Electronic Imaging, vol. 14, pp. 023016-1-023016-17, 2005.
[16] C. Muramatsu, Q. Li, K. Suzuki, R.A. Schmidt, J. Shiraishi, G.M. Newstead, and K. Doi, “Investigation of Psychophysical Measure for Evaluation of Similar Images for Mammographic Masses: Preliminary Results,” Medical Physics, vol. 32, pp. 2295-2304, 2005.
[17] B. Zheng et al., “Interactive Computer Aided Diagnosis of Breast Masses: Computerized Selection of Visually Similar Image Sets from a Reference Library,” Academic Radiology, vol. 14, no. 8, pp.917-927, 2007.
[18] G.D. Tourassi, B. Harrawood, S. Singh, J.Y. Lo, and C.E. Floyd, “Evaluation of Information-Theoretic Similarity Measures for Content-Based Retrieval and Detection of Masses in Mammograms,” Medical Physics, vol. 34, pp. 140-150, 2007.
[19] Y. Tao, S.B. Lo, M.T. Freedman, and J. Xuan, “A Preliminary Study of Content-Based Mammographic Masses Retrieval Book,” Proc. SPIE Conf. Medical Imging '07, 2007.
[20] R.M. Nishikawa, “Current Status and Future Directions of Computer-Aided Diagnosis in Mammography,” Computerized Medical Imaging Graphics, vol. 31, pp. 224-235, 2007.
[21] L. Yang and R. Jin, “Distance Metric Learning: A Comprehensive Survey,” technical report, Michigan State Univ., 2006.
[22] E. Xing, A. Ng, M. Jordan, and S. Russell, “Distance Metric Learning with Application to Clustering with Side Information,” Advances in Neural Information Processing Systems, MIT Press, 2003.
[23] J.T. Kwok and I.W. Tsang, “Learning with Idealized Kernels,” Proc. Int'l Conf. Machine Learning, 2003.
[24] T. Hastie and R. Tibshirani, “Discriminant Adaptive Nearest Neighbor Classification,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 6, pp. 607-616, June 1996.
[25] A.B. Hillel, T. Hertz, N. Shental, and D. Weinshall, “Learning Distance Functions Using Equivalence Relations,” Proc. Int'l Conf. Machine Learning, 2003.
[26] S.C.H. Hoi, W. Liu, M.R. Lyu, and W.-Y. Ma, “Learning Distance Metrics with Contextual Constraints for Image Retrieval,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[27] M. Sugiyama, “Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction,” Proc. Int'l Conf. Machine Learning, 2006.
[28] T.-K. Kim, S.-F. Wong, B. Stenger, J. Kittler, and R. Cipolla, “Incremental Linear Discriminant Analysis Using Sufficient Spanning Set Approximations,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[29] M. Schultz and T. Joachims, “Learning a Distance Metric from Relative Comparisons,” Advances in Neural Information Processing Systems, MIT Press, 2004.
[30] J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, “Neighbourhood Components Analysis,” Advances in Neural Information Processing Systems, MIT Press, 2005.
[31] L. Yang, R. Jin, R. Sukthankar, and Y. Liu, “An Efficient Algorithm for Local Distance Metric Learning,” Proc. Nat'l Conf. Artificial Intelligence, 2006.
[32] A.B. Hillel and D. Weinshall, “Learning Distance Function by Coding Similarity,” Proc. Int'l Conf. Machine Learning, 2007.
[33] A. Woznica, A. Kalousis, and M. Hilario, “Learning to Combine Distances for Complex Representations,” Proc. Int'l Conf. Machine Learning, 2007.
[34] W. Zhang, X. Xue, Z. Sun, Y. Guo, and H. Lu, “Optimal Dimensionality of Metric Space for Classification,” Proc. Int'l Conf. Machine Learning, 2007.
[35] H. Wang, H. Zha, and H. Qin, “Dirichlet Aggregation: Unsupervised Learning Towards an Optimal Metric for Proportional Data,” Proc. Int'l Conf. Machine Learning, 2007.
[36] S. Zhou, B. Georgescu, D. Comaniciu, and J. Shao, “BoostMotion: Boosting a Discriminative Similarity Function for Motion Estimation,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[37] B. Babenko, P. Dollár, and S. Belongie, “Task Specific Local Region Matching,” Proc. Int'l Conf. Computer Vision, 2007.
[38] F. Li, J. Yang, and J. Wang, “A Transductive Framework of Distance Metric Learning by Spectral Dimensionality Reduction,” Proc. Int'l Conf. Machine Learning, 2007.
[39] P. Dollár, V. Rabaud, and S. Belongie, “Non-Isometric Manifold Learning: Analysis and an Algorithm,” Proc. Int'l Conf. Machine Learning, 2007.
[40] J.V. Davis, B. Kulis, P. Jain, S. Sra, and I.S. Dhillon, “Information-Theoretic Metric Learning,” Proc. Int'l Conf. Machine Learning, 2007.
[41] J. Dillon, Y. Mao, G. Lebanon, and J. Zhang, “Statistical Translation, Heat Kernels, and Expected Distance,” Proc. Conf. Uncertainty in Artificial Intelligence, 2007.
[42] L. Yang, R. Jin, and R. Sukthankar, “Bayesian Active Distance Metric Learning,” Proc. Conf. Uncertainty in Artificial Intelligence, 2007.
[43] L. Torresani and K. Lee, “Large Margin Component Analysis,” Advances in Neural Information Processing Systems, MIT Press, 2007.
[44] K.Q. Weinberger, F. Sha, Q. Zhu, and L.K. Saul, “Graph Laplacian Regularization for Large-Scale Semidefinite Programming,” Advances in Neural Information Processing Systems, MIT Press, 2007.
[45] D. Zhou, J. Huang, and B. Schölkopf, “Learning with Hypergraphs: Clustering, Classification, and Embedding,” Advances in Neural Information Processing Systems, MIT Press, 2007.
[46] Z. Zhang and J. Wang, “MLLE: Modified Locally Linear Embedding Using Multiple Weights,” Advances in Neural Information Processing Systems, MIT Press, 2007.
[47] A. Frome, Y. Singer, and J. Malik, “Image Retrieval and Classification Using Local Distance Functions,” Advances in Neural Information Processing Systems, MIT Press, 2007.
[48] O. Boiman and M. Irani, “Similarity by Composition,” Advances in Neural Information Processing Systems, MIT Press, 2007.
[49] M. Belkin and P. Niyogi, “Convergence of Laplacian Eigenmaps,” Advances in Neural Information Processing Systems, MIT Press, 2007.
[50] T. Hertz, A.B. Hillel, and D. Weinshall, “Boosting Margin Based Distance Functions for Clustering,” Proc. Int'l Conf. Machine Learning, http://www.cs.huji.ac.il/daphna/codeDistBoost.zip , 2004.
[51] T. Hertz, A.B. Hillel, and D. Weinshall, “Learning a Kernel Function for Classification with Small Training Samples,” Proc. Int'l Conf. Machine Learning, 2006.
[52] G. Shakhnarovich, “Learning Task-Specific Similarity,” PhD thesis, Massachusetts Inst. of Tech nology, 2005.
[53] Y. Freund and R.E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” J.Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1997.
[54] S. Agarwal, J. Wills, L. Cayton, G. Lanckriet, D. Kriegman, and S. Belongie, “Generalized Non-Metric Multidimensional Scaling,” Proc. Int'l Conf. Artificial Intelligence and Statistics, 2007.
[55] B. Moghaddham and M.-H. Yang, “Gender Classification with Support Vector Machines,” Proc. Int'l Conf. Face and Gesture Recognition, 2000.
[56] Y. Ke, D. Hoiem, and R. Sukthankar, “Computer Vision for Music Identification,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2005.
[57] J. Zhang and R. Yan, “On the Value of Pairwise Constraints in Classification and Consistency,” Proc. Int'l Conf. Machine Learning, 2007.
[58] R.E. Schapire, “Theoretical Views of Boosting and Applications,” Proc. Int'l Conf. Algorithmic Learning Theory, 1999.
[59] R. Salakhutdinov, S. Roweis, and Z. Ghahramani, “On the Convergence of Bound Optimization Algorithms,” Proc. Conf. Uncertainty in Artificial Intelligence, 2003.
[60] ImageCLEF, http://ir.shef.ac.ukimageclef/, 2009.
[61] B. Zheng, J.K. Leader, G. Abrams, B. Shindel, V. Catullo, W.F. Good, and D. Gur, “Computer-Aided Detection Schemes: The Effect of Limiting the Number of Cued Regions in Each Case,” Am. J. Roentgenology, vol. 182, pp. 579-583, 2004.
[62] B. Zheng, A. Lu, L.A. Hardesty, J.H. Sumkin, C.M. Kakim, M.A. Ganott, and D. Gur, “A Method to Improve Visual Similarity of Breast Masses for an Interactive Computer-Aided Diagnosis Environment,” Medical Physics, vol. 33, pp. 111-117, 2006.
[63] M. Lades, J.C. Vorbruggen, J. Buhmann, J. Lange, C. von der Malsburg, R. Wurtz, and W. Konen, “Distortion Invariant Object Trcognition in the Dynamic Line Architecture,” IEEE Trans. Computers, vol. 42, no. 3, pp. 300-311, Mar. 1993.

Index Terms:
Machine learning, image retrieval, distance metric learning, boosting.
Citation:
Liu Yang, Rong Jin, Lily Mummert, Rahul Sukthankar, Adam Goode, Bin Zheng, Steven C.H. Hoi, Mahadev Satyanarayanan, "A Boosting Framework for Visuality-Preserving Distance Metric Learning and Its Application to Medical Image Retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 30-44, Jan. 2010, doi:10.1109/TPAMI.2008.273
Usage of this product signifies your acceptance of the Terms of Use.