This Article 
 Bibliographic References 
 Add to: 
Toward Improved Ranking Metrics
October 2000 (vol. 22 no. 10)
pp. 1132-1143

Abstract—In many computer vision algorithms, a metric or similarity measure is used to determine the distance between two features. The Euclidean or SSD (sum of the squared differences) metric is prevalent and justified from a maximum likelihood perspective when the additive noise distribution is Gaussian. Based on real noise distributions measured from international test sets, we have found that the Gaussian noise distribution assumption is often invalid. This implies that other metrics, which have distributions closer to the real noise distribution, should be used. In this paper, we consider three different applications: content-based retrieval in image databases, stereo matching, and motion tracking. In each of them, we experiment with different modeling functions for the noise distribution and compute the accuracy of the methods using the corresponding distance measures. In our experiments, we compared the SSD metric, the SAD (sum of the absolute differences) metric, the Cauchy metric, and the Kullback relative information. For several algorithms from the research literature which used the SSD or SAD, we showed that greater accuracy could be obtained by using the Cauchy metric instead.

[1] H. Akaike, “Information Theory and an Extension of the Maximum Likelihood Principle,” Proc. Second Int'l Symp. Information Theory, pp. 267–281, 1973.
[2] S. Barnard and M. Fischler, “Computational Stereo,” ACM Computing Surveys, vol. 14, no. 4, pp. 553–572, 1982.
[3] D.N. Bhat and S.K. Nayar, “Ordinal Measures for Image Correspondence,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 415–423, Apr. 1998.
[4] M.J. Black, “Robust Incremental Optical Flow,” PhD thesis, Yale Univ., Sept. 1992.
[5] R. Boie and I. Cox, “An Analysis of Camera Noise,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 6, pp. 671–674, June 1992.
[6] I. Cox, S. Hingorani, and S. Rao, “A Maximum Likelihood Stereo Algorithm,” Computer Vision and Image Understanding, vol. 63, no. 3, pp. 542–567, 1996.
[7] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” IEEE Computer, 1995.
[8] A. Fusiello and V. Roberto, “Efficient Stereo with Multiple Windowing,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 858-863, 1997.
[9] T. Gevers and A. Smeulders, “Color-Based Object Recognition,” Pattern Recognition, vol. 32, no. 3, pp. 453–464, 1999.
[10] W. Grimson, “Computational Experiments with a Feature Based Stereo Algorithm,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 1, pp. 17–34, Jan. 1985.
[11] J. Hafner, H.S. Sawhney, W. Equitz, M. Flickner, and W. Niblack, “Efficient Color Histogram Indexing for Quadratic Form Distance Functions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 7, pp. 729-736, July 1995.
[12] F.R. Hampel, E.M. Ronchetti, P.J. Rousseeuw, and W.A. Stahel, Robust Statistic: The Approach Based on Influence Functions. New York: John Wiley&Sons, 1986.
[13] R.M. Haralick and L.G. Shapiro, Computer and Robot Vision. New York: Addison-Wesley, 1993.
[14] P.J. Huber, Robust Statistic. New York: John Wiley&Sons, 1981.
[15] D.P. Huijsmans and M.S. Lew, “Efficient Content-Based Image Retrieval in Digital Picture Collections Using Projections: (Near)Copy Locations,” Proc. 13th Int'l Conf. Pattern Recognition, vol. 3, pp. 104–108, 1996.
[16] D.P. Huijsmans, M.S. Lew, and D. Denteneer, “Quality Measures for Interactive Image Retrieval with a Performance Evaluation of Two$3\times3$Texel-Based Methods,” Lecture Notes in Computer Science, vol. 1,311, no. 2, pp. 22–29, 1997.
[17] P.M. Kelly and T.M. Cannon, “CANDID: Comparison Algorithm for Navigating Digital Image Databases,” Proc. 17th Int'l Working Conf. Scientific and Statistical Database Management, pp. 252–258, 1994.
[18] P.M. Kelly, T.M. Cannon, and J.E. Barros, “Efficiency Issues Related to Probability Density Function Comparison,” Proc. SPIE—Storage and Retrieval for Image and Video Databases, vol. 2,670, no. 4, pp. 42–49, 1996.
[19] P.M. Kelly, T.M. Cannon, and D.R. Hush, “Query By Image Example: The CANDID Approach,” Proc. SPIE—Storage and Retrieval for Image and Video Databases, vol. 2,420, no. 3, pp. 238–248, 1995.
[20] S. Kullback, Information Theory and Statistics. Dover Publications, 1968.
[21] M.S. Lew, T.S. Huang, and K. Wong, “Learning and Feature Selection in Stereo Matching,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 9, pp. 869-881, Sept. 1994.
[22] W. Luo and H. Maitre, “Using Surface Model to Correct and Fit Disparity Data in Stereo Vision,” Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 60–64, 1990.
[23] D. Marr and T. Poggio, “A Computational Theory of Human Stereo Vision,” Proc. Royal Soc. London, vol. 204, pp. 301–328, 1976.
[24] J. Rissanen, “Modeling by Shortest Data Description,” Automatica, vol. 14, pp. 465–471, 1978.
[25] P. Rousseeuw and A. Leory, Robust Regression and Outlier Detection. Wiley Series in Probability and Statistics, 1987.
[26] H. Sawhney and J. Hafner, “Efficient Color Histogram Indexing,” Proc. Int'l Conf. Image Processing, pp. 66-70, 1994.
[27] N. Sebe, M.S. Lew, and D.P. Huijsmans, “Which Ranking Metric Is Optimal? with Applications in Image Retrieval and Stereo Matching,” Proc. Int'l Conf. Pattern Recognition, pp. 265–271, 1998.
[28] H.S. Stone and C.S. Li, “Image Matching by Means of Intensity and Texture Matching in the Fourier Domain,” Proc. SPIE—Electronic Imaging: Science and Technology, Jan. 1996.
[29] M.J. Swain and B.H. Ballard, “Color Indexing,” Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[30] L. Tang, Y. Kong, L.S. Chen, C. R. Lansing, and T.S. Huang, “Performance Evaluation of a Facial Feature Tracking Algorithm,” Proc. NSF/ARPA Workshop: Performance vs. Methodology in Computer Vision, pp. 218–229, 1994.

Index Terms:
Maximum likelihood, ranking metrics, content-based retrieval, color indexing, stereo matching, motion tracking.
Nicu Sebe, Michael S. Lew, Dionysius P. Huijsmans, "Toward Improved Ranking Metrics," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1132-1143, Oct. 2000, doi:10.1109/34.879793
Usage of this product signifies your acceptance of the Terms of Use.