This Article 
 Bibliographic References 
 Add to: 
Weighted Pseudometric Discriminatory Power Improvement Using a Bayesian Logistic Regression Model Based on a Variational Method
February 2008 (vol. 30 no. 2)
pp. 253-266
In this paper, we investigate the effectiveness of a Bayesian logistic regression model to compute the weights of a pseudo-metric, in order to improve its discriminatory capacity and thereby increase image retrieval accuracy. In the proposed Bayesian model, the prior knowledge of the observations is incorporated and the posterior distribution is approximated by a tractable Gaussian form using variational transformation and Jensen’s inequality, which allow a fast and straightforward computation of the weights. The pseudo-metric makes use of the compressed and quantized versions of wavelet decomposed feature vectors, and in our previous work, the weights were adjusted by classical logistic regression model. A comparative evaluation of the Bayesian and classical logistic regression models is performed for content-based image retrieval as well as for other classification tasks, in a decontextualized evaluation framework. In this same framework, we compare the Bayesian logistic regression model to some relevant state-of-the-art classification algorithms. Experimental results show that the Bayesian logistic regression model outperforms these linear classification algorithms, and is a significantly better tool than the classical logistic regression model to compute the pseudo-metric weights and improve retrieval and classification performance. Finally, we perform a comparison with results obtained by other retrieval methods.

[1] Y. Rui, T.S. Huang, and S.-F. Chang, “Image Retrieval: Current Techniques, Promising Directions, and Open Issues,” J. Visual Comm. and Image Representation, vol. 10, pp. 39-62, 1999.
[2] A. Smeulder, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-Based Image Retrieval at the End of the Early Years,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, pp.1349-1380, 2000.
[3] N. Vasconcelos, “On the Efficient Evaluation of Probabilistic Similarity Functions for Image Retrieval,” IEEE Trans. Information Theory, vol. 50, pp. 1482-1496, 2004.
[4] J. Peng, B. Bhanu, and S. Qing, “Learning Feature Relevance and Similarity Metrics in Image Databases,” Proc. IEEE Workshop Content-Based Access of Image and Video Libraries, pp. 14-18, 1998.
[5] S. Aksoy and R.M. Haralick, “Feature Normalization and Likelihood-Based Similarity Measures for Image Retrieval,” Pattern Recognition Letters, vol. 22, no. 5, pp. 563-582, 2001.
[6] G. Caenen and E.J. Pauwels, “Logistic Regression Models for Relevance Feedback in Content-Based Image Retrieval,” Proc. SPIE Storage and Retrieval for Media Databases, , vol. 4676, pp. 49-58, 2002.
[7] S. Aksoy, R.M. Haralick, F.A. Cheikh, and M. Gabbouj, “A Weighted Distance Approach to Relevance Feedback,” Proc. 15th Int'l Conf. Pattern Recognition, vol. 4, pp. 812-815, 2000.
[8] R. Ksantini, D. Ziou, and F. Dubeau, “Image Retrieval Based on Region Separation and Multiresolution Analysis,” Int'l J. Wavelets, Multiresolution and Information Processing, vol. 4, no. 1, pp. 147-175, 2006.
[9] T.S. Jaakkola and M.I. Jordan, “Bayesian Parameter Estimation via Variational Methods,” Statistics and Computing, vol. 10, no. 1, pp.25-37, 2000.
[10] C.C. Clogg, D.B. Rubin, N. Schenker, B. Schultz, and L. Widman, “Multiple Imputation of Industry and Occupation Codes in Census Public-Use Samples Using Bayesian Logistic Regression,” J. Am. Statistical Assoc., vol. 86, pp. 68-78, 1991.
[11] R. Weiss, R. Berk, W. Li, and M. Farrell-Ross, “Death Penalty Charging in Los Angeles County: An Illustrative Data Analysis Using Skeptical Priors,” Sociological Methods and Research, vol. 28, pp. 91-115, 1999.
[12] F. Galindo-Garre, J.K. Vermunt, and W.P. Bergsma, “Bayesian Posterior Estimation of Logit Parameters with Small Samples,” Sociological Methods and Research, vol. 33, pp. 1-30, 2004.
[13] P. Congdon, Bayesian Statistical Modelling. John Wiley & Sons, 2001.
[14] G. Koop and D. Poirier, “An Empirical Investigation of Wagner's Hypothesis by Using a Model Occurrence Framework,” J. Royal Statistical Soc., Series A, vol. 158, no. 1, pp. 123-141, 1995.
[15] R. Gerlach, R. Bird, and A.D. Hall, “A Bayesian Approach to Variable Selection in Logistic Regression with Application to Predicting Earnings Direction from Accounting Information,” Australian and New Zealand J. Statistics, vol. 44, no. 2, pp. 155-168, 2002.
[16] S.J. Roberts, D. Husmeier, I. Rezek, and W.D. Penny, “Bayesian Approaches to Gaussian Mixture Modeling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1133-1142, Nov. 1998.
[17] T. Deselaers, D. Keysers, and H. Ney, “Classification Error Rate for Quantitative Evaluation of Content-Based Image Retrieval Systems,” Proc. 17th Int'l Conf. Pattern Recognition, vol. 2, pp. 505-508, 2004.
[18] M.L. Yiu and N. Mamoulis, “Iterative Projected Clustering by Subspace Mining,” IEEE Trans. Knowledge and Data Eng., vol. 17, no. 2, pp. 176-189, Feb. 2005.
[19] H.A. Murthy and S. Haykin, “Bayesian Classification of Surface-Based Ice-Radar Images,” IEEE J. Oceanic Eng., vol. 12, no. 3, pp.493-501, 1987.
[20] M.L. Kherfi and D. Ziou, “Relevance Feedback for CBIR: A New Approach Based on Probabilistic Feature Weighting with Positive and Negative Examples,” IEEE Trans. Image Processing, vol. 15, no. 4, pp. 1017-1030, 2006.
[21] E. Xing, A. Ng, M. Jordan, and S. Russell, “Distance Metric Learning, with Application to Clustering with Side-Information,” Advances in Neural Information Processing Systems, vol. 15, pp. 505-512, 2003.
[22] N. Vasconcelos, “Minimum Probability of Error Image Retrieval,” IEEE Trans. Signal Processing, vol. 52, pp. 2322-2336, 2004.
[23] T. Westerveld and A.P. de Vries, “Generative Probabilistic Models for Multimedia Retrieval: Query Generation against Document Generation,” IEE Proc. Vision, Image, and Signal Processing, vol. 152, no. 6, pp. 852-858, 2005.
[24] V. Lavrenko, S.L. Feng, and R. Manmatha, “Statistical Models for Automatic Video Annotation and Retrieval,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 3, pp. 17-21, 2003.
[25] S. Ghebreab, C.C. Jaffe, and A.W.M. Smeulders, “Population-Based Incremental Interactive Concept Learning for Image Retrieval by Stochastic String Segmentations,” IEEE Trans. Medical Imaging, vol. 23, no. 6, pp. 676-689, 2004.
[26] T. Hastie, R. Tibshirani, and J.H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2001.
[27] J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, “Neighbourhood Components Analysis,” Advances in Neural Information Processing Systems, vol. 17, pp. 513-520, 2005.
[28] K. Weinberger, J. Blitzer, and L. Saul, “Neighbourhood Components Analysis,” Advances in Neural Information Processing Systems, vol. 18, pp. 1473-1480, 2006.
[29] T. Hastie and R. Tibshirani, “Discriminant Adaptive Nearest Neighbor Classification,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, pp. 607-616, 1996.
[30] C. Domeniconi, D. Gunopulos, and J. Peng, “Large Margin Nearest Neighbor Classifiers,” IEEE Trans. Neural Networks, vol. 16, no. 4, pp. 899-909, 2005.
[31] A. Globerson and S. Roweis, “Metric Learning by Collapsing Classes,” Advances in Neural Information Processing Systems, vol. 18, pp. 451-458, 2006.
[32] S. Chopra, R. Hadsell, and Y. LeCun, “Learning a Similarity Metric Discriminatively, With Application to Face Verification,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp.539-546, 2005.
[33] J.S. Long, “Regression Models for Categorical and Limited Dependent Variables,” Advanced Quantitative Techniques in the Social Sciences. Sage Publications, 1997.
[34] D. Hosmer and S. Lemeshow, Applied Logistic Regression. John Wiley & Sons, 1989.
[35] J.S. Cramer, Econometric Applications of Maximum Likelihood Methods. Cambridge Univ. Press, 1986.
[36] P. Komarek, “Logistic Regression for Data Mining and High-Dimensional Classification,” PhD dissertation, School of Computer Science, Carnegie Mellon Univ., 2004.
[37] R.J. Freund and P.D. Minton, Regression Methods: A Tool for Data Analysis. Marcel Dekker, 1979.
[38] A. Albert and J.A. Anderson, “On the Existence of Maximum Likelihood Estimates in Logistic Regression Models,” Biometrika, vol. 71, pp. 1-10, 1984.
[39] D.J. Spiegelhalter and S.L. Lauritzen, “Sequential Updating of Conditional Probabilities on Directed Graphical Structures,” Networks, vol. 20, pp. 579-605, 1990.
[40] M.S. Bazaraa and C.M. Shetty, Nonlinear Programming: Theory and Algorithms. John Wiley & Sons, 1979.
[41] P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization. Academic Press, 1989.
[42] M. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge Univ. Press, 2000.
[43] M. Tipping, “Sparse Bayesian Learning and the Relevance Vector Machine,” J. Machine Learning Research, vol. 1, pp. 211-244, 2001.
[44] N.D. Lawrence, M. Seeger, and R. Herbrich, “Fast Sparse Gaussian Process Methods: The Informative Vector Machine,” Advances in Neural Information Processing Systems, vol. 15, pp. 609-616, 2003.
[45], 2007.
[46] L. Breiman, “Bias, Variance and Arcing Classifiers,” Technical Report 460, Dept. Statistics, Univ. of California, 1996.
[47] M. Pohar, M. Blas, and S. Turk, “Comparison of Logistic Regression and Linear Discriminant Analysis: A Simulation Study,” Metodoloski Zvezki, vol. 1, no. 1, pp. 143-161, 2004.
[48] S. Wang and H. Sun, “Measuring Overlap-Rate for Cluster Merging in a Hierarchical Approach to Color Image Segmentation,” Int'l J. Fuzzy Systems, vol. 6, no. 3, pp. 147-156, 2004.
[49], 1999.
[50] A. Klautau, “Discriminative Gaussian Mixture Models: A Comparison with Kernel Classifiers,” Proc. 20th Int'l Conf. Machine Learning, pp. 353-360, 2003.
[51] I. Daubechies, Ten Lectures on Wavelets. SIAM, 1992.
[52] H. Shao, T. Svoboda, T. Tuytelaars, and L.V. Gool, “HPAT Indexing for Fast Object/Scene Recognition Based on Local Appearance,” Proc. Int'l Conf. Image and Video Retrieval, pp. 71-80, 2003.
[53] R. Fergus, P. Perona, and A. Zisserman, “Object Class Recognition by Unsupervised Scale-invariant Learning,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 264-271, 2003.

Index Terms:
Image Retrieval, Logistic Regression, Variational Method, Weighted Pseudo-Metric
Riadh Ksantini, Djemel Ziou, Bernard Colin, Fran?ois Dubeau, "Weighted Pseudometric Discriminatory Power Improvement Using a Bayesian Logistic Regression Model Based on a Variational Method," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 253-266, Feb. 2008, doi:10.1109/TPAMI.2007.1165
Usage of this product signifies your acceptance of the Terms of Use.