CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2008 vol.30 Issue No.02 - February

Subscribe

Issue No.02 - February (2008 vol.30)

pp: 253-266

ABSTRACT

In this paper, we investigate the effectiveness of a Bayesian logistic regression model to compute the weights of a pseudo-metric, in order to improve its discriminatory capacity and thereby increase image retrieval accuracy. In the proposed Bayesian model, the prior knowledge of the observations is incorporated and the posterior distribution is approximated by a tractable Gaussian form using variational transformation and Jensen’s inequality, which allow a fast and straightforward computation of the weights. The pseudo-metric makes use of the compressed and quantized versions of wavelet decomposed feature vectors, and in our previous work, the weights were adjusted by classical logistic regression model. A comparative evaluation of the Bayesian and classical logistic regression models is performed for content-based image retrieval as well as for other classification tasks, in a decontextualized evaluation framework. In this same framework, we compare the Bayesian logistic regression model to some relevant state-of-the-art classification algorithms. Experimental results show that the Bayesian logistic regression model outperforms these linear classification algorithms, and is a significantly better tool than the classical logistic regression model to compute the pseudo-metric weights and improve retrieval and classification performance. Finally, we perform a comparison with results obtained by other retrieval methods.

INDEX TERMS

Image Retrieval, Logistic Regression, Variational Method, Weighted Pseudo-Metric

CITATION

Djemel Ziou, Bernard Colin, Fran?ois Dubeau, "Weighted Pseudometric Discriminatory Power Improvement Using a Bayesian Logistic Regression Model Based on a Variational Method",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.30, no. 2, pp. 253-266, February 2008, doi:10.1109/TPAMI.2007.1165REFERENCES

- [2] A. Smeulder, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-Based Image Retrieval at the End of the Early Years,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, pp.1349-1380, 2000.- [3] N. Vasconcelos, “On the Efficient Evaluation of Probabilistic Similarity Functions for Image Retrieval,”
IEEE Trans. Information Theory, vol. 50, pp. 1482-1496, 2004.- [4] J. Peng, B. Bhanu, and S. Qing, “Learning Feature Relevance and Similarity Metrics in Image Databases,”
Proc. IEEE Workshop Content-Based Access of Image and Video Libraries, pp. 14-18, 1998.- [6] G. Caenen and E.J. Pauwels, “Logistic Regression Models for Relevance Feedback in Content-Based Image Retrieval,”
Proc. SPIE Storage and Retrieval for Media Databases, , vol. 4676, pp. 49-58, 2002.- [8] R. Ksantini, D. Ziou, and F. Dubeau, “Image Retrieval Based on Region Separation and Multiresolution Analysis,”
Int'l J. Wavelets, Multiresolution and Information Processing, vol. 4, no. 1, pp. 147-175, 2006.- [9] T.S. Jaakkola and M.I. Jordan, “Bayesian Parameter Estimation via Variational Methods,”
Statistics and Computing, vol. 10, no. 1, pp.25-37, 2000.- [12] F. Galindo-Garre, J.K. Vermunt, and W.P. Bergsma, “Bayesian Posterior Estimation of Logit Parameters with Small Samples,”
Sociological Methods and Research, vol. 33, pp. 1-30, 2004.- [13] P. Congdon,
Bayesian Statistical Modelling. John Wiley & Sons, 2001.- [14] G. Koop and D. Poirier, “An Empirical Investigation of Wagner's Hypothesis by Using a Model Occurrence Framework,”
J. Royal Statistical Soc., Series A, vol. 158, no. 1, pp. 123-141, 1995.- [15] R. Gerlach, R. Bird, and A.D. Hall, “A Bayesian Approach to Variable Selection in Logistic Regression with Application to Predicting Earnings Direction from Accounting Information,”
Australian and New Zealand J. Statistics, vol. 44, no. 2, pp. 155-168, 2002.- [21] E. Xing, A. Ng, M. Jordan, and S. Russell, “Distance Metric Learning, with Application to Clustering with Side-Information,”
Advances in Neural Information Processing Systems, vol. 15, pp. 505-512, 2003.- [24] V. Lavrenko, S.L. Feng, and R. Manmatha, “Statistical Models for Automatic Video Annotation and Retrieval,”
Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 3, pp. 17-21, 2003.- [26] T. Hastie, R. Tibshirani, and J.H. Friedman,
The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2001.- [27] J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, “Neighbourhood Components Analysis,”
Advances in Neural Information Processing Systems, vol. 17, pp. 513-520, 2005.- [28] K. Weinberger, J. Blitzer, and L. Saul, “Neighbourhood Components Analysis,”
Advances in Neural Information Processing Systems, vol. 18, pp. 1473-1480, 2006.- [29] T. Hastie and R. Tibshirani, “Discriminant Adaptive Nearest Neighbor Classification,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, pp. 607-616, 1996.- [30] C. Domeniconi, D. Gunopulos, and J. Peng, “Large Margin Nearest Neighbor Classifiers,”
IEEE Trans. Neural Networks, vol. 16, no. 4, pp. 899-909, 2005.- [31] A. Globerson and S. Roweis, “Metric Learning by Collapsing Classes,”
Advances in Neural Information Processing Systems, vol. 18, pp. 451-458, 2006.- [33] J.S. Long, “Regression Models for Categorical and Limited Dependent Variables,”
Advanced Quantitative Techniques in the Social Sciences. Sage Publications, 1997.- [34] D. Hosmer and S. Lemeshow,
Applied Logistic Regression. John Wiley & Sons, 1989.- [35] J.S. Cramer,
Econometric Applications of Maximum Likelihood Methods. Cambridge Univ. Press, 1986.- [36] P. Komarek, “Logistic Regression for Data Mining and High-Dimensional Classification,” PhD dissertation, School of Computer Science, Carnegie Mellon Univ., 2004.
- [37] R.J. Freund and P.D. Minton,
Regression Methods: A Tool for Data Analysis. Marcel Dekker, 1979.- [40] M.S. Bazaraa and C.M. Shetty,
Nonlinear Programming: Theory and Algorithms. John Wiley & Sons, 1979.- [41] P.E. Gill, W. Murray, and M.H. Wright,
Practical Optimization. Academic Press, 1989.- [42] M. Cristianini and J. Shawe-Taylor,
An Introduction to Support Vector Machines. Cambridge Univ. Press, 2000.- [44] N.D. Lawrence, M. Seeger, and R. Herbrich, “Fast Sparse Gaussian Process Methods: The Informative Vector Machine,”
Advances in Neural Information Processing Systems, vol. 15, pp. 609-616, 2003.- [45] www.svmlight.joachims.org, 2007.
- [46] L. Breiman, “Bias, Variance and Arcing Classifiers,” Technical Report 460, Dept. Statistics, Univ. of California, 1996.
- [47] M. Pohar, M. Blas, and S. Turk, “Comparison of Logistic Regression and Linear Discriminant Analysis: A Simulation Study,”
Metodoloski Zvezki, vol. 1, no. 1, pp. 143-161, 2004.- [48] S. Wang and H. Sun, “Measuring Overlap-Rate for Cluster Merging in a Hierarchical Approach to Color Image Segmentation,”
Int'l J. Fuzzy Systems, vol. 6, no. 3, pp. 147-156, 2004.- [49] http://ida.first.gmd.de~raetsch/, 1999.
- [50] A. Klautau, “Discriminative Gaussian Mixture Models: A Comparison with Kernel Classifiers,”
Proc. 20th Int'l Conf. Machine Learning, pp. 353-360, 2003.- [51] I. Daubechies,
Ten Lectures on Wavelets. SIAM, 1992.- [52] H. Shao, T. Svoboda, T. Tuytelaars, and L.V. Gool, “HPAT Indexing for Fast Object/Scene Recognition Based on Local Appearance,”
Proc. Int'l Conf. Image and Video Retrieval, pp. 71-80, 2003.- [53] R. Fergus, P. Perona, and A. Zisserman, “Object Class Recognition by Unsupervised Scale-invariant Learning,”
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 264-271, 2003. |