The Community for Technology Leaders
RSS Icon
Issue No.02 - Feb. (2013 vol.25)
pp: 433-447
Konstantinos A. Raftopoulos , National Technical University of Athens, Athens
Klimis S. Ntalianis , National Technical University of Athens, Athens
Dionyssios D. Sourlas , Elais-Unilever Hellas S.A, Athens
Stefanos D. Kollias , National Technical University of Athens, Athens
We propose a novel method for automatic annotation, indexing and annotation-based retrieval of images. The new method, that we call Markovian Semantic Indexing (MSI), is presented in the context of an online image retrieval system. Assuming such a system, the users' queries are used to construct an Aggregate Markov Chain (AMC) through which the relevance between the keywords seen by the system is defined. The users' queries are also used to automatically annotate the images. A stochastic distance between images, based on their annotation and the keyword relevance captured in the AMC, is then introduced. Geometric interpretations of the proposed distance are provided and its relation to a clustering in the keyword space is investigated. By means of a new measure of Markovian state similarity, the mean first cross passage time (CPT), optimality properties of the proposed distance are proved. Images are modeled as points in a vector space and their similarity is measured with MSI. The new method is shown to possess certain theoretical advantages and also to achieve better Precision versus Recall results when compared to Latent Semantic Indexing (LSI) and probabilistic Latent Semantic Indexing (pLSI) methods in Annotation-Based Image Retrieval (ABIR) tasks.
Semantics, Markov processes, Image retrieval, Convergence, Indexing, Probabilistic logic, Eigenvalues and eigenfunctions, annotation-based image retrieval, Markovian semantic indexing, image annotation, query mining
Konstantinos A. Raftopoulos, Klimis S. Ntalianis, Dionyssios D. Sourlas, Stefanos D. Kollias, "Mining User Queries with Markov Chains: Application to Online Image Retrieval", IEEE Transactions on Knowledge & Data Engineering, vol.25, no. 2, pp. 433-447, Feb. 2013, doi:10.1109/TKDE.2011.219
[1] S. Santini and R. Jain, "Similarity Measures," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 9, pp. 871-883, Sept. 1999.
[2] K. Stevenson and C. Leung, "Comparative Evaluation of Web Image Search Engines for Multimedia Applications," Proc. IEEE Int'l Conf. Multimedia and Expo, July 2005.
[3] comScore's Report Article, "Comscore's Qsearch 2.0 Service," comScore's Report Article,, 2007.
[4] B.J. Jansen, A. Spink, and T. Saracevic, "Real Life, Real Users, and Real Needs: A Study and Analysis of User Queries on the Web," Information Processing and Management, vol. 36, no. 2, pp. 207-227, 2000.
[5] R. Datta, D. Joshi, J. Li, and J.Z. Wang, "Image Retrieval: Ideas, Influences, and Trends of the New Age," ACM Computing Surveys, vol. 40, no. 2, pp. 1-60, 2008.
[6] A. Bhattacharya, V. Ljosa, J.-Y. Pan, M.R. Verardo, H. Yang, C. Faloutsos, and A.K. Singh, "Vivo: Visual Vocabulary Construction for Mining Biomedical Images," Proc. IEEE Fifth Int'l Conf. Data Mining, Nov. 2005.
[7] J. Li and J. Wang, "Real-Time Computerized Annotation of Pictures," Proc. ACM 14th Ann. Int'l Conf. Multimedia, 2006.
[8] D. Joshi, J.Z. Wang, and J. Li, "The Story Picturing Engine - A System for Automatic Text Illustration," ACM Trans. Multimedia Computing, Comm. and Applications, vol. 2, no. 1, pp. 68-89, 2006.
[9] M.W. Berry, S.T. Dumais, and G.W. O'Brien, "Using Linear Algebra for Intelligent Information Retrieval," SIAM Rev., vol. 37, no. 4, pp. 573-595, 1995.
[10] T. Hofmann, "Probabilistic Latent Semantic Indexing," Proc. 22nd Int'l Conf. Research and Development in Information Retrieval (SIGIR '99), 1999.
[11] T. Hofmann, "Unsupervised Learning by Probabilistic Latent Semantic Analysis," Machine Learning, vol. 42, no. 1/2, pp. 177-196, 2001.
[12] D.M. Blei and A.Y. Ng, and M.I. Jordan, "Latent Dirichlet Allocation," J. Machine Learning Research, vol. 3, pp. 993-1022, 2003.
[13] T.L. Griffiths and M. Steyvers, "Finding Scientific Topics," Proc. Nat'l Academy of Sciences USA, vol. 101, no. suppl. 1, pp. 5228-5235, 2004.
[14] M. Steyvers, P. Smyth, M. Rosen-Zvi, and T. Griffiths, "Probabilistic Author-Topic Models for Information Discovery," Proc. 10th ACM SIGKDD Conf. Knowledge Discovery and Data Mining, 2004.
[15] Z. Guo, S. Zhu, Y. Chi, Z. Zhang, and Y. Gong, "A Latent Topic Model for Linked Documents," Proc. 32nd Int'l ACM SIGIR Conf. Research and Development in Information Retrieval (SIGIR), 2009.
[16] T.-T. Pham, N.E. Maillot, J.-H. Lim, and J.-P. Chevallet, "Latent Semantic Fusion Model for Image Retrieval and Annotation," Proc. 16th ACM Conf. Information and Knowledge Management (CIKM), 2007.
[17] R. Datta, D. Joshi, J. Li, and J.Z. Wang, "Image Retrieval: Ideas, Influences, and Trends of the New Age," ACM Computing Surveys, vol. 40, no. 2,article 5, pp. 1-60, 2008.
[18] F. Monay and D. Gatica-Perez, "On Image Auto-Annotation with Latent Space Models," Proc. ACM Int'l Conf. Multimedia (MM), 2003.
[19] K. Barnard and D. Forsyth, "Learning the Semantics of Words and Pictures," Proc. Int'l Conf. Computer Vision, vol. 2, pp. 408-415, 2001.
[20] L.-J. Li and G. Wang, and L. Fei-Fei, "OPTIMOL: Automatic Online Picture Collection via Incremental Model Learning," Int'l J. Computer Vision, vol. 88, no. 2, pp. 147-168, 2010.
[21] J. Fan and Y. Gao, and H. Luo, "Integrating Concept Ontology and Multitask Learning to Achieve More Effective Classier Training for Multilevel Image Annotation," IEEE Trans. Image Processing, vol. 17, no. 3, pp. 407-426, Mar. 2008.
[22] G. Shafer, P.P. Shenoy, and K. Mellouli, "Propagating belief Functions in Qualitative Markov Trees," Int'l J. Approximate Reasoning 1, vol. 4, pp. 394-400, 1987.
[23] L.G. Shapiro, "GroundTruth Database," http://www.cs. groundtruth/, Univ. of Washington, 2012.
[24] J. Pearl, Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988.
[25] L.D. Lowrance, T.D. Garvey, and T.M. Strat, "A Framework for Evidential Reasoning Systems," Proc. Fifth Nat'l Conf. Artificial Intelligence (AAAI '86), pp. 896-901, 1986.
[26] U. Montanari, "Networks of Constraints, Fundamental Properties and Applications to Picture Processing," Information Science, vol. 7, pp. 95-132, 1974.
[27] W. Woods, Representation and Understanding, D. Bobrow and A. Collins, eds. Academic Press, 1975.
[28] R.O. Duda, P.E. Hart, and N.J. Nilsson, "Subjective Bayesian Methods for Rule-Based Inference Systems," Proc. Nat'l Computer Conf. and Exposition (AFIPS), vol. 45, pp. 1075-1082, 1976.
[29] R. Schank, "Conceptual Dependency: A Theory of Natural Language Understanding," Cognitive Psychology, vol. 4, pp. 552-631, 1972.
[30] J.F. Sowa, Conceptual Structures: Information Processing in Mind and Machine. Addison-Wesley, 1984.
[31] O. Tuzel, F. Porikli, and P. Meer, "Pedestrian Detection via Classification on Riemannian Manifolds," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1713-1727, Oct. 2008.
[32] R. Howard, Dynamic Probabilistic Systems. John Wiley and Sons, Inc., 1971.
[33] G. Zhen, Z. Shenghuo, C. Yun, Z. Zhongfei, and G. Yihong, "A Latent Topic Model for Linked Documents," Proc. 32nd Int'l ACM SIGIR Conf. Research and Development in Information Retrieval (SIGIR '09), 2009.
[34] W.J. Stewart, Numerical Solution of Markov Chains. Princeton Univ. Press, 1994.
[35] codeindex.html, 2012.
12 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool