The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan. (2013 vol.35)
pp: 28-38
Xingwei Yang , Image Analytics Lab., GE Global Res., Niskayuna, NY, USA
L. Prasad , Space & Remote Sensing Sci. Group, Los Alamos Nat. Lab., Los Alamos, NM, USA
L. J. Latecki , Dept. of Comput. & Inf. Sci., Temple Univ., Philadelphia, PA, USA
ABSTRACT
In many applications, we are given a finite set of data points sampled from a data manifold and represented as a graph with edge weights determined by pairwise similarities of the samples. Often the pairwise similarities (which are also called affinities) are unreliable due to noise or due to intrinsic difficulties in estimating similarity values of the samples. As observed in several recent approaches, more reliable similarities can be obtained if the original similarities are diffused in the context of other data points, where the context of each point is a set of points most similar to it. Compared to the existing methods, our approach differs in two main aspects. First, instead of diffusing the similarity information on the original graph, we propose to utilize the tensor product graph (TPG) obtained by the tensor product of the original graph with itself. Since TPG takes into account higher order information, it is not a surprise that we obtain more reliable similarities. However, it comes at the price of higher order computational complexity and storage requirement. The key contribution of the proposed approach is that the information propagation on TPG can be computed with the same computational complexity and the same amount of storage as the propagation on the original graph. We prove that a graph diffusion process on TPG is equivalent to a novel iterative algorithm on the original graph, which is guaranteed to converge. After its convergence we obtain new edge weights that can be interpreted as new, learned affinities. We stress that the affinities are learned in an unsupervised setting. We illustrate the benefits of the proposed approach for data manifolds composed of shapes, images, and image patches on two very different tasks of image retrieval and image segmentation. With learned affinities, we achieve the bull's eye retrieval score of 99.99 percent on the MPEG-7 shape dataset, which is much higher than the state-of-the-art algorithms. When the data points are image patches, the NCut with the learned affinities not only significantly outperforms the NCut with the original affinities, but it also outperforms state-of-the-art image segmentation methods.
INDEX TERMS
Shape, Manifolds, Diffusion processes, Tensile stress, Noise, Image segmentation, Context,image segmentation, Diffusion process, tensor product graph, affinity learning, image retrieval
CITATION
Xingwei Yang, L. Prasad, L. J. Latecki, "Affinity Learning with Diffusion on Tensor Product Graph", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 1, pp. 28-38, Jan. 2013, doi:10.1109/TPAMI.2012.60
REFERENCES
[1] B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem," Neural Computation, vol. 10, no. 5, pp. 1299-1319, 1998.
[2] D. Zhou, J. Weston, A. Gretton, Q. Bousquet, and B. Scholkopf, "Ranking on Data Manifolds," Proc. Advances in Neural information Processing Systems, 2003.
[3] X. Yang, S. Köknar-Tezel, and L.J. Latecki, "Locally Constrained Diffusion Process on Locally Densified Distance Spaces with Applications to Shape Retrieval," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[4] P. Kontschieder, M. Donoser, and H. Bischof, "Beyond Pairwise Shape Similarity Analysis," Proc. Ninth Asian Conf. Computer Vision , 2009.
[5] X. Bai, X. Yang, L.J. Latecki, W. Liu, and Z. Tu, "Learning Context Sensitive Shape Similarity by Graph Transduction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 861-874, May 2010.
[6] M. Szummer and T. Jaakkola, "Partially Labeled Classification with Markov Random Walks," Proc. Advances in Neural Information Processing Systems, 2001.
[7] D. Zhou, O. Bousquet, T.N. Lal, J. Weston, and B. Scholkopf, "Learning with Local and Global Consistency," Proc. Advances in Neural Information Processing Systems, 2003.
[8] R. Coifman and S. Lafon, "Diffusion Maps," Applied and Computational Harmonic Analysis, vol. 21, pp. 5-30, 2006.
[9] H. Ling, X. Yang, and L.J. Latecki, "Balancing Deformability and Discriminability for Shape Matching," Proc. 11th European Conf. Computer Vision, 2010.
[10] Y. Huang, Q. Liu, and D. Metaxas, "Video Object Segmentation by Hypergraph Cut," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[11] D. Zhou, J. Huang, and B. Schlkopf, "Learning with Hypergraphs: Clustering, Classification, and Embedding," Proc. Advances in Neural Information Processing Systems, 2007.
[12] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[13] D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach toward Feature Space Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[14] A. Temlyakov, B.C. Munsell, J.W. Waggoner1, and S. Wang, "Two Perceptually Motivated Strategies for Shape Classification," Proc. IEEE Computer Vision and Pattern Recognition, 2010.
[15] X. Bai, B. Wang, X. Wang, W. Liu, and Z. Tu, "Co-Transduction for Shape Retrieval," Proc. European Conf. Computer Vision, 2010.
[16] D. Nister and H. Stewenius, "Scalable Recognition with a Vocabulary Tree," Proc. IEEE Computer Vision and Pattern Recognition, 2006.
[17] J. Sivi and A. Zisserman, "Video Google: A Text Retrieval Approach Toobject Matching in Videos," Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[18] H. Jegou, C. Schmid, H. Harzallah, and J. Verbeek, "Accurate Image Search Using the Contextual Dissimilarity Measure," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 2-11, Jan. 2010.
[19] H. Jegou, M. Douze, and C. Schmid, "Improving Bag-of-Features for Large Scale Image Search," Int'l J. Computer Vision, vol. 87, pp. 191-212, 2010.
[20] X. Yang and L.J. Latecki, "Affinity Learning on a Tensor Product Graph with Applications to Shape and Image Retrieval," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2369-2376, 2011.
[21] T.H. Kim, K.M. Lee, and S.U. Lee, "Learning Full Pairwise Affinities for Spectral Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[22] L. Ladicky, C. Russell, and P. Kohli, "Associative Hierarchical CRFs for Object Class Image Segmentation," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[23] N. Plath, M. Toussaint, and S. Nakajima, "Multi-Class Image Segmentation Using Conditional Random Fields and Global Classification," Proc. 26th Ann. Int'l Conf. Machine Learning, 2009.
[24] J.J. Lim, P. Arbelaez, C. Gu, and J. Malik, "Context by Region Ancestry," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[25] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "From Contours to Regions: An Empirical Evaluation," Proc. IEEE Computer Vision and Pattern Recognition, 2009.
[26] L. Prasad and S. Swaminarayan, "Hierarchical Image Segmentation by Polygon Grouping," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2008.
[27] X. Ren and J. Malik, "Learning a Classification Model for Segmentation," Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[28] S. Lafon and A.B. Lee, "Diffusion Maps and Coarse-Graining: A Unified Framework for Dimensionality Reduction Graph Partitioning, and Data Set Parameterization," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1393-1403, Sept. 2006.
[29] P. Lancaster and L. Rodman, Algebraic Riccati Equations. Clarendon Press, 1995.
[30] C. van Loan, "The Ubiquitous Kronecker Product," J. Computational and Applied Math., vol. 123, pp. 85-100, 2000.
[31] D. Qin, S. Gammeter, L. Bossard, and T.Q.L. van Gool, "Hello Neighbor: Accurate Object Retrieval with K-Reciprocal Nearest Neighbors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[32] J. Wang, S.-F. Chang, X. Zhou, and T.C.S. Wong, "Active Microscopic Cellular Image Annotation by Superposable Graph Transduction with Imbalanced Labels," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[33] L.J. Latecki and R. Lakämper, "Shape Similarity Measure Based on Correspondence of Visual Parts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1185-1190, Oct. 2000.
[34] R. Gopalan, P. Turaga, and R. Chellappa, "Articulation-Invariant Representation of Non-Planar Shapes," Proc. European Conf. Computer Vision, 2010.
[35] H. Ling and D. Jacobs, "Shape Classification Using the Inner-Distance," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 286-299, Feb. 2007.
[36] H. Stewenius and D. Nister, "Object Recognition Benchmark," http://vis.uky.edu/steweukbench/, 2012.
[37] K. Mikolajczyk and C. Schmid, "Scale and Affine Invariant Interest Point Detectors," Int'l J. Computer Vision, vol. 60, no. 1, pp. 63-86, 2004.
[38] D. Lowe, "Distinctive Image Features from Scale-Invariant Key Points," Int'l J. Computer Vision, vol. 60, pp. 91-110, 2004.
[39] L. Fei-Fei, R. Fergus, and P. Perona, "One-Shot Learning of Object Categories," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 594-611, Apr. 2006.
[40] S. Lazebnik, C. Schimid, and J. Ponce, "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[41] M.D.H. Jegou and C. Schmid, "Hamming Embedding and Weak Geometry Consistency for Large Scale Image Search," Proc. European Conf. Computer Vision, 2008.
[42] H. Jegou, M. Douze, and C. Schmid, "Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search," Proc. European Conf. Computer Vision, 2008.
[43] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics," Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[44] J. Shotton, J. Winn, C. Rother, and A. Criminisi, "Texton-Boost: Joint Appearance Shape and Context Modeling for Multi-Class Object Recognition and Segmentation," Proc. European Conf. Computer Vision, 2006.
[45] W.M. Rand, "Objective Criteria for the Evaluation of Clustering Methods," J. Am. Statistical Assoc., vol. 66, pp. 846-850, 1971.
[46] M. Meila, "Comparing Clusterings: An Axiomatic View," Proc. 22nd Int'l Conf. Machine Learning, 2005.
[47] P. Felzenszwalb and D. Huttenlocher, "Efficient Graph-Based Segmentation Algorithm," Int'l J. Computer Vision, vol. 59, no. 2, pp. 167-181, Sept. 2004.
[48] J. Wang, Y. Jia, X.-S. Hua, C. Zhang, and L. Quan, "Normalized Tree Partitioning for Image Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[49] M. Donoser, M. Urschler, M. Hirzer, and H. Bischof, "Saliency Driven Total Variation Segmentation," Proc. 11th IEEE Int'l Conf. Computer Vision, 2009.
[50] S.R. Rao, H. Mobahi, A.Y. Yang, S.S. Sastry, and Y. Ma, "Natural Image Segmentation with Adaptive Texture and Boundary Encoding," Proc. Ninth Asian Conf. Computer Vision, 2009.
[51] T. Malisiewicz and A.A. Efros, "Improving Spatial Support for Objects via Multiple Segmentations," Proc. British Machine Vision Conf., 2007.
71 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool