The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - December (2009 vol.31)
pp: 2143-2157
Brian Kulis , University of California at Berkeley, Berkeley
Prateek Jain , University of Texas at Austin, Austin
Kristen Grauman , University of Texas at Austin, Austin
ABSTRACT
We introduce a method that enables scalable similarity search for learned metrics. Given pairwise similarity and dissimilarity constraints between some examples, we learn a Mahalanobis distance function that captures the examples' underlying relationships well. To allow sublinear time similarity search under the learned metric, we show how to encode the learned metric parameterization into randomized locality-sensitive hash functions. We further formulate an indirect solution that enables metric learning and hashing for vector spaces whose high dimensionality makes it infeasible to learn an explicit transformation over the feature dimensions. We demonstrate the approach applied to a variety of image data sets, as well as a systems data set. The learned metrics improve accuracy relative to commonly used metric baselines, while our hashing construction enables efficient indexing with learned distances and very large databases.
INDEX TERMS
Metric learning, similarity search, locality-sensitive hashing, LogDet divergence, kernel learning, image search.
CITATION
Brian Kulis, Prateek Jain, Kristen Grauman, "Fast Similarity Search for Learned Metrics", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 12, pp. 2143-2157, December 2009, doi:10.1109/TPAMI.2009.151
REFERENCES
[1] D. Lowe, “Distinctive Image Features from Scale Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, 2004.
[2] N. Snavely, S. Seitz, and R. Szeliski, “Photo Tourism: Exploring Photo Collections in 3D,” Proc. ACM SIGGRAPH, pp. 835-846, 2006.
[3] A. Torralba, R. Fergus, and W.T. Freeman, “80 Million Tiny Images: A Large Database for Non-Parametric Object and Scene Recognition.” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 11, pp. 1958-1970, Nov. 2008.
[4] J. Sivic and A. Zisserman, “Video Data Mining Using Configurations of Viewpoint Invariant Regions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[5] G. Shakhnarovich, P. Viola, and T. Darrell, “Fast Pose Estimation with Parameter-Sensitive Hashing,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[6] V. Athitsos and S. Sclaroff, “Estimating 3D Hand Pose from a Cluttered Image,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[7] H. Zhang, A. Berg, M. Maire, and J. Malik, “SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[8] A. Frome, Y. Singer, F. Sha, and J. Malik, “Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[9] K. Grauman and T. Darrell, “The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features,” Proc. IEEE Int'l Conf. Computer Vision, 2005.
[10] E. Xing, A. Ng, M. Jordan, and S. Russell, “Distance Metric Learning, with Application to Clustering with Side Information,” Advances in Neural Information Processing Systems, 2002.
[11] A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall, “Learning a Mahalanobis Metric from Equivalence Constraints,” J. Machine Learning Research, vol. 6, pp. 937-965, June 2005.
[12] T. Hertz, A. Bar-Hillel, and D. Weinshall, “Learning Distance Functions for Image Retrieval,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[13] K. Weinberger, J. Blitzer, and L. Saul, “Distance Metric Learning for Large Margin Nearest Neighbor Classification,” Advances in Neural Information Processing Systems, 2006.
[14] A. Globerson and S. Roweis, “Metric Learning by Collapsing Classes,” Advances in Neural Information Processing Systems, 2005.
[15] J. Davis, B. Kulis, P. Jain, S. Sra, and I. Dhillon, “Information-Theoretic Metric Learning,” Proc. Int'l Conf. Machine Learning, 2007.
[16] J. Freidman, J. Bentley, and A. Finkel, “An Algorithm for Finding Best Matches in Logarithmic Expected Time,” ACM Trans. Math. Software, vol. 3, no. 3, pp. 209-226, Sept. 1977.
[17] J. Uhlmann, “Satisfying General Proximity/Similarity Queries with Metric Trees,” Information Processing Letters, vol. 40, pp. 175-179, 1991.
[18] P. Jain, B. Kulis, and K. Grauman, “Fast Image Search for Learned Metrics,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[19] G. Lanckriet, N. Cristinanini, P. Bartlett, L. Ghaoui, and M. Jordan, “Learning the Kernel Matrix with Semidefinite Programming,” J. Machine Learning Research, vol. 5, pp. 27-72, 2004.
[20] M. Varma and D. Ray, “Learning the Discriminative Power Invariance Trade Off,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[21] J. Goldberger, S.T. Roweis, G.E. Hinton, and R. Salakhutdinov, “Neighbourhood Components Analysis,” Advances in Neural Information Processing Systems, 2004.
[22] R. Hadsell, S. Chopra, and Y. LeCun, “Dimensionality Reduction by Learning an Invariant Mapping,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1735-1742, 2006.
[23] M. Schultz and T. Joachims, “Learning a Distance Metric from Relative Comparisons,” Advances in Neural Information Processing Systems, 2003.
[24] A. Frome, Y. Singer, and J. Malik, “Image Retrieval and Classification Using Local Distance Functions,” Advances in Neural Information Processing Systems 19, B. Scholkopf, J. Platt, and T. Hofmann, eds., MIT Press, 2007.
[25] K. Crammer, J. Keshet, and Y. Singer, “Kernel Design Using Boosting,” Advances in Neural Information Processing Systems, 2002.
[26] T. Hertz, A. Bar-Hillel, and D. Weinshall, “Learning a Kernel Function for Classification with Small Training Samples,” Proc. Int'l Conf. Machine Learning, 2006.
[27] P. Jain, T. Huynh, and K. Grauman, “Learning Discriminative Matching Functions for Local Image Features,” technical report, Univ. of Texas at Austin, Apr. 2007.
[28] A. Bosch, A. Zisserman, and X. Munoz, “Representing Shape with a Spatial Pyramid Kernel,” Proc. Int'l Conf. Image and Video Retrieval, 2007.
[29] V. Athitsos, J. Alon, S. Sclaroff, and G. Kollios, “BoostMap: A Method for Efficient Approximate Similarity Rankings,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[30] R. Duda, P. Hart, and D. Stork, Pattern Classification, second ed., chap. 10. John Wiley and Sons, Inc., 2001.
[31] S. Roweis and L. Saul, “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
[32] J. Tenenbaum, V. de Silva, and J. Langford, “A Global Geometric Framework for Nonlinear Dimensionality Reduction,” Science, vol. 290, no. 5500, pp. 2319-2323, Dec. 2000.
[33] M. Badoiu, E. Demaine, M. Hajiaghayi, and P. Indyk, “Low-Dimensional Embedding with Extra Information,” Proc. 20th Symp. Computational Geometry, 2004.
[34] A. Torralba, R. Fergus, and Y. Weiss, “Small Codes and Large Image Databases for Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[35] J. Beis and D. Lowe, “Shape Indexing Using Approximate Nearest-Neighbour Search in High Dimensional Spaces,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[36] D. Nister and H. Stewenius, “Scalable Recognition with a Vocabulary Tree,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[37] P. Indyk and R. Motwani, “Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality,” Proc. 30th Ann. Symp. Theory of Computing, 1998.
[38] M. Charikar, “Similarity Estimation Techniques from Rounding Algorithms,” Proc. ACM Ann. Symp. Theory of Computing, 2002.
[39] M. Datar, N. Immorlica, P. Indyk, and V. Mirrokni, “Locality-Sensitive Hashing Scheme Based on p-Stable Distributions,” Proc. Ann. Symp. Computational Geometry, 2004.
[40] B. Georgescu, I. Shimshoni, and P. Meer, “Mean Shift Based Clustering in High Dimensions: A Texture Classification Example,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[41] P. Indyk and N. Thaper, “Fast Image Retrieval via Embeddings,” Proc. Int'l Workshop Statistical and Computational Theories of Vision, 2003.
[42] K. Grauman and T. Darrell, “Pyramid Match Hashing: Sub-Linear Time Indexing over Partial Correspondences,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[43] M. Muja and D.G. Lowe, “Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration,” Proc. Int'l Conf. Computer Vision Theory and Applications, 2009.
[44] Nearest-Neighbor Methods in Learning and Vision: Theory and Practice, G. Shakhnarovich, T. Darrell, and P. Indyk, eds. The MIT Press, 2006.
[45] M. Goemans and D. Williamson, “Improved Approximation Algorithms for Maximum Cut and Satisfiability Problems Using Semidefinite Programming,” J. ACM, vol. 42, no. 6, pp. 1115-1145, 1995.
[46] H. Ling and S. Soatto, “Proximity Distribution Kernels for Geometric Context in Category Recognition,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[47] K. Grauman and T. Darrell, “The Pyramid Match Kernel: Efficient Learning with Sets of Features,” J. Machine Learning Research, vol. 8, pp. 725-760, Apr. 2007.
[48] K. Grauman and T. Darrell, “Approximate Correspondences in High Dimensions,” Advances in Neural Information Processing Systems, 2007.
[49] F. Odone, A. Barla, and A. Verri, “Building Kernels from Binary Strings for Image Matching,” IEEE Trans. Image Processing, vol. 14, no. 2, pp. 169-180, Feb. 2005.
[50] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[51] J. Ha, C. Rossbach, J. Davis, I. Roy, H. Ramadan, D. Porter, D. Chen, and E. Witchel, “Improved Error Reporting for Software That Uses Black-Box Components,” Proc. Conf. Programming Language Design and Implementation, 2007.
[52] L. Taycher, G. Shakhnarovich, D. Demirdjian, and T. Darrell, “Conditional Random People: Tracking Humans with CRFs and Grid Filters,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[53] K. Mikolajczyk and C. Schmid, “Scale and Affine Invariant Interest Point Detectors,” Int'l J. Computer Vision, vol. 60, no. 1, pp. 63-86, Oct. 2004.
[54] J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust Wide Baseline Stereo from Maximally Stable Extremal Regions,” Proc. British Machine Vision Conf., 2002.
[55] G. Hua, M. Brown, and S. Winder, “Discriminant Embedding for Local Image Descriptors,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
5 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool