This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Local Distance Functions: A Taxonomy, New Algorithms, and an Evaluation
April 2011 (vol. 33 no. 4)
pp. 794-806
Deva Ramanan, University of California Irvine, Irvine
Simon Baker, Microsoft Research, Redmond
We present a taxonomy for local distance functions where most existing algorithms can be regarded as approximations of the geodesic distance defined by a metric tensor. We categorize existing algorithms by how, where, and when they estimate the metric tensor. We also extend the taxonomy along each axis. How: We introduce hybrid algorithms that use a combination of techniques to ameliorate overfitting. Where: We present an exact polynomial-time algorithm to integrate the metric tensor along the lines between the test and training points under the assumption that the metric tensor is piecewise constant. When: We propose an interpolation algorithm where the metric tensor is sampled at a number of references points during the offline phase. The reference points are then interpolated during the online classification phase. We also present a comprehensive evaluation on tasks in face recognition, object recognition, and digit recognition.

[1] http://en.wikipedia.org/wikiMetric_tensor , 2009.
[2] http://mathworld.wolfram.comMetricTensor.html , 2010.
[3] C. Atkeson, A. Moore, and S. Schaal, "Locally Weighted Learning," Artificial Intelligence Rev., vol. 11, pp. 11-73, 1997.
[4] B. Babenko, S. Branson, and S. Belongie, "Similarity Metrics for Categorization: From Monolithic to Category Specific," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[5] A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall, "Learning and Mahalanobis Metric from Equivalence Constraints," J. Machine Learning Research, vol. 6, pp. 937-965, 2005.
[6] O. Boiman, E. Shechtman, and M. Irani, "In Defense of Nearest-Neighbor Based Image Classification," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[7] D. Broomhead and D. Lowe, "Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks," 1988.
[8] T. Cormen, C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms. MIT Press, 2001.
[9] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines: And Other Kernel-Based Learning Methods. Cambridge Univ. Press, 2000.
[10] C. Domeniconi, J. Peng, and D. Gunopulos, "Locally Adaptive Metric Nearest-Neighbor Classification," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1281-1285, Sept. 2002.
[11] L. Fei-Fei, R. Fergus, and P. Perona, "Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, Workshop Generative-Model Based Vision, 2004.
[12] A. Frome, Y. Singer, and J. Malik, "Image Retrieval and Classification Using Local Distance Functions," Advances in Neural Information Processing Systems, vol. 19, pp. 417-424, MIT Press, 2007.
[13] A. Frome, Y. Singer, F. Sha, and J. Malik, "Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[14] K. Fukunaga, Introduction to Statistical Pattern Recognition. Academic Press, 1990.
[15] J. Goldberger, S. Roweis, and R. Salakhutdinov, "Neighborhood Components Analysis," Advances in Neural Information Processing Systems, vol. 17, pp. 513-520, MIT Press, 2005.
[16] M. Gonen and E. Alpaydin, "Localized Multiple Kernel Learning," Proc. Int'l Conf. Machine Learning, 2008.
[17] K. Grauman and T. Darrell, "Efficient Learning with Sets of Features," J. Machine Learning Research, vol. 8, pp. 725-760, 2007.
[18] R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, "Multi-PIE," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2008.
[19] T. Hastie and R. Tibshirani, "Discriminant Adaptive Nearest Neighbor Classification," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 6, pp. 607-616, June 1996.
[20] P. Indyk and R. Motwani, "Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality," Proc. 13th Ann. ACM Symp. Theory of Computing, pp. 604-613, 1998.
[21] A. Kapoor, K. Grauman, R. Urtasun, and T. Darrell, "Gaussian Processes for Object Categorization," Int'l J. Computer Vision, 2009.
[22] B. Kulis, P. Jain, and K. Grauman, "Fast Similarity Search for Learned Metrics," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 12, pp. 2143-2157, Dec. 2009.
[23] M. Kumar, P. Torr, and A. Zisserman, "An Invariant Large Margin Nearest Neighbour Classifier," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[24] K. Weinbeger and L. Saul, "Distance Metric Learning for Large Margin Nearest Neighbor Classification," J. Machine Learning Research, 2009.
[25] F. Labelle and J. Shewchuk, "Anisotropic Voronoi Diagrams and Guaranteed-Quality Anisotropic Mesh Generation," Proc. 19th Ann. Symp. Computational Geometry, pp. 191-200, 2003.
[26] S. Lazebnik, C. Schmid, and J. Ponce, "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2169-2178, 2006.
[27] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-Based Learning Applied to Document Recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
[28] S. Mahamud and M. Hebert, "The Optimal Distance Measure for Object Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 248-255, 2003.
[29] S. Maji and A. Berg, "Max-Margin Additive Classifiers for Detection," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[30] T. Malisiewicz and A. Efros, "Recognition by Association via Learning Per-Exemplar Distances," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[31] A. Ng and M. Jordan, "On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes," Advances in Neural Information Processing Systems, vol. 14, pp. 841-848, MIT Press, 2002.
[32] D. Ramanan and S. Baker, "Local Distance Functions: A Taxonomy, New Algorithms, and an Evaluation," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[33] R. Rifkin and A. Klautau, "In Defense of One-vs-All Classification," The J. Machine Learning Research, vol. 5, pp. 101-141, 2004.
[34] E. Rosch and C. Mervis, "Family Resemblances: Studies in the Internal Structure of Categories," Cognitive Psychology, vol. 7, pp. 573-605, 1975.
[35] G. Shakhnarovich, P. Viola, and T. Darrell, "Fast Pose Estimation with Parameter-Sensitive Hashing," Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 750-757, 2003.
[36] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression (PIE) Database," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2002.
[37] P. Simard, Y. LeCun, and J. Denker, "Efficient Pattern Recognition Using a New Transformation Distance," Advances in Neural Information Processing Systems, vol. 5, pp. 50-58, MIT Press, 1993.
[38] L. Torresani and K. Lee, "Large Margin Component Analysis," Advances in Neural Information Processing Systems, vol. 19, pp. 1385-1392, MIT Press, 2007.
[39] R. Urtasun and T. Darrell, "Local Probabilistic Regression for Activity-Independent Human Pose Inference," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[40] M. Varma and D. Ray, "Learning the Discriminative Power-Invariance Trade-Off," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[41] P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 511-518, 2001.
[42] X. Wang and X. Tang, "Random Sampling LDA for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 259-265, 2004.
[43] H. Zhang, A. Berg, M. Maire, and J. Malik, "SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2126-2136, 2006.

Index Terms:
Nearest neighbor classification, metric learning, metric tensor, local distance functions, taxonomy, database, evaluation.
Citation:
Deva Ramanan, Simon Baker, "Local Distance Functions: A Taxonomy, New Algorithms, and an Evaluation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 4, pp. 794-806, April 2011, doi:10.1109/TPAMI.2010.127
Usage of this product signifies your acceptance of the Terms of Use.