CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2011 vol.33 Issue No.04 - April

Subscribe

Issue No.04 - April (2011 vol.33)

pp: 794-806

Deva Ramanan , University of California Irvine, Irvine

Simon Baker , Microsoft Research, Redmond

ABSTRACT

We present a taxonomy for local distance functions where most existing algorithms can be regarded as approximations of the geodesic distance defined by a metric tensor. We categorize existing algorithms by how, where, and when they estimate the metric tensor. We also extend the taxonomy along each axis. How: We introduce hybrid algorithms that use a combination of techniques to ameliorate overfitting. Where: We present an exact polynomial-time algorithm to integrate the metric tensor along the lines between the test and training points under the assumption that the metric tensor is piecewise constant. When: We propose an interpolation algorithm where the metric tensor is sampled at a number of references points during the offline phase. The reference points are then interpolated during the online classification phase. We also present a comprehensive evaluation on tasks in face recognition, object recognition, and digit recognition.

INDEX TERMS

Nearest neighbor classification, metric learning, metric tensor, local distance functions, taxonomy, database, evaluation.

CITATION

Deva Ramanan, Simon Baker, "Local Distance Functions: A Taxonomy, New Algorithms, and an Evaluation",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.33, no. 4, pp. 794-806, April 2011, doi:10.1109/TPAMI.2010.127REFERENCES

- [1] http://en.wikipedia.org/wikiMetric_tensor , 2009.
- [2] http://mathworld.wolfram.comMetricTensor.html , 2010.
- [3] C. Atkeson, A. Moore, and S. Schaal, "Locally Weighted Learning,"
Artificial Intelligence Rev., vol. 11, pp. 11-73, 1997.- [4] B. Babenko, S. Branson, and S. Belongie, "Similarity Metrics for Categorization: From Monolithic to Category Specific,"
Proc. IEEE Int'l Conf. Computer Vision, 2009.- [5] A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall, "Learning and Mahalanobis Metric from Equivalence Constraints,"
J. Machine Learning Research, vol. 6, pp. 937-965, 2005.- [6] O. Boiman, E. Shechtman, and M. Irani, "In Defense of Nearest-Neighbor Based Image Classification,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.- [7] D. Broomhead and D. Lowe, "Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks," 1988.
- [8] T. Cormen, C. Leiserson, R. Rivest, and C. Stein,
Introduction to Algorithms. MIT Press, 2001.- [9] N. Cristianini and J. Shawe-Taylor,
An Introduction to Support Vector Machines: And Other Kernel-Based Learning Methods. Cambridge Univ. Press, 2000.- [10] C. Domeniconi, J. Peng, and D. Gunopulos, "Locally Adaptive Metric Nearest-Neighbor Classification,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1281-1285, Sept. 2002.- [11] L. Fei-Fei, R. Fergus, and P. Perona, "Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, Workshop Generative-Model Based Vision, 2004.- [12] A. Frome, Y. Singer, and J. Malik, "Image Retrieval and Classification Using Local Distance Functions,"
Advances in Neural Information Processing Systems, vol. 19, pp. 417-424, MIT Press, 2007.- [13] A. Frome, Y. Singer, F. Sha, and J. Malik, "Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification,"
Proc. IEEE Int'l Conf. Computer Vision, 2007.- [14] K. Fukunaga,
Introduction to Statistical Pattern Recognition. Academic Press, 1990.- [15] J. Goldberger, S. Roweis, and R. Salakhutdinov, "Neighborhood Components Analysis,"
Advances in Neural Information Processing Systems, vol. 17, pp. 513-520, MIT Press, 2005.- [16] M. Gonen and E. Alpaydin, "Localized Multiple Kernel Learning,"
Proc. Int'l Conf. Machine Learning, 2008.- [17] K. Grauman and T. Darrell, "Efficient Learning with Sets of Features,"
J. Machine Learning Research, vol. 8, pp. 725-760, 2007.- [18] R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, "Multi-PIE,"
Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2008.- [19] T. Hastie and R. Tibshirani, "Discriminant Adaptive Nearest Neighbor Classification,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 6, pp. 607-616, June 1996.- [20] P. Indyk and R. Motwani, "Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality,"
Proc. 13th Ann. ACM Symp. Theory of Computing, pp. 604-613, 1998.- [21] A. Kapoor, K. Grauman, R. Urtasun, and T. Darrell, "Gaussian Processes for Object Categorization,"
Int'l J. Computer Vision, 2009.- [22] B. Kulis, P. Jain, and K. Grauman, "Fast Similarity Search for Learned Metrics,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 12, pp. 2143-2157, Dec. 2009.- [23] M. Kumar, P. Torr, and A. Zisserman, "An Invariant Large Margin Nearest Neighbour Classifier,"
Proc. IEEE Int'l Conf. Computer Vision, 2007.- [24] K. Weinbeger and L. Saul, "Distance Metric Learning for Large Margin Nearest Neighbor Classification,"
J. Machine Learning Research, 2009.- [25] F. Labelle and J. Shewchuk, "Anisotropic Voronoi Diagrams and Guaranteed-Quality Anisotropic Mesh Generation,"
Proc. 19th Ann. Symp. Computational Geometry, pp. 191-200, 2003.- [26] S. Lazebnik, C. Schmid, and J. Ponce, "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2169-2178, 2006.- [27] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-Based Learning Applied to Document Recognition,"
Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.- [28] S. Mahamud and M. Hebert, "The Optimal Distance Measure for Object Detection,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 248-255, 2003.- [29] S. Maji and A. Berg, "Max-Margin Additive Classifiers for Detection,"
Proc. IEEE Int'l Conf. Computer Vision, 2009.- [30] T. Malisiewicz and A. Efros, "Recognition by Association via Learning Per-Exemplar Distances,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.- [31] A. Ng and M. Jordan, "On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes,"
Advances in Neural Information Processing Systems, vol. 14, pp. 841-848, MIT Press, 2002.- [32] D. Ramanan and S. Baker, "Local Distance Functions: A Taxonomy, New Algorithms, and an Evaluation,"
Proc. IEEE Int'l Conf. Computer Vision, 2009.- [33] R. Rifkin and A. Klautau, "In Defense of One-vs-All Classification,"
The J. Machine Learning Research, vol. 5, pp. 101-141, 2004.- [34] E. Rosch and C. Mervis, "Family Resemblances: Studies in the Internal Structure of Categories,"
Cognitive Psychology, vol. 7, pp. 573-605, 1975.- [35] G. Shakhnarovich, P. Viola, and T. Darrell, "Fast Pose Estimation with Parameter-Sensitive Hashing,"
Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 750-757, 2003.- [36] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression (PIE) Database,"
Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2002.- [37] P. Simard, Y. LeCun, and J. Denker, "Efficient Pattern Recognition Using a New Transformation Distance,"
Advances in Neural Information Processing Systems, vol. 5, pp. 50-58, MIT Press, 1993.- [38] L. Torresani and K. Lee, "Large Margin Component Analysis,"
Advances in Neural Information Processing Systems, vol. 19, pp. 1385-1392, MIT Press, 2007.- [39] R. Urtasun and T. Darrell, "Local Probabilistic Regression for Activity-Independent Human Pose Inference,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.- [40] M. Varma and D. Ray, "Learning the Discriminative Power-Invariance Trade-Off,"
Proc. IEEE Int'l Conf. Computer Vision, 2007.- [41] P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 511-518, 2001.- [42] X. Wang and X. Tang, "Random Sampling LDA for Face Recognition,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 259-265, 2004.- [43] H. Zhang, A. Berg, M. Maire, and J. Malik, "SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2126-2136, 2006. |