This Article 
 Bibliographic References 
 Add to: 
Minimum Distance between Pattern Transformation Manifolds: Algorithm and Applications
July 2009 (vol. 31 no. 7)
pp. 1225-1238
Effrosyni Kokiopoulou, Ecole Polytechnic Fédérale de Lausane, Lausanne
Pascal Frossard, Ecole Polytechnic Fédérale de Lausane, Lausanne
Transformation invariance is an important property in pattern recognition, where different observations of the same object typically receive the same label. This paper focuses on a transformation-invariant distance measure that represents the minimum distance between the transformation manifolds spanned by patterns of interest. Since these manifolds are typically nonlinear, the computation of the manifold distance (MD) becomes a nonconvex optimization problem. We propose representing a pattern of interest as a linear combination of a few geometric functions extracted from a structured and redundant basis. Transforming the pattern results in the transformation of its constituent parts. We show that, when the transformation is restricted to a synthesis of translations, rotations, and isotropic scalings, such a pattern representation results in a closed-form expression of the manifold equation with respect to the transformation parameters. The MD computation can then be formulated as a minimization problem whose objective function is expressed as the difference of convex functions (DC). This interesting property permits optimally solving the optimization problem with DC programming solvers that are globally convergent. We present experimental evidence which shows that our method is able to find the globally optimal solution, outperforming existing methods that yield suboptimal solutions.

[1] R.F.I Ventura, P. Vandergheynst, and P. Frossard, “Low Rate and Flexible Image Coding with Redundant Representations,” IEEE Trans. Image Processing, vol. 15, no. 3, pp. 726-739, Mar. 2006.
[2] E. Kokiopoulou and P. Frossard, “Image Alignment with Rotation Manifolds Built on Sparse Geometric Expansions,” Proc. IEEE Int'l Workshop Multimedia Signal Processing, Oct. 2007.
[3] N. Vasconcelos and A. Lippman, “A Multiresolution Manifold Distance for Invariant Image Similarity,” IEEE Trans. Multimedia, vol. 7, no. 1, pp. 127-142, Feb. 2005.
[4] P. Simard, Y.L. Cun, J. Denker, and B. Victorri, “Transformation Invariance in Pattern Recognition—Tangent Distance and Tangent Propagation,” Neural Networks: Tricks of the Trade, pp. 239-274, 1998.
[5] D. Keysers, J. Dahmen, T. Theiner, and H. Ney, “Experiments with an Extended Tangent Distance,” Proc. 15th IEEE Int'l Conf. Pattern Recognition, vol. 2, pp. 38-42, 2000.
[6] A. Fitzgibbon and A. Zisserman, “Joint Manifold Distance: A New Approach to Appearance Based Clustering,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2003.
[7] D. DeCoste and M. Burl, “Distortion-Invariant Recognition via Jittered Queries,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2000.
[8] D. Decoste and B. Schölkopf, “Training Invariant Support Vector Machines,” Machine Learning, vol. 46, no. 1, pp. 161-190, 2002.
[9] B. Haasdonk and D. Keysers, “Tangent Distance Kernels for Support Vector Machines,” Proc. 16th IEEE Int'l Conf. Pattern Recognition, vol. 2, pp. 864-868, 2002.
[10] B. Schölkopf, C. Burges, and V. Vapnik, “Incorporating Invariances in Support Vector Learning Machines,” Artificial Neural Networks—ICANN, vol. 96, pp. 47-52, 1996.
[11] B. Schölkopf, P. Simard, A. Smola, and V. Vapnik, “Prior Knowledge in Support Vector Kernels,” Proc. Advances in Neural Information Processing Systems, vol. 10, pp. 640-646, 1998.
[12] T. Graepel and R. Herbrich, “Invariant Pattern Recognition by Semidefinite Programming Machines,” Proc. Advances in Neural Information Processing Systems, vol. 16, pp. 33-40, 2004.
[13] P. Thévenaz, U.E. Ruttimann, and M. Unser, “A Pyramidal Approach to Subpixel Registration Based on Intensity,” IEEE Trans. Image Processing, vol. 7, no. 1, pp. 27-41, Jan. 1998.
[14] A. Neri and G. Jacovitti, “Maximum Likelihood Localization of 2-D Patterns in the Gauss-Laguerre Transform Domain: Theoretic Framework and Preliminary Results,” IEEE Trans. Image Processing, vol. 13, no. 1, pp. 72-85, Jan. 2004.
[15] M. Carli, F. Coppola, G. Jacovitti, and A. Neri, “Translation, Orientation and Scale Estimation Based on Laguerre-Gauss Circular Harmonic Pyramids,” Proc. SPIE, Image Processing: Algorithms and Systems, vol. 4667, pp. 55-65, 2002.
[16] B. Zitová and J. Flusser, “Image Registration Methods: A Survey,” Image and Vision Computing, vol. 21, no. 11, pp. 977-1000, 2003.
[17] S. Mallat, A Wavelet Tour of Signal Processing, second ed. Academic Press, 1998.
[18] A.J. Smola and B. Schölkopf, “Sparse Greedy Matrix Approximation for Machine Learning,” Proc. Int'l Conf. Machine Learning, pp.911-918, 2000.
[19] B. Schölkopf and A.J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Dec. 2001.
[20] P. Jost, P. Vandergheynst, and P. Frossard, “Tree-Based Pursuit: Algorithm and Properties,” IEEE Trans. Signal Processing, vol. 54, no. 12, pp. 4685-4697, 2006.
[21] J. Nocedal and S.J. Wright, Numerical Optimization, P. Glynn and S.M. Robinson, eds. Springer, 1999.
[22] R. Horst and N.V. Thoai, “DC Programming: Overview,” J.Optimization Theory and Applications, vol. 102, no. 1, pp. 1-43, Oct. 1999.
[23] R. Horst, P.M. Pardalos, and N.V. Thoai, Introduction to Global Optimization, second ed. Kluwer Academic Publishers, 2000.
[24] R. Horst and P.M. Pardalos, Handbook of Global Optimization, Kluwer Academic Publishers, 1995.
[25] S. Ellis and M. Nayakkankuppam, “Phylogenetic Analysis via DC Programming (Preprint),” Dept. of Math. and Statistics, Univ. of Maryland, Baltimore County, 2003.
[26] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
[27] L.T.H. An and P.D. Tao, “The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems,” Annals of Operations Research, vol. 133, pp. 23-46, 2005.
[28] F. Samaria and A. Harter, “Parameterisation of a Stochastic Model for Human Face Identification,” Proc. Second IEEE Workshop Applications of Computer Vision, Dec. 1994.

Index Terms:
Transformation invariance, pattern manifolds, sparse approximations.
Effrosyni Kokiopoulou, Pascal Frossard, "Minimum Distance between Pattern Transformation Manifolds: Algorithm and Applications," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 7, pp. 1225-1238, July 2009, doi:10.1109/TPAMI.2008.156
Usage of this product signifies your acceptance of the Terms of Use.