This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Data Fusion and Multicue Data Matching by Diffusion Maps
November 2006 (vol. 28 no. 11)
pp. 1784-1797
Data fusion and multicue data matching are fundamental tasks of high-dimensional data analysis. In this paper, we apply the recently introduced diffusion framework to address these tasks. Our contribution is three-fold: First, we present the Laplace-Beltrami approach for computing density invariant embeddings which are essential for integrating different sources of data. Second, we describe a refinement of the Nyström extension algorithm called "geometric harmonics.” We also explain how to use this tool for data assimilation. Finally, we introduce a multicue data matching scheme based on nonlinear spectral graphs alignment. The effectiveness of the presented schemes is validated by applying it to the problems of lipreading and image sequence alignment.

[1] S. Roweis and L. Saul, “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” Science, vol. 290, pp. 2323-2326, 2000.
[2] M. Belkin and P. Niyogi, “Laplacian Eigenmaps for Dimensionality Reduction and Data Representation,” Neural Computation, vol. 6, no. 15, pp. 1373-1396, June 2003.
[3] D. Donoho and C. Grimes, “Hessian Eigenmaps: New Locally Linear Embedding Techniques for High-Dimensional Data,” Proc. Nat'l Academy of Sciences, vol. 100, no. 10, pp. 5591-5596, May 2003.
[4] Z. Zhang and H. Zha, “Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignement,” Technical Report CSE-02-019, Dept. of Computer Science and Eng., Pennsylvania State Univ., 2002.
[5] R.R. Coifman and S. Lafon, “Diffusion Maps,” Applied and Computational Harmonic Analysis, special issue on diffusion maps and wavelets, vol. 21, pp. 5-30, July 2006.
[6] M. Belkin, P. Niyogi, and V. Sindhwani, “Manifold Regularization: A Geometric Framework for Learning from Examples,” Technical Report TR-2004-06, Univ. of Chicago, 2004.
[7] C. Fowlkes, S. Belongie, F. Chung, and J. Malik, “Spectral Grouping Using the Nyström Method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 214-225, Feb. 2004.
[8] Y. Bengio, J.-F. Paiement, and P. Vincent, “Out-of-Sample Extensions for Lle, Isomap, Mds, Eigenmaps, and Spectral Clustering,” Technical Report 1238, Univ. de Montréal, 2003.
[9] R. Coifman and S. Lafon, “Geometric Harmonics: A Novel Tool for Multiscale Out-of-Sample Extension of Empirical Functions,” Applied and Computational Harmonic Analysis, special issue on diffusion maps and wavelets, vol. 21, pp. 31-52, July 2006.
[10] R.R. Coifman, S. Lafon, A. Lee, M. Maggioni, B. Nadler, F. Warner, and S.W. Zucker, “Geometric Diffusions as a Tool for Harmonic Analysis and Structure Definition of Data: Diffusion Maps,” Proc. Nat'l Academy of Sciences, vol. 102, no. 21, pp. 7426-7431, May 2005.
[11] R. Coifman, S. Lafon, A. Lee, M. Maggioni, B. Nadler, F. Warner, and S. Zucker, “Geometric Diffusions as a Tool for Harmonics Analysis and Structure Definition of Data: Multiscale Methods,” Proc. Nat'l Academy of Sciences, vol. 102, no. 21, pp. 7432-7437, May 2005.
[12] B. Nadler, S. Lafon, R.R. Coifman, and I.G. Kevrekidis, “Diffusion Maps, Spectral Clustering and Reaction Coordinate of Dynamical Systems,” Applied and Computational Harmonic Analysis, special issue on diffusion maps and wavelets, vol. 21, pp. 113-127, July 2006.
[13] S. Lafon and A.B. Lee, “Diffusion Maps and Coarse-Graining: A Unified Framework for Dimensionality Reduction, Graph Partitioning, and Data Set Parameterization,” IEEE Trans. Pattern Analysis and Machine Intelligence, pp. 1393-1403, vol. 28, no. 9, Sept. 2006.
[14] R.I. Kondor and J.D. Lafferty, “Diffusion Kernels on Graphs and Other Discrete Input Spaces,” Proc. 19th Int'l Conf. Machine Learning (ICML '02), pp. 315-322, 2002.
[15] F. Chung, Spectral Graph Theory. CBMS-AMS, 1997.
[16] J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[17] Y. Weiss, “Segmentation Using Eigenvectors: A Unifying View,” Proc. Int'l Conf. Computer Vision, pp. 975-982, 1999.
[18] M. Meila and J. Shi, “A Random Walk's View of Spectral Segmentation,” Proc. Int'l Workshop Aartifical Intelligence and Statistics, 2001.
[19] S.X. Yu and J. Shi, “Multiclass Spectral Clustering,” Proc. IEEE Int'l Conf. Computer Vision, pp. 313-319, 2003.
[20] M. Belkin and P. Niyogi, “Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering,” Advances in Neural Information Processing, pp. 585-591, 2001.
[21] P. Diaconis and D. Stroock, “Geometric Bounds for Eigenvalues of Markov Chains,” The Annals of Applied Probability, vol. 1, no. 1, pp.36-61, 1991.
[22] M. Belkin and P. Niyogi, “Towards a Theoretical Foundation for Laplacian-Based Manifold Methods,” Proc. Conf. Learning Theory (COLT), pp. 486-500, 2005.
[23] W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in C. Cambridge Univ., 1988.
[24] M. Gori, M. Maggini, and L. Sarti, “Exact and Approximate Graph Matching Using Random Walks,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 7, pp. 1100-1111, July 2005.
[25] J. Ham, D. Lee, and L. Saul, “Semisupervised Alignment of Manifolds,” Proc. 10th Int'l Workshop Artificial Intelligence and Statistics, pp. 120-127, Jan. 2005.
[26] X. Bai, H. Yu, and E.R. Hancock, “Graph Matching Using Spectral Embedding and Alignment,” Proc. Int'l Conf. Pattern Recognition, pp. 398-401, 2004.
[27] Y. Keselman, A. Shokoufandeh, M.F. Demirci, and S.J. Dickinson, “Many-to-Many Graph Matching via Metric Embedding,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 850-857, 2003.
[28] A.W. Fitzgibbon, “Robust Registration of 2D and 3D Point Sets,” Proc. British Machine Vision Conf., pp. 662-670, 2001.
[29] H.J. Wolfson and I. Rigoutsos, “Geometric Hashing: An Overview,” IEEE Computing in Science and Eng., vol. 4, no. 4, pp. 10-21, 1997.
[30] M. Aharon and R. Kimmel, “Representation Analysis and Synthesis of Lip Images Using Dimensionality Reduction,” Int'l J. Computer Vision, to appear.
[31] B. Christoph, C. Michele, and S. Malcolm, “Video Rewrite: Driving Visual Speech with Audio,” Proc. 24th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 353-360, 1997.
[32] E. Cosatto and H.P. Graf, “Photo-Realistic Talking-Heads from Image Samples,” IEEE Trans. Multimedia, vol. 2, no. 3, pp. 152-163, 2000.
[33] C. Bregler, S. Manke, and H. Hild, “Improving Connected Letter Recognition by Lipreading,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 1993.
[34] N. Dettmer and M. Shah, “Visually Recognizing Speech Using Eigensequences,” Computational Imaging and Vision, pp. 345-371, 1997.
[35] I. Matthews, T. Cootes, A. Bangham, S. Cox, and R. Harvey, “Extraction of Visual Features for Lipreading,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 2, pp. 198-213, Feb. 2002.
[36] A.V. Nefian, L.H. Liang, X.X. Liu, X. Pi, and K. Murphy, “Dynamic Bayesian Networks for Audio-Visual Speech Recognition,” J.Applied Signal Processing, vol. 2002, no. 11, pp. 1274-1288, 2002.
[37] J. Luettin, N.A. Thacker, and S.W. Beet, “Active Shape Models for Visual Speech Feature Extraction,” Speechreading by Humans and Machines, vol. 150, pp. 383-390, 1996.
[38] J. Luettin and N.A. Thacker, “Speechreading Using Probabilistic Models,” Computer Vision and Image Understanding, vol. 65, no. 2, pp. 163-178, 1997.
[39] Y.-L. Tian, T. Kanade, and J. Cohn, “Robust Lip Tracking by Combining Shape, Color and Motion,” Proc. Fourth Asian Conf. Computer Vision (ACCV '00), Jan. 2000.
[40] I. Borg and P. Groenen, Modern Multidimensional Scaling—Theory and Applications. Springer-Verlag, 1997.
[41] C. Bregler, S. Omohundro, M. Covell, M. Slaney, S. Ahmad, D.A. Forsyth, and J.A. Feldman, “Probabilistic Models of Verbal and Body Gestures,” Computer Vision in Man-Machine Interfaces, Cambridge Univ. Press, 1998.
[42] S. Lafon, “Demo of the Mask Alignment,” 2005, http://www.math.yale.edu/sl349/demos_data demos.htm.

Index Terms:
Pattern matching, graph theory, graph algorithms, Markov processes, machine learning, data mining, image databases.
Citation:
St?phane Lafon, Yosi Keller, Ronald R. Coifman, "Data Fusion and Multicue Data Matching by Diffusion Maps," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1784-1797, Nov. 2006, doi:10.1109/TPAMI.2006.223
Usage of this product signifies your acceptance of the Terms of Use.