This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Adaptive Manifold Learning
February 2012 (vol. 34 no. 2)
pp. 253-265
Zhenyue Zhang, Dept. of Math., Zhejiang Univ., Hangzhou, China
Jing Wang, Sch. of Comput. Sci. & Technol., Huaqiao Univ., Xiamen, China
Hongyuan Zha, Coll. of Comput., Georgia Inst. of Technol., Atlanta, GA, USA
Manifold learning algorithms seek to find a low-dimensional parameterization of high-dimensional data. They heavily rely on the notion of what can be considered as local, how accurately the manifold can be approximated locally, and, last but not least, how the local structures can be patched together to produce the global parameterization. In this paper, we develop algorithms that address two key issues in manifold learning: 1) the adaptive selection of the local neighborhood sizes when imposing a connectivity structure on the given set of high-dimensional data points and 2) the adaptive bias reduction in the local low-dimensional embedding by accounting for the variations in the curvature of the manifold as well as its interplay with the sampling density of the data set. We demonstrate the effectiveness of our methods for improving the performance of manifold learning algorithms using both synthetic and real-world data sets.

[1] M.D. Abramoff, P.J. Magelhaes, and S.J. Ram, "Image Processing with ImageJ," Biophotonics Int'l, vol. 11, no. 7, pp. 36-42, 2004.
[2] M. Brand, "Charting a Manifold," Advances in Neural Information Processing Systems, S. Becker, S. Thrun, and K. Obermayer, eds., vol. 15, pp. 961-968, MIT Press, 2003.
[3] H. Bunke and K. Riesen, "Graph Classification Based on Dissimilarity Space Embedding," Proc. Joint IAPR Int'l Workshop Structural, Syntactic, and Statistical Pattern Recognition, pp. 996-1007, 2008.
[4] M. do Carmo, Differential Geometry of Curves and Surfaces. Prentice-Hall, 1976.
[5] C. Cortes and V. Vapnik, "Support-Vector Networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
[6] D. Donoho and C. Grimes, "Hessian Eigenmaps: New Locally Linear Embedding Techniques for High-Dimensional Data," Proc. Nat'l Academy of Sciences USA, vol. 100, pp. 5591-5596, 2003.
[7] A. Elgammal and C. Lee, "Inferring 3D Body Pose from Silhouettes Using Activity Manifold Learning," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2004.
[8] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, "From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[9] G.H. Golub and C.F. Van Loan, Matrix Computations, third ed. Johns Hopkins Univ. Press. 1996.
[10] G. Guo, Y. Fu, C. Dyer, and T. Huang, "Image-Based Human Age Estimation by Manifold Learning and Locally Adjusted Robust Regression," IEEE Trans. Image Processing, vol. 17, no. 7, pp. 1178-1188, July 2008.
[11] A.J. Izenman, Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning. Series: Springer Texts in Statistics, Springer, 2008.
[12] A. Jansen and P. Niyogi, "Intrinsic Fourier Analysis on the Manifold of Speech Sounds," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 2006.
[13] M.H. Law and A.K. Jain, "Incremental Nonlinear Dimensionality Reduction by Manifold Learning," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 377-391, Mar. 2006.
[14] K.C. Lee, J. Ho, and D. Kriegman, "Acquiring Linear Subspaces for Face Recognition under Variable Lighting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684-698, May 2005.
[15] S. Li, K. Chan, and C. Wang, "Performance Evaluation of the Nearest Feature Line Method in Image Classification and Retrieval," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1335-1339, Nov. 2000.
[16] L. Li and H. Li, "Dimension Reduction Methods for Microarrays with Application to Censored Survival Data," Bioinformatics. vol. 20, no. 18, pp. 3406-3412, 2004.
[17] T.B. Moeslunda, A. Hiltonb, and V. Krgerc, "A Survey of Advances in Vision-Based Human Motion Capture and Analysis," Computer Vision and Image Understanding, vol. 104, nos. 2/3, pp. 90-126, 2006.
[18] E. Murphy-Chutorian and M. Trivedi, "Head Pose Estimation in Computer Vision: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 4, pp. 607-626, Apr. 2009.
[19] S.A. Nene, S.K. Nayar, and H. Murase, "Columbia Object Image Library (COIL-20)," Technical Report CUCS-005-96, Columbia Univ., 1996.
[20] L. Rabiner and B. Juang, Fundamentals of Speech Recognition. Prentice-Hall, 1993.
[21] D. Reynolds and R. Rose, "Robust Text-Independent Speaker Identification Using Gaussian Mixture Speaker Model," IEEE Trans. Speech and Audio Processing, vol. 3, no. 1, pp. 72-83, Jan. 1995.
[22] S. Roweis and L. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, pp. 2323-2326, 2000.
[23] L. Saul and S. Roweis, "Think Globally, Fit Locally: Unsupervised Learning of Nonlinear Manifolds," J. Machine Learning Research, vol. 4, pp. 119-155, 2003.
[24] J. Tenenbaum, V. De Silva, and J. Langford, "A Global Geometric Framework for Nonlinear Dimension Reduction," Science, vol. 290, pp. 2319-2323, 2000.
[25] J. Venna, J. Peltonen, K. Nybo, H. Aidos, and S. Kaski, "Information Retrieval Perspective to Nonlinear Dimensionality Reduction for Data Visualization," J. Machine Learning Research, vol. 11, pp. 451-490, 2010.
[26] K.Q. Weinberger and L.K. Saul, "Unsupervised Learning of Image Manifolds by Semidefinite Programming," Int'l J. Computer Vision, vol. 70, no. 1, pp. 77-90, 2006.
[27] J. Zhang, S.Z. Li, and J. Wang, "Nearest Manifold Approach for Face Recognition," Proc. Sixth Int'l Conf. Automatic Face and Gesture Recognition, May 2004.
[28] Z. Zhang, J. Wang, and H. Zha, "Adaptive Manifold Learning," Advances in Neural Information Processing Systems, L.K. Saul, Y. Weiss, and L. Bottou, eds., vol. 17, pp. 1473-1480, MIT Press, 2005.
[29] Z. Zhang and H. Zha, "Structure and Perturbation Analysis of Truncated SVD for Column-Partitioned Matrices," SIAM J. Matrix Analysis and Applications, vol. 22, no. 4, pp. 1245-1262, 2001.
[30] Z. Zhang and H. Zha, "Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment," SIAM J. Scientific Computing, vol. 26, no. 1, pp. 313-338, 2004.
[31] H. Zha and Z. Zhang, "Continuum Isomap for Manifold Learning," Computational Statistics and Data Analysis, vol. 52, pp. 184-200, 2007.
[32] H. Zha and Z. Zhang, "Spectral Properties of the Alignment Matrices in Manifold Learning," SIAM Rev., vol. 51, no. 3, pp. 545-566, 2009.

Index Terms:
sampling methods,data analysis,learning (artificial intelligence),real-world data sets,low-dimensional parameterization,high-dimensional data points,global parameterization,adaptive manifold learning,adaptive selection,local neighborhood sizes,connectivity structure,adaptive bias reduction,sampling density,synthetic data sets,Manifolds,Linear approximation,Algorithm design and analysis,Accuracy,Approximation algorithms,Estimation,classification.,Manifold learning,dimensionality reduction,neighborhood selection,bias reduction
Citation:
Zhenyue Zhang, Jing Wang, Hongyuan Zha, "Adaptive Manifold Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 2, pp. 253-265, Feb. 2012, doi:10.1109/TPAMI.2011.115
Usage of this product signifies your acceptance of the Terms of Use.