This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Incremental Nonlinear Dimensionality Reduction by Manifold Learning
March 2006 (vol. 28 no. 3)
pp. 377-391
ASCII Text
x 
 
Martin H.C. Law, Anil K. Jain, "Incremental Nonlinear Dimensionality Reduction by Manifold Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 377-391, March, 2006.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2006.56,
author = {Martin H.C. Law and Anil K. Jain},
title = {Incremental Nonlinear Dimensionality Reduction by Manifold Learning},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {28},
number = {3},
issn = {0162-8828},
year = {2006},
pages = {377-391},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2006.56},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Incremental Nonlinear Dimensionality Reduction by Manifold Learning
IS - 3
SN - 0162-8828
SP377
EP391
EPD - 377-391
A1 - Martin H.C. Law,
A1 - Anil K. Jain,
PY - 2006
KW - Incremental learning
KW - dimensionality reduction
KW - ISOMAP
KW - manifold learning
KW - unsupervised learning.
VL - 28
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Understanding the structure of multidimensional patterns, especially in unsupervised cases, is of fundamental importance in data mining, pattern recognition, and machine learning. Several algorithms have been proposed to analyze the structure of high-dimensional data based on the notion of manifold learning. These algorithms have been used to extract the intrinsic characteristics of different types of high-dimensional data by performing nonlinear dimensionality reduction. Most of these algorithms operate in a "batch” mode and cannot be efficiently applied when data are collected sequentially. In this paper, we describe an incremental version of ISOMAP, one of the key manifold learning algorithms. Our experiments on synthetic data as well as real world images demonstrate that our modified algorithm can maintain an accurate low-dimensional representation of the data in an efficient manner.

[1] P. Baldi and K. Hornik, “Neural Networks and Principal Component Analysis: Learning from Examples without Local Minima,” Neural Networks, vol. 2, pp. 53-58, 1989.
[2] M. Belkin and P. Niyogi, “Laplacian Eigenmaps for Dimensionality Reduction and Data Representation,” Neural Computation, vol. 15, no. 6, pp. 1373-1396, June 2003.
[3] Y. Bengio, J.-F. Paiement, and P. Vincent, “Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering,” Advances in Neural Information Processing Systems 16, MIT Press, 2004.
[4] M. Bernstein, V. de Silva, J. Langford, and J. Tenenbaum, “Graph Approximations to Geodesics on Embedded Manifolds,” technical report, Dept. of Psychology, Stanford Univ., 2000.
[5] A. Beygelzimer, S.M. Kakade, and J. Langford, “Cover Trees for Nearest Neighbor,” technical report, Univ. of Pennsylvania, 2005, http://www.cis.upenn.edu/~skakade/papers/ mlcover_tree.pdf.
[6] C.M. Bishop, M. Svensen, and C.K. I. Williams, “GTM: The Generative Topographic Mapping,” Neural Computation, vol. 10, pp. 215-234, 1998.
[7] M. Brand, “Charting a Manifold,” Advances in Neural Information Processing Systems 15, pp. 961-968, 2003.
[8] A. Brun, H.-J. Park, H. Knutsson, and C.-F. Westin, “Coloring of DT-MRI Fiber Traces Using Laplacian Eigenmaps,” Proc. Ninth Int'l Conf. Computer Aided Systems Theory, vol. 2809, Feb. 2003.
[9] J. Bruske and G. Sommer, “Intrinsic Dimensionality Estimation with Optimally Topology Preserving Maps,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 5, pp. 572-575, May 1998.
[10] F. Camastra and A. Vinciarelli, “Estimating the Intrinsic Dimension of Data with a Fractal-Based Method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 10, pp. 1404-1407, Oct. 2002.
[11] Y. Chang, C. Hu, and M.M. Turk, “Probabilistic Expression Analysis on Manifolds,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 520-527, 2004.
[12] J. Costa and A.O. Hero, “Manifold Learning Using Euclidean K-Nearest Neighbor Graphs,” Proc. IEEE Int'l Conf. Acoustic Speech and Signal Processing, vol. 3, pp. 988-991, 2004.
[13] T.F. Cox and M.A.A. Cox, Multidimensional Scaling. Chapman and Hall, 2001.
[14] V. de Silva and J.B. Tenenbaum, “Global versus Local Approaches to Nonlinear Dimensionality Reduction,” Advances in Neural Information Processing Systems 15, pp. 705-712, 2003.
[15] D. DeMers and G. Cottrell, “Nonlinear Dimensionality Reduction,” Advances in Neural Information Processing Systems 5, pp. 580-587, 1993.
[16] C. Demetrescu and G.F. Italiano, “A New Approach to Dynamic All Pairs Shortest Paths,” J. ACM, vol. 51, no. 6, pp. 968-992, Nov. 2004.
[17] D.L. Donoho and C. Grimes, “When Does Isomap Recover Natural Parameterization of Families of Articulated Images?” Technical Report 2002-27, Dept. of Statistics, Stanford Univ., Aug. 2002.
[18] R. Duda, P. Hart, and D. Stork, Pattern Classification, second ed. John Wiley and Sons, 2001.
[19] A. Elgammal and C.S. Lee, “Inferring 3D Body Pose from Silhouettes Using Activity Manifold Learning,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 681-688, 2004.
[20] A. Elgammal and C.S. Lee, “Separating Style and Content on a Nonlinear Manifold,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 478-489, 2004.
[21] C. Fowlkes, S. Belongie, F. Chung, and J. Malik, “Spectral Grouping Using the Nyström Method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 214-225, Feb. 2004.
[22] G.H. Golub and C.F. Van Loan, Matrix Computations. Johns Hopkins Univ. Press, 1996.
[23] A. Hadid, O. Kouropteva, and M. Pietikainen, “Unsupervised Learning Using Locally Linear Embedding: Experiments in Face Pose Analysis,” Proc. 16th Int'l Conf. Pattern Recognition, pp. I: 111-114, 2002.
[24] T. Hastie and W. Stuetzle, “Principal Curves,” J. Am. Statistical Assoc., vol. 84, pp. 502-516, 1989.
[25] G.E. Hinton, P. Dayan, and M. Revow, “Modeling the Manifolds of Handwritten Digits,” IEEE Trans. Neural Networks, vol. 8, no. 1, pp. 65-74, Jan. 1997.
[26] O. Jenkins and M. Mataric, “A Spatio-Temporal Extension to ISOMAP Nonlinear Dimension Reduction,” Proc. 21st Int'l Conf. Machine Learning, 2004.
[27] B. Kégl, “Intrinsic Dimension Estimation Using Packing Numbers,” Advances in Neural Information Processing Systems 15, 2003.
[28] B. Kégl, A. Krzyzak, T. Linder, and K. Zeger, “Learning and Design of Principal Curves,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 3, pp. 281-297, Mar. 2000.
[29] T. Kohonen, Self-Organizing Maps, third ed. Springer-Verlag, 2001.
[30] M.H. Law, N. Zhang, and A.K. Jain, “Nonlinear Manifold Learning for Data Stream,” Proc. SIAM Int'l Conf. Data Mining, pp. 33-44, 2004.
[31] E. Levina and P.J. Bickel, “Maximum Likelihood Estimation of Intrinsic Dimension,” Advances in Neural Information Processing Systems 17, 2005.
[32] S.Z. Li, R. Xiao, Z.Y. Li, and H.J. Zhang, “Nonlinear Mapping from Multi-View Face Patterns to a Gaussian Distribution in a Low Dimensional Space,” Proc. IEEE ICCV Workshop Recognition, Analysis and Tracking of Faces and Gestures in Real-Time Systems, 2001.
[33] X. Lu and A.K. Jain, “Ethnicity Identification from Face Images,” Proc. Int'l Soc. Optical Eng. (SPIE), vol. 5404, pp. 114-123, 2004.
[34] J. Mao and A.K. Jain, “Artificial Neural Networks for Feature Extraction and Multivariate Data Projection,” IEEE Trans. Neural Networks, vol. 6, no. 2, pp. 296-317, Mar. 1995.
[35] A. Martinez and R. Benavente, “The AR Face Database,” Technical Report 24, Computer Vision Center, Univ. Alabama at Birmingham, 1998, http://rvl1.ecn.purdue.edu/~aleixaleix_face_DB. html .
[36] P. Narváez, K.-Y. Siu, and H.-Y. Tzeng, “New Dynamic Algorithms for Shortest Path Tree Computation,” IEEE/ACM Trans. Networking, vol. 8, no. 6, pp. 734-746, Dec. 2000.
[37] P. Narváez, K.-Y. Siu, and H.-Y. Tzeng, “New Dynamic SPT Algorithm Based on a Ball-and-String Model,” IEEE/ACM Trans. Networking, vol. 9, no. 6, pp. 706-718, Dec. 2001.
[38] M. Niskanen and O. Silvén, “Comparison of Dimensionality Reduction Methods for Wood Surface Inspection,” Proc. Sixth Int'l Conf. Quality Control by Artificial Vision, pp. 178-188, 2003.
[39] K. Pettis, T. Bailey, A.K. Jain, and R. Dubes, “An Intrinsic Dimensionality Estimator from Near-Neighbor Information,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 1, no. 1, pp. 25-36, Jan. 1979.
[40] S.T. Roweis, L.K. Saul, and G.E. Hinton, “Global Coordination of Local Linear Models,” Advances in Neural Information Processing Systems 14, pp. 889-896, 2002.
[41] J.W. Sammon, “A Nonlinear Mapping for Data Structure Analysis,” IEEE Trans. Computers, vol. 18, no. 5, pp. 401-409, May 1969.
[42] B. Schölkopf, A.J. Smola, and K.-R. Müller, “Nonlinear Component Analysis as a Kernel Eigenvalue Problem,” Neural Computation, vol. 10, pp. 1299-1319, 1998.
[43] P.Y. Simard, D. Steinkraus, and J. Platt, “Best Practice for Convolutional Neural Networks Applied to Visual Document Analysis,” Proc. Int'l Conf. Document Analysis and Recogntion, pp. 958-962, 2003.
[44] E. Sjöström, “Singular Value Computations for Toeplitz Matrices,” Licentiate thesis, 1996, http://www.mai.liu.se/~evlun/pub/liclic.html .
[45] A.J. Smola, S. Mika, B. Schölkopf, and R.C. Williamson, “Regularized Principal Manifolds,” J. Machine Learning Research, vol. 1, pp. 179-209, June 2001.
[46] J.B. Tenenbaum, V. de Silva, and J.C. Langford, “A Global Geometric Framework for Nonlinear Dimensionality Reduction,” Science, vol. 290, pp. 2319-2323, 2000.
[47] R. Tibshirani, “Principal Curves Revisited,” Statistics and Computing, vol. 2, pp. 183-190, 1992.
[48] J.J. Verbeek, N. Vlassis, and B. Krose, “Coordinating Principal Component Analyzers,” Proc. Int'l Conf. Artificial Neural Networks, pp. 914-919, 2002.
[49] K.Q. Weinberger and L.K. Saul, “Unsupervised Learning of Image Manifolds by Semidefinite Programming,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 988-995, 2004.
[50] J. Weng, Y. Zhang, and W.S. Hwang, “Candid Covariance-Free Incremental Principal Component Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 1034-1040, Aug. 2003.
[51] M.-H. Yang, “Face Recognition Using Extended Isomap,” Proc. IEEE Int'l Conf. Image Processing, pp. II: 117-120, 2002.
[52] H. Zha and Z. Zhang, “Isometric Embedding and Continuum Isomap,” Proc. 20th Int'l Conf. Machine Learning, 2003.
[53] J. Zhang, S.Z. Li, and J. Wang, “Nearest Manifold Approach for Face Recognition,” Proc. Sixth Int'l Conf. Automatic Face and Gesture Recognition, May 2004.

Index Terms:
Incremental learning, dimensionality reduction, ISOMAP, manifold learning, unsupervised learning.
Citation:
Martin H.C. Law, Anil K. Jain, "Incremental Nonlinear Dimensionality Reduction by Manifold Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 377-391, March 2006, doi:10.1109/TPAMI.2006.56
Usage of this product signifies your acceptance of the Terms of Use.