This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Hierarchical GTM: Constructing Localized Nonlinear Projection Manifolds in a Principled Way
May 2002 (vol. 24 no. 5)
pp. 639-656

It has been argued that a single two-dimensional visualization plot may not be sufficient to capture all of the interesting aspects of complex data sets and, therefore, a hierarchical visualization system is desirable. In this paper, we extend an existing locally linear hierarchical visualization system PhiVis in several directions: 1) We allow for nonlinear projection manifolds. The basic building block is the Generative Topographic Mapping (GTM). 2) We introduce a general formulation of hierarchical probabilistic models consisting of local probabilistic models organized in a hierarchical tree. General training equations are derived, regardless of the position of the model in the tree. 3) Using tools from differential geometry, we derive expressions for local directional curvatures of the projection manifold. Like PhiVis, our system is statistically principled and is built interactively in a top-down fashion using the EM algorithm. It enables the user to interactively highlight those data in the ancestor visualization plots which are captured by a child model. We also incorporate into our system a hierarchical, locally selective representation of magnification factors and directional curvatures of the projection manifolds. Such information is important for further refinement of the hierarchical visualization plot, as well as for controlling the amount of regularization imposed on the local models. We demonstrate the principle of the approach on a toy data set and apply our system to two more complex 12- and 18-dimensional data sets.

[1] C.M. Bishop and M.E. Tipping, “A Hierarchical Latent Variable Model for Data Visualization,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 281-293, Mar. 1998.
[2] C.M. Bishop, M. Svensén, and C.K.I. Williams, “GTM: The Generative Topographic Mapping,” Neural Computation, vol. 10, no. 1, pp. 215-235, 1998.
[3] T. Kohonen, “The Self-Organizing Map,” Proc. IEEE, vol. 78, no. 9, pp. 1464-1480, Sept. 1990.
[4] C.M. Bishop, M. Svensén, and C.K.I. Williams, “Developments of the Generative Topographic Mapping,” Neurocomputing, vol. 21, pp. 203-224, 1998.
[5] C.M. Bishop, M. Svensén, and C.K.I. Williams, “Magnification Factors for the SOM and GTM Algorithms,” Proc. 1997 Workshop Self-Organizing Maps, 1997.
[6] R. Miikkulainen, “Script Recognition with Hierarchical Feature Maps,” Connection Science, vol. 2, pp. 83-101, 1990.
[7] C. Versino and L.M. Gambardella, “Learning Fine Motion by Using the Hierarchical Extended Kohonen Map,” Proc. ICANN 96, Int'l Conf. Artificial Neural Networks, pp. 221-226, 1996.
[8] C. Versino and L.M. Gambardella, “Learning Fine Motion in Robotics: Experiments with the Hierarchical Extended Kohonen Map,” Proc. ICONIP 96, Int'l Conf. Neural Information Processing, vol. 2, pp. 921-925, 1996.
[9] C.K.I. Williams, “A MCMC Approach to Hierarchical Mixture Modelling,” Advances in Neural Information Processing Systems 12, S. Solla, T. Leen, and K.R. Muller, eds. pp. 680-686, 2000.
[10] C.M. Bishop, Neural Networks for Pattern Recognition. Clarendon Press, 1995.
[11] A.P. Dempster, N.M. Laird, and D.B. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” J. Royal Statistical Soc., B, vol. 39, no. 1, pp. 1-38, 1977.
[12] W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C.Cambridge, England: Cambridge Univ. Press, 1988.
[13] F. Aurenhammer, "Voronoi Diagrams: A Survey of a Fundamental Geometric Data Structure," ACM Computing Surveys, vol. 23, no. 3, 1991, pp. 345-405.
[14] M.E. Tipping and C.M. Bishop, “Mixtures of Probabilistic Principal Component Analysers,” Neural Computation, vol. 11, no. 2, pp. 443-482, 1999.
[15] C.M. Bishop, M. Svensén, and C.K.I. Williams, “Magnification Factors for the GTM Algorithm,” Proc. IEE Fifth Int'l Conf. Artificial Neural Networks, pp. 64-69, 1997.
[16] A. Ultsch and H.P. Siemon, “Kohonen's Self Organizing Feature Maps for Exploratory Data Analysis,” Proc. Int'l Neural Network Conf. (INNC '90), pp. 305-308, 1990.
[17] A. Ultsch, “Knowledge Extraction from Self-Organizing Neural Networks,” Information and Classification, pp. 301-306, 1993.
[18] J. Vesanto, “SOM-Based Data Visualization Methods,” Intelligent Data Analysis, vol. 3, pp. 111-126, 1999.
[19] R.A. Horn and C.R. Johnson, Matrix Analysis. Cambridge Univ. Press, 1990.
[20] Nonlinear Regression. G.A.F. Seber and C.J. Wild, eds. New York: John Wiley, 1989.
[21] D.M. Bates and D.G. Watts, “Relative Curvature Measures of Nonlinearity (with discussion),” J. Royal Statistics Soc. B, vol. 42, pp. 1-25, 1980.
[22] S.-I. Amari, Differential-Geometrical Methods in Statistics. Berlin: Springer-Verlag, 1985.
[23] K. Rose, E. Gurewitz, and G.C. Fox, “Statistical Mechanics and Phase Transitions in Clustering,” Physical Review Letters, vol. 65, no. 8, pp. 945-948, 1990.

Index Terms:
Hierarchical probabilistic model, generative topographic mapping, data visualization, EM algorithm, density estimation, directional curvature
Citation:
P. Tino, I. Nabney, "Hierarchical GTM: Constructing Localized Nonlinear Projection Manifolds in a Principled Way," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 639-656, May 2002, doi:10.1109/34.1000238
Usage of this product signifies your acceptance of the Terms of Use.