The Community for Technology Leaders
RSS Icon
Issue No.11 - November (2009 vol.31)
pp: 2093-2098
Kevin M. Carter , Information Systems Technology Group, Lexington
Raviv Raich , Oregon State University, Corvallis
William G. Finn , University of Michigan, Ann Arbor
Alfred O. Hero III , University of Michigan, Ann Arbor
We consider the problems of clustering, classification, and visualization of high-dimensional data when no straightforward euclidean representation exists. In this paper, we propose using the properties of information geometry and statistical manifolds in order to define similarities between data sets using the Fisher information distance. We will show that this metric can be approximated using entirely nonparametric methods, as the parameterization and geometry of the manifold is generally unknown. Furthermore, by using multidimensional scaling methods, we are able to reconstruct the statistical manifold in a low-dimensional euclidean space; enabling effective learning on the data. As a whole, we refer to our framework as Fisher Information Nonparametric Embedding (FINE) and illustrate its uses on practical problems, including a biomedical application and document classification.
Information geometry, statistical manifold, dimensionality reduction, multidimensional scaling.
Kevin M. Carter, Raviv Raich, William G. Finn, Alfred O. Hero III, "FINE: Fisher Information Nonparametric Embedding", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 11, pp. 2093-2098, November 2009, doi:10.1109/TPAMI.2009.67
[1] J. Lafferty and G. Lebanon, “Diffusion Kernels on Statistical Manifolds,” J.Machine Learning Research, vol. 6, pp. 129-163, Jan. 2005.
[2] O. Arandjelovic, G. Shakhnarovich, J. Fisher, R. Cipolla, and T. Darrell, “Face Recognition with Image Sets Using Manifold Density Divergence,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 581-588, June 2005.
[3] S.-M. Lee, A.L. Abbott, and P.A. Araman, “Dimensionality Reduction and Clustering on Statistical Manifolds,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-7, June 2007.
[4] A. Srivastava, I.H. Jermyn, and S. Joshi, “Riemannian Analysis of Probability Density Functions with Applications in Vision,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2007.
[5] J. Salojarvi, S. Kaski, and J. Sinkkonen, “Discriminative Clustering in Fisher Metrics,” Proc. Int'l Conf. Artificial Neural Networks and Neural Information Processing, pp. 161-164, June 2003.
[6] M. Belkin and P. Niyogi, “Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering,” Advances in Neural Information Processing Systems, T.G. Dietterich, S. Becker, and Z. Ghahramani, eds., vol. 14, MIT Press, 2002.
[7] J.B. Tenenbaum, V. de Silva, and J.C. Langford, “A Global Geometric Framework for Nonlinear Dimensionality Reduction,” Science, vol. 290, pp.2319-2323, 2000.
[8] S. Roweis and L. Saul, “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” Science, vol. 290, no. 1, pp. 2323-2326, 2000.
[9] T. Cox and M. Cox, Multidimensional Scaling. Chapman & Hall, 1994.
[10] R. Kass and P. Vos, Geometrical Foundations of Asymptotic Inference. John Wiley and Sons, 1997.
[11] S. Amari and H. Nagaoka, Methods of Information Geometry (Translations of Mathematical Monographs), vol. 191, Am. Math. Soc. and Oxford Univ. Press, 2000.
[12] S.K. Zhou and R. Chellappa, “From Sample Similarity to Ensemble Similarity: Probabilistic Distance Measures in Reproducing Kernel Hilbert Space,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 917-929, June 2006.
[13] K.M. Carter, “Dimensionality Reduction on Statistical Manifolds,” PhD thesis, Univ. of Michigan, Jan. 2009.
[14] R. Raich, J.A. Costa, and A.O. Hero, “On Dimensionality Reduction for Classification and Its Application,” Proc. IEEE Int'l Conf. Acoustic Speech and Signal Processing, vol. 5, May 2006.
[15] K.M. Carter, R. Raich, W.G. Finn, and A.O. Hero, “Information Preserving Component Analysis: Data Projections for Flow Cytometry Analysis,” IEEE J. Selected Topics in Signal Processing, special issue on digital image processing techniques for oncology, vol. 3, no. 1, pp. 148-158, Feb. 2009.
[16] W.G. Finn, K.M. Carter, R. Raich, and A.O. Hero, “Analysis of Clinical Flow Cytometric Immunophenotyping Data by Clustering on Statistical Manifolds: Treating Flow Cytometry Data as High-Dimensional Objects,” Cytometry Part B: Clinical Cytometry, vol. 76B, no. 1, pp. 1-7, Jan. 2009.
[17] G. Terrell, “The Maximal Smoothing Principle in Density Estimation,” J.Am. Statistical Assoc., vol. 85, no. 410, pp. 470-477, June 1990.
[18] H. Kim, P. Howland, and H. Park, “Dimension Reduction in Text Classification with Support Vector Machines,” J. Machine Learning Research, vol. 6, pp. 37-53, Jan. 2005.
[19] S. Huang, M.O. Ward, and E.A. Rundensteiner, “Exploration of Dimensionality Reduction for Text Visualization,” Proc. IEEE Third Int'l Conf. Coordinated and Multiple Views in Exploratory Visualization, pp. 63-74, July 2005.
[20] K.M. Carter, R. Raich, and A.O. Hero, “Fine: Information Embedding for Document Classification,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 1861-1864, Apr. 2008.
15 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool