
This Article  
 
Share  
Bibliographic References  
Add to:  
Digg Furl Spurl Blink Simpy Del.icio.us Y!MyWeb  
Search  
 
ASCII Text  x  
Kuiyu Chang, Joydeep Ghosh, "A Unified Model for Probabilistic Principal Surfaces," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 1, pp. 2241, January, 2001.  
BibTex  x  
@article{ 10.1109/34.899944, author = {Kuiyu Chang and Joydeep Ghosh}, title = {A Unified Model for Probabilistic Principal Surfaces}, journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence}, volume = {23}, number = {1}, issn = {01628828}, year = {2001}, pages = {2241}, doi = {http://doi.ieeecomputersociety.org/10.1109/34.899944}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, }  
RefWorks Procite/RefMan/Endnote  x  
TY  JOUR JO  IEEE Transactions on Pattern Analysis and Machine Intelligence TI  A Unified Model for Probabilistic Principal Surfaces IS  1 SN  01628828 SP22 EP41 EPD  2241 A1  Kuiyu Chang, A1  Joydeep Ghosh, PY  2001 KW  Principal curve KW  principal surface KW  probabilistic KW  dimensionality reduction KW  nonlinear manifold KW  generative topographic mapping. VL  23 JA  IEEE Transactions on Pattern Analysis and Machine Intelligence ER   
Abstract—Principal curves and surfaces are nonlinear generalizations of principal components and subspaces, respectively. They can provide insightful summary of highdimensional data not typically attainable by classical linear methods. Solutions to several problems, such as proof of existence and convergence, faced by the original principal curve formulation have been proposed in the past few years. Nevertheless, these solutions are not generally extensible to principal surfaces, the mere computation of which presents a formidable obstacle. Consequently, relatively few studies of principal surfaces are available. Recently, we proposed the probabilistic principal surface (PPS) to address a number of issues associated with current principal surface algorithms. PPS uses a manifold oriented covariance noise model, based on the generative topographical mapping (GTM), which can be viewed as a parametric formulation of Kohonen's selforganizing map. Building on the PPS, we introduce a unified covariance model that implements PPS
[1] J.H. Friedman, “An Overview of Predictive Learning and Function Approximation,” From Statistics to Neural Networks, Proc. NATO/ASI Workshop, V. Cherkassky, J.H. Friedman, and H. Wechsler, eds., pp. 161, 1994.
[2] R. Fisher, “The Case of Multiple Measurements in Taxonomic Problems,” Annals of Eugenics, vol. 7, no. part II, pp. 179188, 1936.
[3] R.A. Johnson and D.W. Wichern,Applied multivariate statistical analysis, Prentice Hall, 1988.
[4] A. Hyvärinen, “Survey on Independent Component Analysis,” Neural Computing Surveys, vol. 2, pp. 94128, 1999.
[5] J.H. Friedman, “Exploratory Projection Pursuit,” J. Am. Statistical Assoc., vol. 82, no. 397, pp. 249266, Mar. 1987.
[6] L. Fahrmeir and G. Tutz, Multivariate Statistical Modelling Based on Generalized Linear Models. New York: SpringerVerlag, 1994.
[7] G.W. Cottrell, P. Munroe, and D. Zipser, “Image Compression by Back Propagation: An Example of Extensional Programming,” Technical Report ICS 8702, Univ. of California at San Diego, 1987.
[8] T. Kohonen, SelfOrganizing Maps. Berlin: SpringerVerlag, 1995.
[9] T. Hastie and W. Stuetzle, “Principal Curves,” J. Am. Statistical Assoc., vol. 84, no. 406, pp. 502516, June 1988.
[10] M. LeBlanc and R. Tibshirani, “Adaptive Principal Surfaces,” J. Am. Statistical Assoc., vol. 89, no. 425, pp. 5364, Mar. 1994.
[11] C.M. Bishop, Neural Networks for Pattern Recognition. Clarendon Press, 1995.
[12] M.E. Tipping and C.M. Bishop, “Mixtures of Probabilistic Principal Component Analysers,” Technical Report NCRG/97/003, Aston Univ., June 1997.
[13] C.M. Bishop and M.E. Tipping, “A Hierarchical Latent Variable Model for Data Visualization,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 281293, Mar. 1998.
[14] H. Attias, “Independent Factor Analysis,” Neural Computation, vol. 11, no. 4, pp. 803851, May 1999.
[15] K.y. Chang and J. Ghosh, “Probabilistic Principal Surfaces,” Proc. Int'l Joint Conf. Neural Networks, p. 605, July 1999.
[16] C.M. Bishop, M. Svensén, and C.K.I. Williams, “GTM: The Generative Topographic Mapping,” Neural Computation, vol. 10, no. 1, pp. 215235, 1998.
[17] H. Ritter, T. Martinetz, and K. Schulten, Neural Computation and SelfOrganizing Maps: An Introduction. Reading, Mass.: AddisonWesley, 1992.
[18] F. Mulier and V. Cherkassky, “SelfOrganization as an Iterative Kernel Smoothing Process,” Neural Computation, vol. 7, pp. 1,1651,177, 1995.
[19] C. de Boor, A Practical Guide to Splines. New York: SpringerVerlag, 1978.
[20] W. Cleveland and S. Devlin, “Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting,” J. Am. Statistical Assoc., vol. 83, pp. 596610, 1988.
[21] T. Duchamp and W. Stuetzle, “Extremal Properties of Principal Curves in the Plane,” Annals of Statistics, vol. 24, no. 4, pp. 15111520, 1996.
[22] J.D. Banfield and A.E. Raftery, “Ice Floe Identification in Satellite Images Using Mathematical Morphology and Clustering about Principal Curves,” J. Am. Statistical Assoc., vol. 87, no. 417, pp. 716, Mar 1992.
[23] K.y. Chang and J. Ghosh, “Principal Curves for Nonlinear Feature Extraction and Classification,” SPIE: Applications of Artificial Neural Networks in Image Processing III, vol. 3307, pp. 120129, Jan. 1998.
[24] R. Tibshirani, “Principal Curves Revisited,” Statistics and Computing, vol. 2, pp. 183190, 1992.
[25] A.P. Dempster, N.M. Laird, and D.B. Rubin, “MaximumLikelihood from Incomplete Data via the EM Algorithm,” J. Royal Statistical Soc., vol. 39, no. 1, pp. 138, 1977.
[26] C.F.J. Wu, “On the Convergence Properties of the EM Algorithm,” Annals of Statistics, vol. 11, pp. 95103, 1983.
[27] B. Kégl, A. Krzyzak, T. Linder, and K. Zeger, “Learning and Design of Principal Curves,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 3, pp. 281297, Mar. 2000.
[28] B. Kégl, A. Krzyzak, T. Linder, and K. Zeger, “Principal Curves: Learning and Convergence,” Proc. IEEE Int'l Symp. Information Theory, 1998.
[29] B. Kégl, A. Krzyzak, T. Linder, and K. Zeger, “A Polygonal Line Algorithm for Constructing Principal Curves,” Neural Information Processing Systems, vol. 11, pp. 501507, 1998.
[30] P. Delicado, “Principal Curves and Principal Oriented Points,” Technical Report 309, Departament d'Economia i Empresa, Universitat Pompeu Fabra, 1998.
[31] P. Delicado, “Another Look at Principal Curves and Surfaces,” unpublished, 1999.
[32] T. Hastie, “Principal Curves and Surfaces,” PhD thesis, Stanford Univ., 1984.
[33] J.H. Friedman, “Multivariate Adaptive Regression Splines,” Annals of Statistics, vol. 19, pp. 1141, 1991.
[34] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. John Wiley&Sons, 1973.
[35] E. Erwin, K. Obermayer, and K. Schulten, “SelfOrganizing Maps: Ordering, Convergence Properties, and Energy Functions,” Biological Cybernetics, vol. 67, pp. 4755, 1992.
[36] J. Hertz, A. Krogh, and R.G. Palmer, Introduction to the Theory of Neural Computation. AddisonWesley, 1991.
[37] C.M. Bishop and M. Svensén, “GTM: The Generative Topographic Mapping,” Technical Report NCRG/96/015, Aston Univ., Apr. 1997.
[38] T.K. Moon, The ExpectationMaximization Algorithm in Signal Processing IEEE Signal Processing Magazine, vol. 13, no. 6, pp. 4760, 1996.
[39] K.y. Chang, “Image and Signal Processing Using Neural Networks,” MS thesis, Dept. of Electrical Eng., Univ. of Hawaii at Ma noa, Dec. 1994.
[40] H. Bourland and Y. Kamp, “AutoAssociation by Multilayer Perceptrons and Singular Value Decomposition,” Biological Cybernetics, vol. 59, pp. 291294, 1988.
[41] M.A. Kramer, “Nonlinear Principal Component Analysis Using Autoassociative Neural Networks,” Am. Inst. Chemical Eng. J., vol. 37, no. 2, pp. 233243, 1991.
[42] S. Tan and M.L. Mavrovouniotis, “Reducing Data Dimensionality through Optimizing Neural Network Inputs,” Am. Inst. Chemical Eng. J., vol. 41, no. 6, pp. 14711480, June 1995.
[43] D. Dong and T.J. McAvoy, “Nonlinear Principal Component AnalysisBased on Principal Curves and Neural Networks,” Computers and Chemical Eng., vol. 20, no. 1, pp. 6578, 1996.
[44] J. Karhunen, E. Oja, L. Wang, R. Vigario, and J. Joutsensalo, “A Class of Neural Networks for Independent Component Analysis,” IEEE Trans. Neural Networks, vol. 8, no. 3, pp. 486504, 1997.
[45] E.C. Malthouse, “Limitations of Nonlinear PCA as Performed with Generic Neural Networks,” IEEE Trans. Neural Networks, vol. 9, no. 1, pp. 165173, Jan. 1998.
[46] C.M. Bishop, M. Svensén, and C.K.I. Williams, “Developments of the Generative Topographic Mapping,” Neurocomputing, vol. 21, pp. 203224, 1998.
[47] M. Svensén, “GTM: The Generative Topographic Mapping,” PhD thesis, Aston Univ., Birmingham, UK, 1998, http://www.ncrg.aston.ac.ukGTM/.
[48] C.L. Blake and C.J. Merz, “UCI Repository of Machine Learning Databases,” 1998.
[49] K.y. Chang and J. Ghosh, “ThreeDimensional ModelBased Object Recognition and Pose Estimation Using Probabilistic Principal Surfaces,” SPIE: Applications of Artificial Neural Networks in Image Processing V, pp. 192203, Jan. 2000.
[50] K.y. Chang, “Nonlinear Dimensionality Reduction Using Probabilistic Principal Surfaces,” PhD thesis, Dept. of Electrical and Computer Eng., Univ. of Texas at Austin, May 2000.
[51] J.R. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics. John Wiley&Sons, 1988.