
This Article  
 
Share  
Bibliographic References  
Add to:  
Digg Furl Spurl Blink Simpy Del.icio.us Y!MyWeb  
Search  
 
ASCII Text  x  
Sarunas Raudys, Ausra Saudargiene, "FirstOrder TreeType Dependence between Variables and Classification Performance," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 233239, February, 2001.  
BibTex  x  
@article{ 10.1109/34.908975, author = {Sarunas Raudys and Ausra Saudargiene}, title = {FirstOrder TreeType Dependence between Variables and Classification Performance}, journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence}, volume = {23}, number = {2}, issn = {01628828}, year = {2001}, pages = {233239}, doi = {http://doi.ieeecomputersociety.org/10.1109/34.908975}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, }  
RefWorks Procite/RefMan/Endnote  x  
TY  JOUR JO  IEEE Transactions on Pattern Analysis and Machine Intelligence TI  FirstOrder TreeType Dependence between Variables and Classification Performance IS  2 SN  01628828 SP233 EP239 EPD  233239 A1  Sarunas Raudys, A1  Ausra Saudargiene, PY  2001 KW  Firstorder treetype dependence KW  a priori information KW  classification KW  generalization KW  sample size KW  dimensionality. VL  23 JA  IEEE Transactions on Pattern Analysis and Machine Intelligence ER   
Abstract—Structuralization of the covariance matrix reduces the number of parameters to be estimated from the training data and does not affect an increase in the generalization error asymptotically as both the number of dimensions and training sample size grow. A method to benefit from approximately correct assumptions about the first order tree dependence between components of the feature vector is proposed. We use a structured estimate of the covariance matrix to decorrelate and scale the data and to train a singlelayer perceptron in the transformed feature space. We show that training the perceptron can reduce negative effects of inexact a priori information. Experiments performed with 13 artificial and 10 real world data sets show that the firstorder treetype dependence model is the most preferable one out of two dozen of the covariance matrix structures investigated.
[1] J.J. Atick and A.N. Redlich, “Towards a Theory of Early Visual Processing,” Neural Computation, vol. 2, pp. 308320, 1990.
[2] C.K. Chow and C.N. Liu, “Approximating Discrete Probability Distributions with Dependence Trees,” IEEE Trans. Information Theory, vol. 14, pp. 462467, 1968.
[3] A.D. Deev, “Representation of Statistics of Discriminant Analysis and Asymptotic Expansions in Dimensionalities Comparable with Sample Size,” Reports of Academy of Sciences of the USSR, vol. 195, no. 4, pp. 756762, 1970 (in Russian).
[4] A.D. Deev, “Asymptotic Expansions for Distributions of StatisticsW,M,W* in Discriminant Analysis,” Statistical Methods of Classification, J.N. Blagoveshenskij, ed., vol. 31, pp. 657, Moscow: Moscow Univ. Press, 1972 (in Russian).
[5] A.D. Deev, “Discriminant Function Designed on Independent Blocks of Variables,” Eng. Cybernetics (Proc. Academy of Sciences of the USSR), no. 12, pp. 153156, 1974 (in Russian).
[6] J.M. Friedman, “Regularized Discriminant Analysis,” J. Am. Statistical Assoc., vol. 84, pp. 165175, 1989.
[7] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[8] S. Halkaaer and O. Winter, “The Effect of Correlated Input Data on the Dynamics of Learning,” Advances in Neural Information Processing Systems, M.C. Mozer, M.I. Jordan, and T. Petsche, eds., vol. 9, pp. 169175, Cambridge, Mass.: MIT Press, A Radford Book, 1996.
[9] I.B. Kruskal Jr., “On the Shortest Spanning Subtree of a Graph and the Travelling Salesman Problem,” Proc. Am. Math. Soc., vol. 7, pp. 4850, 1956.
[10] Y. le Cun, I. Kanter, and S. Solla, “Eigenvalues of Covariance Matrices: Application to NeuralNetwork Learning,” Physical Review Letters, vol. 66, no. 18, pp. 23962399, 1991.
[11] G.J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition. New York: Wiley, 1992.
[12] L.D. Meshalkin, “Assignment of Numerical Values to Nominal Variables,” Statistical Problems Control, S. Raudys and L. Meshalkin, eds., vol. 14, pp. 4956, Vilnius: Inst. of Math. and Cybernetics Press, 1976 (in Russian).
[13] L.D. Meshalkin and V.I. Serdobolskij, “Errors in Classifying Multivariate Observations,” Theory of Probabilities and Its Applications, vol. 23, no. 4, pp. 772781, 1978 (in Russian).
[14] D. Morgera and D.B. Cooper, “Structurized Estimation: Sample Size Reduction for Adaptive Pattern Classification,” IEEE Trans. Information Theory, vol. 23, pp. 728741, 1977.
[15] R. Prochorskas, V. Ziuznis, and N. Misiuniene, “Use of Different Classifiers to Predict Outcomes of Heart Attacks,” Problems of Ischemic Heart Diseases, pp. 216267, Vilnius, Lithuania: Mokslas Publishing House, 1976 (in Russian).
[16] S. Raudys, “On Determining Training Sample Size of Linear Classifier,” Computing Systems, N.G. Zagoruiko ed., vol. 28, pp. 7987, Inst. of Math. Press, Novosibirsk: Nauka, 1967 (in Russian).
[17] S. Raudys, “On the Amount of a priori Information in Designing the Classification Algorithm,” Eng. Cybernetics (Proc. Academy of Sciences of the USSR), no. 4, pp. 168174, 1972 (in Russian).
[18] S. Raudys, “Methods to Overcome Dimensionality Problems in Statistical Pattern Recognition: A Review,” Zavodskaya Laboratorya (Factory Lab., Interdisciplinary USSR J.), no. 3, pp. 45&4955, Moscow: Nauka, 1991 (in Russian).
[19] S. Raudys, “Evolution and Generalization of a Single Neurone: I. Singlelayer Perception as Seven Statistical Classifiers,” Neural Networks, vol. 11, no. 2, pp. 283296, 1998.
[20] S. Raudys, “Scaled Rotation Regularization,” Pattern Recognition, vol. 33, pp. 19891998, 2000.
[21] S. Raudys and S. Amari, “Effect of Initial Values in Simple Perception,” Proc. 1998 IEEE World Congress Computational Intelligence, IJCNN '98, pp. 15301535, 1998.
[22] S.J. Raudys and A.K. Jain, "Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, pp. 252264, 1991.
[23] S. Raudys and V. Pikelis, “On Dimensionality, Sample Size, Classification Error and Complexity of the Classification Algorithm in Pattern Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 2, pp. 242252, 1980.
[24] S. Raudys and A. Saudargiene, “Structures of the Covariance Matrices in the Classifier Design,” Proc. Joint IAPR Int'l Workshops/SSPR '98 and SPR '98, pp. 583592, 1998.
[25] A. Saudargiene, “Structurization of the Covariance Matrix by Process Type and BlockDiagonal Models in the Classifier Design,” Informatica, vol. 10, no. 2, pp. 245269, Vilnius: Inst. of Math. and Informatics Press, 1999.
[26] V.I. Serdobolskij, “The Moments of Discriminant Function and Classification for a Large Number of Variables,” S. Raudys, ed., vol. 38, pp. 2751, Vilnius: Statistical Problems of Control. Inst. of Math. and Cyb. Press, 1979 (in Russian).
[27] V.I. Zarudskij, “The Use of Models of Simple Dependence Problems in Classification,” Statistical Problems of Control, S. Raudys, ed., vol. 38, pp. 5375, Vilnius: Inst. of Math. and Cyb. Press, 1979 (in Russian).
[28] V.I. Zarudskij, “Determination of Some Graph Connections for Normal Vectors in Large Dimensional Case,” Algorithmic and Programic Supply of Applied Multivariate Statistical Analysis, S.A. Aivazian, ed., pp. 189208, Moscow: Nauka, 1980 (in Russian).