This Article 
 Bibliographic References 
 Add to: 
Robust Linear Dimensionality Reduction
July/August 2004 (vol. 10 no. 4)
pp. 459-470

Abstract—We present a novel family of data-driven linear transformations, aimed at finding low-dimensional embeddings of multivariate data, in a way that optimally preserves the structure of the data. The well-studied PCA and Fisher's LDA are shown to be special members in this family of transformations, and we demonstrate how to generalize these two methods such as to enhance their performance. Furthermore, our technique is the only one, to the best of our knowledge, that reflects in the resulting embedding both the data coordinates and pairwise relationships between the data elements. Even more so, when information on the clustering (labeling) decomposition of the data is known, this information can also be integrated in the linear transformation, resulting in embeddings that clearly show the separation between the clusters, as well as their internal structure. All of this makes our technique very flexible and powerful, and lets us cope with kinds of data that other techniques fail to describe properly.

[1] T. Allison and D. Cicchetti, Sleep in Mammals: Ecological and Constitutional Correlates Science, vol. 194, pp. 732-734, 1976.
[2] E. Alpaydin and C. Kaynak, Cascading Classifiers Kybernetika, vol. 34, pp. 369-374, 1998.
[3] C.L. Blake and C.J. Merz, UCI Repository of Machine Learning Databases Irvine, Calif.: Univ. of California, Irvine, Dept. of Information and Computer Science, , 1998.
[4] L. Carmel, Y. Koren, and D. Harel, Visualizing and Classifying Odors Using a Similarity Matrix Proc. Ninth Int'l Symp. Olfaction and Electronic Nose (ISOEN '02), pp. 141-146, 2003.
[5] B.S. Everitt and G. Dunn, Applied Multivariate Data Analysis. Ar nold, 1991.
[6] R.A. Fisher, The Use of Multiple Measurements in Taxonomic Problems Annals of Eugenics, vol. 7, pp. 179-188, 1936.
[7] D.H. Foley and J.W. SammonJr., An Optimal Set of Discriminant Vectors IEEE Trans. Computers, vol. 24, pp. 281-289, 1975.
[8] K.M. Hall, An$r{\hbox{-}}{\rm{Dimensional}}$Quadratic Placement Algorithm Management Science, vol. 17, pp. 219-229, 1970.
[9] Y. Koren, L. Carmel, and D. Harel, ACE: A Fast Multiscale Eigenvectors Computation for Drawing Huge Graphs Proc. IEEE Symp. Information Visualization 2002 (InfoVis 2002), pp. 137-144, 2002.
[10] S.S. Schiffman, M.L. Reynolds, and F.W. Young, Introduction to Multidimensional Scaling: Theory, Methods and Applications. Academic Press, 1981.
[11] R. Johnson, Dataset Sleep StatLib Data Set Index, Carnegie Mellon Univ.,http:/ 1994.
[12] A.R. Webb, Statistical Pattern Recognition. John Wiley and Sons, 2002.

Index Terms:
Dimensionality reduction, visualization, classification, feature extraction, projection, linear transformation, principal component analysis, Fisher's linear discriminant analysis.
Yehuda Koren, Liran Carmel, "Robust Linear Dimensionality Reduction," IEEE Transactions on Visualization and Computer Graphics, vol. 10, no. 4, pp. 459-470, July-Aug. 2004, doi:10.1109/TVCG.2004.17
Usage of this product signifies your acceptance of the Terms of Use.