CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2009 vol.31 Issue No.02 - February

Issue No.02 - February (2009 vol.31)

pp: 288-305

Yixin Chen , University of Mississippi, University

Xin Dang , University of Mississippi, University

Hanxiang Peng , Purdue School of Science, IUPUI, Indianapolis

Henry L. Bart Jr. , Tulane University and Tulane University Museum of Natural History, New Orleans

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2008.72

ABSTRACT

Statistical depth functions provide from the ?deepest

INDEX TERMS

Outlier detection, novelty detection, anomaly detection, statistical depth function, spatial depth, kernel method, unsupervised learning.

CITATION

Yixin Chen, Xin Dang, Hanxiang Peng, Henry L. Bart Jr., "Outlier Detection with the Kernelized Spatial Depth Function",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.31, no. 2, pp. 288-305, February 2009, doi:10.1109/TPAMI.2008.72REFERENCES

- [1] N. Abe, B. Zadrozny, and J. Langford, “Outlier Detection by Active Learning,”
Proc. ACM SIGKDD '06, pp. 504-509, 2006.- [2] D.C. Adams and F.J. Rohlf, “Ecological Character Displacement in Plethodon: Biomechanical Differences Found from a Geometric Morphometric Study,”
Proc. Nat'l Academy of Sciences, vol. 97, pp.4106-4111, 2000.- [3] C.C. Aggarwal and P.S. Yu, “Outlier Detection for High Dimensional Data,”
Proc. ACM SIGMOD '01, pp. 37-46, 2001.- [4] F. Angiulli, S. Basta, and C. Pizzuti, “Distance-Based Detection and Prediction of Outliers,”
IEEE Trans. Knowledge and Data Eng., vol. 18, no. 2, pp. 145-160, Feb. 2006.- [5] A.C. Atkinson, “Fast Very Robust Methods for the Detection of Multiple Outliers,”
J. Am. Statistical Assoc., vol. 89, no. 428, pp.1329-1339, 1994.- [6] A. Banerjee, P. Burlina, and C. Diehl, “A Support Vector Method for Anomaly Detection in Hyperspectral Imagery,”
IEEE Trans. Geoscience and Remote Sensing, vol. 44, no. 8, pp. 2282-2291, Aug. 2006.- [7] V. Barnett and T. Lewis,
Outliers in Statistical Data. John Wiley & Sons, 1994.- [8] M.M. Breunig, H.-P. Kriegel, R.T. Ng, and J. Sander, “LOF: Identifying Density-Based Local Outliers,”
Proc. ACM SIGMOD '00, pp. 93-104, 2000.- [9] C. Campbell and K.P. Bennett, “A Linear Programming Approach to Novelty Detection,”
Advances in Neural Information Processing Systems 13, pp. 395-401, 2001.- [10] M.J. Carlotto, “A Cluster-Based Approach for Detecting Man-Made Objects and Changes in Imagery,”
IEEE Trans. Geoscience and Remote Sensing, vol. 43, no. 2, pp. 374-387, Feb. 2005.- [11] D. Castaño and A. Kunoth, “Robust Regression of Scattered Data with Adaptive Spline-Wavelets,”
IEEE Trans. Image Processing, vol. 15, no. 6, pp. 1621-1632, June 2006.- [12] P. Chaudhuri, “Multivariate Location Estimation Using Extension of $R\hbox{-}{\rm Estimates}$ through $U\hbox{-}{\rm Statistics}$ Type Approach,”
The Annals of Statistics, vol. 20, no. 2, pp. 897-916, 1992.- [13] P. Chaudhuri, “On a Geometric Notion of Quantiles for Multivariate Data,”
J. Am. Statistical Assoc., vol. 91, no. 434, pp. 862-872, 1996.- [14] Y. Chen, H.L. Bart Jr., S. Huang, and H. Chen, “A Computational Framework for Taxonomic Research: Diagnosing Body Shape within Fish Species Complexes,”
Proc. Fifth IEEE Int'l Conf. Data Mining, pp. 593-596, 2005.- [15] C. Croux and P.J. Rousseeuw, “Alternatives to the Median Absolute Deviation,”
J. Am. Statistical Assoc., vol. 88, no. 424, pp.1273-1283, 1993.- [16] X. Dang and R. Serfling, “Nonparametric Depth-Based Multivariate Outlier Identifiers, and Robustness Properties,” submitted for journal publication, 2006.
- [17] G. Danuser and M. Stricker, “Parametric Model Fitting: From Inlier Characterization to Outlier Detection,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 263-280, Mar. 1998.- [18] E. Eskin, “Anomaly Detection over Noisy Data Using Learned Probability Distributions,”
Proc. 17th Int'l Conf. Machine Learning, pp. 255-262, 2000.- [19] S. Fidler, D. Skočaj, and A. Leonardis, “Combining Reconstructive and Discriminative Subspace Methods for Robust Classification and Regression by Subsampling,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 337-350, Mar. 2006.- [20] A.B. Frakt, W.C. Karl, and A.S. Willsky, “A Multiscale Hypothesis Testing Approach to Anomaly Detection and Localization from Noisy Tomographic Data,”
IEEE Trans. Image Processing, vol. 7, no. 6, pp. 825-837, June 1998.- [21] M.G. Genton, “Classes of Kernels for Machine Learning: A Statistics Perspective,”
J. Machine Learning Research, vol. 2, pp.299-312, 2001.- [22] A.K. Ghosh and P. Chaudhuri, “On Data Depth and Distribution-Free Discriminant Analysis Using Separating Surfaces,”
Bernoulli, vol. 11, no. 1, pp. 1-27, 2005.- [23] A.K. Ghosh and P. Chaudhuri, “On Maximum Depth Classifiers,”
Scandinavian J. Statistics, vol. 32, no. 2, pp. 327-350, 2005.- [24] R.C. Gonzalez and R.E. Woods,
Digital Image Processing, third ed. Addison-Wesley, 1992.- [25] J.C. Gower, “Generalized Procrustes Analysis,”
Psychometrika, vol. 40, pp. 33-51, 1975.- [26] H. Hajji, “Statistical Analysis of Network Traffic for Adaptive Faults Detection,”
IEEE Trans. Neural Networks, vol. 16, no. 5, pp.1053-1063, Sept. 2005.- [27] S.-J. Han and S.-B. Cho, “Evolutionary Neural Networks for Anomaly Detection Based on the Behavior of a Program,”
IEEE Trans. Systems, Man, and Cybernetics B, vol. 36, no. 3, pp. 559-570, June 2006.- [28] D.M. Hawkins,
Identification of Outliers. Chapman and Hall, 1980.- [29] W. Hu, X. Xiao, Z. Fu, D. Xie, T. Tan, and S. Maybank, “A System for Learning Statistical Motion Patterns,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1450-1464, Sept. 2006.- [30] J. Hugg, E. Rafalin, K. Seyboth, and D. Souvaine, “An Experimental Study of Old and New Depth Measures,”
Proc. Workshop Algorithm Eng. and Experiments (ALENEX '06), pp. 51-64, 2006.- [31] R. Jörnsten, “Clustering and Classification Based on the $L_{1}$ Data Depth,”
J. Multivariate Analysis, vol. 90, no. 1, pp. 67-89, 2004.- [32] D.G. Kendall, “Shape-Manifolds, Procrustean Metrics and Complex Projective Spaces,”
Bull. London Math. Soc., vol. 16, no. 2 pp.81-121, 1984.- [33] E. Keogh, J. Lin, A.W. Fu, and H. van Herle, “Finding Unusual Medical Time-Series Subsequences: Algorithms and Applications,”
IEEE Trans. Information Technology in Biomedicine, vol. 10, no. 3, pp. 429-439, Mar. 2006.- [34] E.M. Knorr and R.T. Ng, “Algorithms for Mining Distance-Based Outliers in Large Datasets,”
Proc. 24th Int'l Conf. Very Large Data Bases, pp. 392-403, 1998.- [35] G. Kollios, D. Gunopulos, N. Koudas, and S. Berchtold, “Efficient Biased Sampling for Approximate Clustering and Outlier Detection in Large Data Sets,”
IEEE Trans. Knowledge and Data Eng., vol. 15, no. 5, pp. 1170-1187, Sept./Oct. 2003.- [36] H. Kwon and N.M. Nasrabadi, “Kernel RX-Algorithm: A Nonlinear Anomaly Detector for Hyperspectral Imagery,”
IEEE Trans. Geoscience and Remote Sensing, vol. 43, no. 2, pp. 388-397, Feb 2005.- [37] J. Langford, “Tutorial on Practical Prediction Theory for Classification,”
J. Machine Learning Research, vol. 6, pp. 273-306, 2005.- [38] A. Lazarevic and V. Kumar, “Feature Bagging for Outlier Detection,”
Proc. ACM SIGKDD '05, pp. 157-166, 2005.- [39] S. Lele and J.T. Richtsmeier, “Euclidean Distance Matrix Analysis: A Coordinate Free Approach for Comparing Biological Shapes Using Landmark Data,”
Am. J. Physical Anthropology, vol. 86, pp.415-427, 1991.- [40] A. Leonardis and H. Bischof, “Robust Recognition Using Eigenimages,”
Computer Vision and Image Understanding, vol. 78, no. 1, pp. 99-118, 2000.- [41] R.Y. Liu, “On a Notion of Data Depth Based on Random Simplices,”
The Annals of Statistics, vol. 18, no. 1, pp. 405-414, 1990.- [42] C. Manikopoulos and S. Papavassiliou, “Network Intrusion and Fault Detection: A Statistical Anomaly Approach,”
IEEE Comm. Magazine, vol. 40, no. 10, pp. 76-83, 2002.- [43] M. Markou and S. Singh, “Novelty Detection: A Review-Part 1: Statistical Approaches,”
Signal Processing, vol. 83, no. 12, pp.2481-2497, 2003.- [44] M. Markou and S. Singh, “Novelty Detection: A Review-Part 2: Neural Network Based Approaches,”
Signal Processing, vol. 83, no. 12, pp. 2499-2521, 2003.- [45] M. Markou and S. Singh, “A Neural Network-Based Novelty Detection for Image Sequence Analysis,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1664-1677, Oct. 2006.- [46] D.J. Miller and J. Browning, “A Mixture Model and EM-Based Algorithm for Class Discovery, Robust Classification, and Outlier Rejection in Mixed Labeled/Unlabeled Data Sets,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 11, pp. 1468-1483, Nov. 2003.- [47] L. Parra, G. Deco, and S. Miesbach, “Statistical Independence and Novelty Detection with Information Preserving Non-Linear Maps,”
Neural Computation, vol. 8, no. 2, pp. 260-269, 1996.- [48] F. Preparata and M. Shamos,
Computational Geometry: An Introduction. Springer, 1988.- [49] G. Rätsch, S. Mika, B. Schölkopf, and K.-R. Müller, “Constructing Boosting Algorithms from SVMs: An Application to One-Class Classification,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1184-1199, Sept. 2002.- [50] S. Ramaswamy, R. Rastogi, and S. Kyuseok, “Efficient Algorithms for Mining Outliers from Large Data Sets,”
Proc. ACM SIGMOD '00, pp. 427-438, 2000.- [51] I.S. Reed and X. Yu, “Adaptive Multiple-Band CFAR Detection of an Optical Pattern with Unknown Spectral Distribution,”
IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 38, no. 10, pp.1760-1770, Oct. 1990.- [52] B.D. Ripley,
Pattern Recognition and Neural Networks. Cambridge Univ. Press, 1996.- [53] S. Roberts and L. Tarassenko, “A Probabilistic Resource Allocating Network for Novelty Detection,”
Neural Computation, vol. 6, no. 2, pp. 270-284, 1994.- [54] D.M. Rocke and D.L. Woodruff, “Identification of Outliers in Multivariate Data,”
J. Am. Statistical Assoc., vol. 91, no. 435, pp.1047-1061, 1996.- [55] P.J. Rousseeuw and K. van Driessen, “A Fast Algorithm for the Minimum Covariance Determinant Estimator,”
Technometrics, vol. 41, no. 3, pp. 212-223, 1999.- [56] P.J. Rousseeuw and A.M. Leroy,
Robust Regression and Outlier Detection. John Wiley & Sons, 1987.- [57] P.J. Rousseeuw and I. Ruts, “Algorithm AS 307: Bivariate Location Depth,”
Applied Statistics, vol. 45, no. 4, pp. 516-526, 1996.- [58] I. Ruts and P. Rousseeuw, “Computing Depth Contours of Bivariate Point Clouds,”
Computational Statistics and Data Analysis, vol. 23, no. 1, pp. 153-168, 1996.- [59] B. Schölkopf, J.C. Platt, J. Shawe-Taylor, A.J. Smola, and R.C. Williamson, “Estimating the Support of a High-Dimensional Distribution,”
Neural Computation, vol. 13, no. 7, pp. 1443-1471, 2001.- [60] S.M. Schweizer and J.M.F. Moura, “Hyperspectral Imagery: Clutter Adaptation in Anomaly Detection,”
IEEE Trans. Information Theory, vol. 46, no. 5, pp. 1855-1871, Aug. 2000.- [61] R. Serfling, “A Depth Function and a Scale Curve Based on Spatial Quantiles,”
Statistical Data Analysis Based on the L1-Norm and Related Methods, Y. Dodge, ed., pp. 25-38, 2002.- [62] J. Shawe-Taylor and N. Cristianini,
Kernel Methods for Pattern Analysis. Cambridge Univ. Press, 2004.- [63] D.E. Slice, “Landmark Coordinates Aligned by Procrustes Analysis Do Not Lie in Kendall's Shape Space,”
Systematic Biology, vol. 50, pp. 141-149, 2001.- [64] C.G. Small, “A Survey of Multidimensional Medians,”
Int'l Statistical Rev., vol. 58, no. 3, pp. 263-277, 1990.- [65] D. Song, M.I. Heywood, and A.N. Zincir-Heywood, “Training Genetic Programming on Half a Million Patterns: An Example from Anomaly Detection,”
IEEE Trans. Evolutionary Computation, vol. 9, no. 3, pp. 225-239, June 2005.- [66] C. Stauffer and W.E. Grimson, “Learning Patterns of Activity Using Real-Time Tracking,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 747-757, Aug. 2000.- [67] I. Steinwart, D. Hush, and C. Scovel, “A Classification Framework for Anomaly Detection,”
J. Machine Learning Research, vol. 6, pp.211-232, 2005.- [68] P. Sun and S. Chawla, “On Local Spatial Outliers,”
Proc. Fourth IEEE Int'l Conf. Data Mining, pp. 209-216, 2004.- [69] J. Takeuchi and K. Yamanishi, “A Unifying Framework for Detecting Outliers and Change Points from Time Series,”
IEEE Trans. Knowledge and Data Eng., vol. 18, no. 4, pp. 482-492, Apr. 2006.- [70] J. Tang, Z. Chen, A.W.-C. Fu, and D. Cheung, “A Robust Outlier Detection Scheme in Large Data Sets,”
Proc. Pacific-Asia Conf. Knowledge Discovery and Data Mining, pp. 535-548, 2002.- [71] M. Thottan and C. Ji, “Anomaly Detection in IP Networks,”
IEEE Trans. Signal Processing, vol. 51, no. 8, pp. 2191-2204, Aug. 2003.- [72] J.W. Tukey, “Mathematics and Picturing Data,”
Proc. Int'l Congress of Math., vol. 2, pp. 523-531, 1974.- [73] V. Vapnik,
The Nature of Statistical Learning Theory. Springer, 1995.- [74] Y. Vardi and C.-H. Zhang, “The Multivariate $L_{1}\hbox{-}{\rm Median}$ and Associated Data Depth,”
Proc. Nat'l Academy of Sciences, vol. 97, no. 4, pp. 1423-1436, 2000.- [75] A. Weber,
Theory of the Location of Industries (translated by C.J.Friedrich from Weber's 1909 book). Univ. of Chicago Press, 1929.- [76] L. Wei, E. Keogh, and X. Xi, “SAXually Explicit Images: Finding Unusual Shapes,”
Proc. IEEE Int'l Conf. Data Mining, pp. 711-720, 2006.- [77] W. Zhou and R. Serfling, “Multivariate Spatial U-Quantiles: ABahadur-Kiefer Representation, a Theil-Sen Estimator for Multiple Regression, and a Robust Dispersion Estimator,” manuscript, http://www.utdallas.edu/~serfling/papers Zhou Serfling2006.pdf, 2006.
- [78] Y. Zuo and R. Serfling, “General Notions of Statistical Depth Function,”
The Annals of Statistics, vol. 28, no. 2, pp. 461-482, 2000. |