Subscribe
Issue No.12 - Dec. (2013 vol.35)
pp: 2997-3009
David Martinez-Rego , Dept. of Comput. Sci., Univ. of A Coruna, A Coruña, Spain
Enrique Castillo , Dept. of Appl. Math. & Comput. Sci., Univ. of Cantabria, Santander, Spain
Oscar Fontenla-Romero , Dept. of Comput. Sci., Univ. of A Coruna, A Coruña, Spain
Amparo Alonso-Betanzos , Dept. of Comput. Sci., Univ. of A Coruna, A Coruña, Spain
ABSTRACT
A technique for adjusting a minimum volume set of covering ellipsoids technique is elaborated. Solutions to this problem have potential application in one-class classification and clustering problems. Its main original features are: 1) It avoids the direct evaluation of determinants by using diagonalization properties of the involved matrices, 2) it identifies and removes outliers from the estimation process, 3) it avoids binary variables resulting from the combinatorial character of the assignment problem that are replaced by continuous variables in the range [0, 1], 4) the problem can be solved by a bilevel algorithm that in its first level determines the ellipsoids and in its second level reassigns the data points to ellipsoids and identifies outliers based on an algorithm that forces the Karush-Kuhn-Tucker conditions to be satisfied. Two theorems provide rigorous bases for the proposed methods. Finally, a set of examples of application in different fields is given to illustrate the power of the method and its practical performance.
INDEX TERMS
Ellipsoids, Volume measurements, Cluster approximation, Data models, Classification,minimum volume covering ellipsoids, One class classification, data clustering, bilevel algorithm
CITATION
David Martinez-Rego, Enrique Castillo, Oscar Fontenla-Romero, Amparo Alonso-Betanzos, "A Minimum Volume Covering Approach with a Set of Ellipsoids", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 12, pp. 2997-3009, Dec. 2013, doi:10.1109/TPAMI.2013.94
REFERENCES
 [1] F. John, "Extreme Problems with Inequalities as Subsidiary Conditions," Studies and Essays Presented to R. Courant on his 60th Birthday, Wiley Interscience, 1948. [2] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004. [3] P. Sun and R.M. Freund, "Computation of Minimum-Volume Covering Ellipsoids," Operations Research, vol. 52, no. 5, pp. 690-706, 2004. [4] E. Barnes, "An Algorithm for Separating Patterns by Ellipsoids," IBM J. Research and Development, vol. 26, pp. 759-764, 1982. [5] L. Khachiyan, "Rounding of Polytopes in the Real Number Model of Computation," Math. Operations Research, vol. 2, pp. 307-320, 1996. [6] P. Kumar and E.A. Yildirim, "Minimum-Volume Enclosing Ellipsoids and Core Sets," J. Optimization Theory and Applications, vol. 126, no. 1, pp. 1-21, 2005. [7] M.J. Todd and E.A. Yildirim, "On Khachiyan's Algorithm for the Computation of Minimum-Volume Enclosing Ellipsoids," Discrete Applied Math., vol. 155, no. 13, pp. 1731-1744, 2007. [8] L. Khachiyan and M. Todd, "On the Complexity of Approximating the Maximal Inscribed Ellipsoid for a Polytope," Math. Programming, vol. 61, pp. 137-159, 1993. [9] Y. Nesterov and A. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming. SIAM, 1994. [10] S. Boyd, L.E. Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory. SIAM, 1994. [11] C. Wang, D. Sun, and K.C. Toh, "Solving Log-Determinant Optimization Problems by a Newton-CG Primal Proximal Point Algorithm," SIAM J. Optimization, vol. 20, pp. 2994-3013, 2010. [12] D. Martinus and J. Tax, "One-Class Classification: Concept Learning in the Absence of Counter-Examples," PhD dissertation, Technische Universiteit Delft, 2001. [13] P. Hayton, B. Scholkopf, L. Tarassenko, and P. Anuzis, "Support Vector Novelty Detection Applied to Jet Engine Vibration Spectra," Proc. Neural Information Processing Systems Conf., pp. 946-952, 2000. [14] D. Martinez-Rego, O. Fontenla-Romero, and A. Alonso-Betanzos, "Power Wind Mill Fault Detection via One-Class $\nu$ -SVM Vibration Signal Analysis," Proc. Int'l Joint Conf. Neural Networks, 2011. [15] N. Japkowicz, "Concept Learning in the Absence of Counter-Examples: An Autoassociation-Based Approach to Classification," PhD dissertation, Rutgers, The State Univ. of New Jersey, 1999. [16] V. Chandola, A. Banerjee, and V. Kumar, "Anomaly Detection: A Survey," ACM Computing Surveys, vol. 41, no. 3, 2009. [17] V. Chandola, A. Banerjee, and V. Kumar, "Supervised versus Unsupervised Binary-Learning by Feedforward Neural Networks," Machine Learning, vol. 42, nos. 1/2, pp. 97-122, 2001. [18] V. Barnett and T. Lewis, Outliers in Statistical Data. Wiley, Apr. 1994. [19] R. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. Wiley-Interscience, Nov. 2001. [20] E. Parzen, "On Estimation of a Probability Density Function and Mode," The Annals of Math. Statistics, vol. 33, no. 3, pp. 1065-1076, 1962. [21] A. Ypma, D. Tax, and R. Duin, "Robust Machine Fault Detection with Independent Component Analysis and Support Vector Data Description," Proc. IEEE Workshop Neural Networks for Signal Processing, pp. 67-76, 1999. [22] D.M.J. Tax and R. Duin, "Support Vector Data Description," Machine Learning, vol. 54, no. 1, pp. 45-66, 2004. [23] B. Schölkopf, J. Platt, J. Shawe-Taylor, A. Smola, and R. Williamson, "Estimating the Support of a High-Dimensional Distribution," Neural Computation, vol. 13, p. 2001, 1999. [24] E. Dolia, T.D. Bie, C. Harris, J. Shawe-taylor, and D.M. Titterington, "The Minimum Volume Covering Ellipsoid Estimation in Kernel-Defined Feature Spaces," Machine Learning, pp. 630-637, 2006. [25] C. Bishop, Neural Networks for Pattern Recognition, first ed. Oxford Univ. Press, Jan. 1996. [26] P. Somervuo and T. Kohonen, "Self-Organizing Maps and Learning Vector Quantization for Feature Sequences," Neural Processing Letters, vol. 10, no. 2, pp. 151-159, 1999. [27] P.K. Shivaswamy and T. Jebara, "Ellipsoidal Kernel Machines," Proc. 11th Int'l Conf. Artificial Intelligence and Statistics, 2007. [28] M. Turk and A. Pentland, "Eigenfaces for Recognition," J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991. [29] H. Bourlard and Y. Kamp, "Auto-Association by Multilayer Perceptrons and Singular Value Decomposition," Biological Cybernetics, vol. 59, no. 4, pp. 291-294, Sept. 1988. [30] C. Bishop, Pattern Recognition and Machine Learning, first ed. Springer, Oct. 2007. [31] H. Hoffmann, "Kernel PCA for Novelty Detection," Pattern Recognition, vol. 40, no. 3, pp. 863-874, 2006. [32] M. Kramer, "Nonlinear Principal Component Analysis Using Autoassociative Neural Networks," AIChE J., vol. 37, no. 2, pp. 233-243, 1991. [33] R. Xu and D.C. Wunsch, Clustering. Wiley-IEEE Press, 2008. [34] S.P. Lloyd, "Least Squares Quantization in PCM," IEEE Trans. Information Theory, vol. 28, no. 2, pp. 129-137, Mar. 1982. [35] D.E. Gustafson and W. Kessel, "Fuzzy Clustering with a Fuzzy Covariance Matrix," Chemometrics and Intelligent Laboratory Systems, vol. 86, no. 2, pp. 761-766, 1979. [36] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification. John Wiley & Sons, 2001. [37] T. Hastie, R. Tibshirani, and J. Friedman, Hierarchical Clustering, in the Elements of Statistical Learning. Springer, 2009. [38] S. Chiu, "Fuzzy Model Identification Based on Cluster Estimation," J. Intelligent and Fuzzy Systems, vol. 2, pp. 267-278, 1994. [39] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, "A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise," Proc. Second Int'l Conf. Knowledge Discovery and Data Mining, pp. 6226-231, 1996. [40] A. Hinneburg and H. Gabriel, "Denclue 2.0: Fast Clustering Based on Kernel Density Estimation," Proc. Seventh Int'l Symp. Intelligent Data Analysis, pp. 70-80, 2007. [41] M. Vrahatis, B. Boutsinas, P. Alevizos, and G. Pavlides, "The New K-Windows Algorithm for Improving the K-Means Clustering Algorithm," J. Complexity, vol. 18, pp. 375-391, 2002. [42] M. Moshtaghi, S. Rajasegarar, C. Leckie, and S. Karunasekera, "An Efficient Hyperellipsoidal Clustering Algorithm for Resource-Constrained Environments," Pattern Recognition, vol. 44, pp. 2197-2209, 2011. [43] A. Conejo, E. Castillo, R. Mínguez, and R. García-Bertrand, Decomposition Techniques in Mathematical Programming: Engineering and Science Applications. Springer, 2005. [44] D. Harville, Matrix Algebra from a Statistician's Perspective, first ed. Springer, 1997. [45] E. Castillo, A. Conejo, A. Hadi, and A. Fernández-Canteli, "A General Method for Local Sensitivity Analysis with Application to Regression Models and Other Optimization Problems," Technometrics, vol. 46, no. 4, pp. 430-444, 2004. [46] E. Castillo, A. Conejo, R. Mínguez, and C. Castillo, "A Closed Formula for Local Sensitivity Analysis in Mathematical Programming," Eng. Optimization, vol. 38, no. 1, pp. 93-112, 2006. [47] E. Castillo, A. Conejo, C. Castillo, and R. Mínguez, "Closed Formulas in Local Sensitivity Analysis for Some Classes of Linear and Non-Linear Problems," TOP, vol. 15, no. 2, pp. 355-371, 2007. [48] E. Castillo, A. Conejo, C. Castillo, R. Mínguez, and D. Ortigosa, "Perturbation Approach to Sensitivity Analysis in Mathematical Programming," J. Optimization Theory and Applications, vol. 128, no. 1, pp. 49-74, 2006. [49] E. Castillo, A. Conejo, P. Pedregal, R. García, and N. Alguacil, Building and Solving Mathematical Programming Models in Engineering and Science. John Wiley & Sons Inc., 2001. [50] J.I. Taylor, The Vibration Analysis Handbook. VCI, 2003. [51] P. McFadden and J. Smith, "Model for the Vibration Produced by a Single Point Defect in a Rolling Element Bearing," J. Sound and Vibration, vol. 1, pp. 69-82, 1984. [52] J. Lee, H. Qiu, G. Yu, and J. Lin, "Bearing Data Set, IMS Center, University of Cincinnati," NASA Ames Prognostics Data Repository, vol. 1, pp. 69-82, http://ti.arc.nasa.gov/projectprognostic-data-repository , 2007. [53] P. Fränti and O. Virmajoki, "Iterative Shrinking Method for Clustering Problems," Pattern Recognition, vol. 39, pp. 761-775, May 2006. [54] R. Burkard, M. Dell'Amico, and S. Martello, Assignment Problems. SIAM, 2009.