Subscribe

Issue No.04 - July/August (2011 vol.8)

pp: 510-522

Miao Jiang , University of Waterloo, Waterloo

Mohammad A. Munawar , University of Waterloo, Waterloo

Thomas Reidemeister , University of Waterloo, Waterloo

Paul A.S. Ward , University of Waterloo, Waterloo

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TDSC.2011.16

ABSTRACT

Management metrics of complex software systems exhibit stable correlations which can enable fault detection and diagnosis. Current approaches use specific analytic forms, typically linear, for modeling correlations. In practice, more complex nonlinear relationships exist between metrics. Moreover, most intermetric correlations form clusters rather than simple pairwise correlations. These clusters provide additional information and offer the possibility for optimization. In this paper, we address these issues by using Normalized Mutual Information (NMI) as a similarity measure to identify clusters of correlated metrics, without assuming any specific form for the metric relationships. We show how to apply the Wilcoxon Rank-Sum test on the entropy measures to detect errors in the system. We also present three diagnosis algorithms to locate faulty components: RatioScore, based on the Jaccard coefficient, SigScore, which incorporates knowledge of component dependencies, and BayesianScore, which uses Bayesian inference to assign a fault probability to each component. We evaluate our approach in the context of a complex enterprise application, and show that 1) stable, nonlinear correlations exist and can be captured with our approach; 2) we can detect a large fraction of faults with a low false positive rate (we detect up to 18 of the 22 faults we injected); and 3) we improve the diagnosis with our new diagnosis algorithms.

INDEX TERMS

Self-managing systems, fault detection, fault diagnosis, information theory, mutual information, autonomic systems.

CITATION

Miao Jiang, Mohammad A. Munawar, Thomas Reidemeister, Paul A.S. Ward, "Efficient Fault Detection and Diagnosis in Complex Software Systems with Information-Theoretic Monitoring",

*IEEE Transactions on Dependable and Secure Computing*, vol.8, no. 4, pp. 510-522, July/August 2011, doi:10.1109/TDSC.2011.16REFERENCES

- [1] J.O. Kephart and D.M. Chess, “The Vision of Autonomic Computing,”
Computer, vol. 36, no. 1, pp. 41-50, Jan. 2003.- [2] Z. Guo, G. Jiang, H. Chen, and K. Yoshihira, “Tracking Probabilistic Correlation of Monitoring Data for Fault Detection in Complex Systems,”
Proc. Int'l Conf. Dependable Systems and Networks (DSN '06), pp. 259-268, 2006.- [3] G. Jiang, H. Chen, and K. Yoshihira, “Modelling and Tracking of Transaction Flow Dynamics for Fault Detection in Complex Systems,”
IEEE Trans. Dependable and Secure Computing, vol. 3, no. 4, pp. 312-326, Oct.-Dec. 2006.- [4] M.A. Munawar and P.A. Ward, “Adaptive Monitoring in Enterprise Software Systems,”
Proc. Workshop Systems Modeling Language (SysML), June 2006.- [5] M.A. Munawar and P.A. Ward, “A Comparative Study of Pairwise Regression Techniques for Problem Determination,”
Proc. Conf. Center for Advanced Studies on Collaborative Research (CASCON '07), pp. 152-166, 2007.- [6] M.A. Munawar and P.A. Ward, “Leveraging Many Simple Statistical Models to Adaptively Monitor Software Systems,”
Proc. Int'l Symp. Parallel and Distributed Processing and Applications (ISPA), 2007.- [7] A. Agogino and K. Tumer, “Entropy Based Anomaly Detection Applied to Space Shuttle Main Engines,”
Proc. IEEE Aerospace Conf., Mar. 2006.- [8] I.H. Witten and E. Frank,
Data Mining: Practical Machine Learning Tools and Techniques, second ed. Morgan Kaufmann, 2005.- [9] M.Y. Chen, E. Kiciman, E. Fratkin, A. Fox, and E.A. Brewer, “Pinpoint: Problem Determination in Large, Dynamic Internet Services.”
Proc. Int'l Conf. Dependable Systems and Networks (DSN), pp. 595-604, 2002.- [10] I. Cohen, S. Zhang, M. Goldszmidt, J. Symons, T. Kelly, and A. Fox, “Capturing, Indexing, Clustering, and Retrieving System History,”
Proc. Symp. Operating Systems Principles (SOSP), pp. 105-118, 2005.- [11] S. Ghanbari and C. Amza, “Semantic-Driven Model Composition for Accurate Anomaly Diagnosis,”
Proc. Int'l Conf. Autonomic Computing (ICAC), 2008.- [12] M. Jiang, M.A. Munawar, T. Reidemeister, and P.A.S. Ward, “Detection and Diagnosis of Recurrent Faults in Software Systems by Invariant Analysis,”
Proc. 11th IEEE High Assurance Systems Eng. Symp. (HASE), 2008.- [13] J. Han and M. Kamber,
Google's Pagerank and Beyond: The Science of Search Engine Rankings. Princeton University Press, 2006.- [14] K. Inoue, R. Yokomori, H. Rujiwara, T. Yamamoto, M. Matsushita, and S. Kusumoto, “Component Rank: Relative Significance Rank for Software Component Search,”
Proc. 25th Int'l Conf. Software Eng. (ICSE), 2003.- [15] C.E. Shannon, “A Mathematical Theory of Communication,”
Key Papers in the Development of Information Theory, http://cm.bell-labs. com/cm/ms/what/shannonday shannon1948.pdf, 1948.- [16] A. Strehl and J. Ghosh, “Cluster Ensembles—a Knowledge Reuse Framework for Combining Multiple Partitions,”
J. Machine Learning Research, vol. 3, pp. 583-617, http://citeseer.ist.psu. edu/articlestrehl02cluster.html , Dec. 2002.- [17] J. Han and M. Kamber,
Data Mining: Concepts and Techniques, second ed. Morgan Kaufmann, 2006.- [18] D.M. Levine, C.P.P. Ramsey, and R.K. Smidt,
Applied Statistics for Engineers and Scientists. Prentice Hall, 2000.- [19] IBM Corporation, iBM Trade Performance Benchmark, http://www-01.ibm.com/software/webservers/ appserv/was performance, 2011.
- [20] R. Chillarege, I.S. Bhandari, J.K. Chaar, M.J. Halliday, D.S. Moebus, B.K. Ray, and M.-Y. Wong, “Orthogonal Defect Classification—A Concept for in—Process Measurements,”
IEEE Trans. Software Eng., vol. 18, no. 11, pp. 943-956, Nov. 1992. |