The Community for Technology Leaders
Green Image
Issue No. 11 - Nov. (2012 vol. 24)
ISSN: 1041-4347
pp: 1921-1936
A. Nur Zincir-Heywood , Dalhousie University, Nova Scotia
Adetokunbo Makanju , Dalhousie University, Nova Scotia
Evangelos E. Milios , Dalhousie University, Nova Scotia
ABSTRACT
Message type or message cluster extraction is an important task in the analysis of system logs in computer networks. Defining these message types automatically facilitates the automatic analysis of system logs. When the message types that exist in a log file are represented explicitly, they can form the basis for carrying out other automatic application log analysis tasks. In this paper, we introduce a novel algorithm for carrying out message type extraction from event log files. IPLoM, which stands for Iterative Partitioning Log Mining, works through a 4-step process. The first three steps hierarchically partition the event log into groups of event log messages or event clusters. In its fourth and final stage, IPLoM produces a message type description or line format for each of the message clusters. IPLoM is able to find clusters in data irrespective of the frequency of its instances in the data, it scales gracefully in the case of long message type patterns and produces message type descriptions at a level of abstraction, which is preferred by a human observer. Evaluations show that IPLoM outperforms similar algorithms statistically significantly.
INDEX TERMS
Kernel, Data mining, Humans, Clustering algorithms, Buildings, Observers, Partitioning algorithms, clustering, Algorithms, experimentation, event log mining, fault management
CITATION
A. Nur Zincir-Heywood, Adetokunbo Makanju, Evangelos E. Milios, "A Lightweight Algorithm for Message Type Extraction in System Application Logs", IEEE Transactions on Knowledge & Data Engineering, vol. 24, no. , pp. 1921-1936, Nov. 2012, doi:10.1109/TKDE.2011.138
96 ms
(Ver 3.1 (10032016))