21st International Conference on Data Engineering (ICDE'05)
Network-Based Problem Detection for Distributed Systems
Tokyo, Japan
April 05-April 08
ISBN: 0-7695-2285-8
We introduce a network-based problem detection framework for distributed systems, which includes a data-mining method for discovering dynamic dependencies among distributed services from transaction data collected from network, and a novel problem detection method based on the discovered dependencies. From observed containments of transaction execution time periods, we estimate the probabilities of accidental and non-accidental containments, and build a competitive model for discovering direct dependencies by using a model estimation method based on the online EM algorithm. Utilizing the discovered dependency information, we also propose a hierarchical problem detection framework, where microscopic dependency information is incorporated with a macroscopic anomaly metric that monitors the behavior of the system as a whole. This feature is made possible by employing a network-based design which provides overall information of the system without any impact on the performance.
Citation:
Hisashi Kashima, Tadashi Tsumura, Tsuyoshi Idé, Takahide Nogayama, Ryo Hirade, Hiroaki Etoh, Takeshi Fukuda, "Network-Based Problem Detection for Distributed Systems," icde, pp.978-989, 21st International Conference on Data Engineering (ICDE'05), 2005