This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Resource-Aware Application State Monitoring
Dec. 2012 (vol. 23 no. 12)
pp. 2315-2329
Shicong Meng, Georgia Institute of Technology, Atlanta
Chitra Venkatramani, IBM T.J. Watson Research Center, Hawthorne
Ling Liu, Georgia Institute of Technology, Atlanta
The increasing popularity of large-scale distributed applications in datacenters has led to the growing demand of distributed application state monitoring. These application state monitoring tasks often involve collecting values of various status attributes from a large number of nodes. One challenge in such large-scale application state monitoring is to organize nodes into a monitoring overlay that achieves monitoring scalability and cost effectiveness at the same time. In this paper, we present REMO, a REsource-aware application state MOnitoring system, to address the challenge of monitoring overlay construction. REMO distinguishes itself from existing works in several key aspects. First, it jointly considers intertask cost-sharing opportunities and node-level resource constraints. Furthermore, it explicitly models the per-message processing overhead which can be substantial but is often ignored by previous works. Second, REMO produces a forest of optimized monitoring trees through iterations of two phases. One phase explores cost-sharing opportunities between tasks, and the other refines the tree with resource-sensitive construction schemes. Finally, REMO also employs an adaptive algorithm that balances the benefits and costs of overlay adaptation. This is particularly useful for large systems with constantly changing monitoring tasks. Moreover, we enhance REMO in terms of both performance and applicability with a series of optimization and extension techniques. We perform extensive experiments including deploying REMO on a BlueGene/P rack running IBM's large-scale distributed streaming system—System S. Using REMO in the context of collecting over 200 monitoring tasks for an application deployed across 200 nodes results in a 35-45 percent decrease in the percentage error of collected attributes compared to existing schemes.
Index Terms:
Monitoring,Resource management,Vegetation,Database systems,Optimization,Scalability,data-intensive,Resource-aware,state monitoring,distributed monitoring,datacenter monitoring,adaptation
Citation:
Shicong Meng, Srinivas Raghav Kashyap, Chitra Venkatramani, Ling Liu, "Resource-Aware Application State Monitoring," IEEE Transactions on Parallel and Distributed Systems, vol. 23, no. 12, pp. 2315-2329, Dec. 2012, doi:10.1109/TPDS.2012.82
Usage of this product signifies your acceptance of the Terms of Use.