Issue No. 06 - June (2009 vol. 20)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPDS.2009.13
Abhishek Chandra , University of Minnesota, Minneapolis
Jon B. Weissman , University of Minnesota, Minneapolis
Jinoh Kim , University of Minnesota, Minneapolis
Large-scale distributed systems provide an attractive scalable infrastructure for network applications. However, the loosely coupled nature of this environment can make data access unpredictable, and in the limit, unavailable. We introduce the notion of accessibility to capture both availability and performance. An increasing number of data-intensive applications require not only considerations of node computation power but also accessibility for adequate job allocations. For instance, selecting a node with intolerably slow connections can offset any benefit to running on a fast node. In this paper, we present accessibility-aware resource selection techniques by which it is possible to choose nodes that will have efficient data access to remote data sources. We show that the local data access observations collected from a node's neighbors are sufficient to characterize accessibility for that node. By conducting trace-based, synthetic experiments on PlanetLab, we show that the resource selection heuristics guided by this principle significantly outperform conventional techniques such as latency-based or random allocations. The suggested techniques are also shown to be stable even under churn despite the loss of prior observations.
Data Accessibility, resource selection, passive network performance estimation, data-intensive computing, large-scale distributed systems.
Abhishek Chandra, Jon B. Weissman, Jinoh Kim, "Using Data Accessibility for Resource Selection in Large-Scale Distributed Systems", IEEE Transactions on Parallel & Distributed Systems, vol. 20, no. , pp. 788-801, June 2009, doi:10.1109/TPDS.2009.13