This Article 
 Bibliographic References 
 Add to: 
The Globus Replica Location Service: Design and Experience
September 2009 (vol. 20 no. 9)
pp. 1260-1272
Ann L. Chervenak, University of Southern California, Marina del Rey
Robert Schuler, University of Southern California, Marina del Rey
Matei Ripeanu, University of British Columbia, Vancouver
Muhammad Ali Amer, University of Southern California, Marina del Rey
Shishir Bharathi, University of Southern California, Marina del Rey
Ian Foster, University of Chicago, Chicago
Adriana Iamnitchi, University of South Florida, Tampa
Carl Kesselman, University of Southern California, Marina del Rey
Distributed computing systems employ replication to improve overall system robustness, scalability, and performance. A Replica Location Service (RLS) offers a mechanism to maintain and provide information about physical locations of replicas. This paper defines a design framework for RLSs that supports a variety of deployment options. We describe the RLS implementation that is distributed with the Globus Toolkit and is in production use in several Grid deployments. Features of our modular implementation include the use of soft-state protocols to populate a distributed index and Bloom filter compression to reduce overheads for distribution of index information. Our performance evaluation demonstrates that the RLS implementation scales well for individual servers with millions of entries and up to 100 clients. We describe the characteristics of existing RLS deployments and discuss how RLS has been integrated with higher-level data management services.

[1] A.L. Chervenak, N. Palavalli, S. Bharathi, C. Kesselman, and R. Schwartzkopf, “Performance and Scalability of a Replica Location Service,” Proc. 13th IEEE Int'l Symp. High Performance Distributed Computing (HPDC), 2004.
[2] A. Chervenak, E. Deelman, I. Foster, L. Guy, W. Hoschek, A. Iamnitchi, C. Kesselman, P. Kunst, M. Ripeanu, B. Schwartzkopf, H. Stockinger, K. Stockinger, and B. Tierney, “Giggle: A Framework for Constructing Scalable Replica Location Services,” Proc. ACM/IEEE Conf. Supercomputing (SC), 2002.
[3] A. Abramovici, W. Althouse, R. Drever, Y. Gürsel, S. Kawamura, F. Raab, D. Shoemaker, L. Sievers, R. Spero, K. Thorne, R. Vogt, R. Weiss, S. Whitcomb, and M. Zucker, “LIGO: The Laser Interferometer Gravitational-Wave Observatory,” Science, vol. 256, pp.325-333, 1992.
[4] The Earth System Grid, ESG Project,, 2005.
[5] D. Bernholdt, S. Bharathi, D. Brown, K. Chancio, M. Chen, A. Chervenak, L. Cinquini, B. Drach, I. Foster, P. Fox, J. Garcia, C. Kesselman, R. Markel, D. Middleton, V. Nefedova, L. Pouchard, A. Shoshani, A. Sim, G. Strand, and D. Williams, “The Earth System Grid: Supporting the Next Generation of Climate Modeling Research,” Proc. IEEE, vol. 93, pp. 485-495, 2005.
[6] QCDGrid: Probing the Building Blocks of Matter with the Power of the Grid, QCDGrid Project,, 2005.
[7] Southern California Earthquake Center, SCEC Project, http:/, 2005.
[8] Lightweight Data Replicator, LIGO Project, http://www.lsc-group. phys.uwm.eduLDR/, 2004.
[9] E. Deelman, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, K. Blackburn, A. Lazzarini, A. Arbree, and R. Cavanaugh, “Mapping Abstract Complex Workflows onto Grid Environments,” J. Grid Computing, vol. 1, pp. 25-39, 2003.
[10] Java UUID Generator, Safehaus, http:/, 2008.
[11] B. Bloom, “Space/Time Trade-Offs in Hash Coding with Allowable Errors,” Comm. ACM, vol. 13, pp. 422-426, 1970.
[12] M. Cai, A. Chervenak, and M. Frank, “A Peer-to-Peer Replica Location Service Based on a Distributed Hash Table,” Proc. ACM/IEEE Conf. Supercomputing (SC), 2004.
[13] M. Ripeanu and I. Foster, “A Decentralized, Adaptive, Replica Location Mechanism,” Proc. 11th IEEE Int'l Symp. High Performance Distributed Computing (HPDC), 2002.
[14] G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels, “Dynamo: Amazon's Highly Available Key-Value Store,” Proc. 21st ACM SIGOPS Symp. Operating Systems Principles (SOSP), 2007.
[15] LIGO—Laser Interferometer Gravitational Wave Observatory, LIGO Project, http:/, 2004.
[16] L. Lamport, “Concurrent Reading and Writing,” Comm. ACM, vol. 20, pp. 806-811, 1977.
[17] K. Czajkowski et al., The WS-Resource Framework Version 1.0, ws-resourcews-wsrf.pdf, 2004.
[18] L. Fan, P. Cao, J. Almeida, and A.Z. Broder, “Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol,” IEEE/ACM Trans. Networking, vol. 8, pp. 281-293, 2000.
[19] M. Ripeanu, “Using Peer-to-Peer Experience to Build Large Scale Grid Services,” PhD dissertation, The Univ. of Chicago, 2005.
[20] W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, and I. Foster, “The Globus Striped GridFTP Framework and Server,” Proc. Supercomputing Conf. (SC '05), 2005.
[21] E. Deelman, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, S. Patil, M. Su, K. Vahi, and M. Livny, “Pegasus: Mapping Scientific Workflows onto the Grid,” Proc. Second European AcrossGrids Conf. (AxGrids), 2004.
[22] J. Frey, T. Tannenbaum, I. Foster, M. Livny, and S. Tuecke, “Condor-G: A Computation Management Agent for Multiinstitutional Grids,” Cluster Computing, vol. 5, pp. 237-246, 2002.
[23] G.B. Berriman et al., “Montage: A Grid-Enabled Image Mosaic Service for the NVO,” Proc. Astronomical Data Analysis Software and Systems (ADASS), 2003.
[24] E. Deelman et al., “Grid-Based Galaxy Morphology Analysis for the National Virtual Observatory,” Proc. ACM/IEEE Conf. Supercomputing (SC), 2003.
[25] I. Stoica, R. Morris, D. Karger, M.F. Kaashoek, and H. Balakrishnan, “Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications,” Proc. ACM SIGCOMM, 2001.
[26] A. Chervenak, R. Schuler, C. Kesselman, S. Koranda, and B. Moe, “Wide Area Data Replication for Scientific Collaborations,” Proc. Sixth IEEE/ACM Int'l Workshop Grid Computing (Grid), 2005.
[27] A. Rajasekar et al., “Storage Resource Broker—Managing Distributed Data in a Grid,” Computer Soc. of India J., special issue on SAN, vol. 33, pp. 42-54, 2003.
[28] O. Tatebe et al., “Worldwide Fast File Replication on Grid Datafarm,” Proc. Computing in High Energy and Nuclear Physics (CHEP), 2003.
[29] P. Kunszt, E. Laure, H. Stockinger, and K. Stockinger, “Advanced Replica Management with Reptor,” Proc. Fifth Int'l Conf. Parallel Processing and Applied Math. (PPAM), 2003.
[30] J.P. Baud, J. Casey, S. Lemaitre, and C. Nicholson, “Performance Analysis of a File Catalog for the LHC Computing Grid,” Proc. 14th IEEE Int'l Symp. High Performance Distributed Computing (HPDC), 2005.
[31] C. Munro and B. Koblitz, “Performance Comparison of the LCG2 and gLite File Catalogues,” Nuclear Instruments and Methods in Physics Research Section A, vol. 559, pp. 48-52, 2006.
[32] P.F. Kunszt, P. Badino, A. Frohner, G. McCance, K. Nienartowicz, R. Rocha, and D. Rodrigues, “Data Storage, Access and Catalogs in gLite,” Local to Global Data Interoperability—Challenges and Technologies, 2005.
[33] M. Ripeanu, “Peer-to-Peer Architecture Case Study: Gnutella Network,” Proc. IEEE First Int'l Conf. Peer-to-Peer Computing (P2P), 2001.
[34] Y. Chawathe, S. Ratnasamy, L. Breslau, N. Lanham, and S. Shenker, “Making Gnutella-Like P2P Systems Scalable,” Proc. ACM SIGCOMM, 2003.
[35] M. Ripeanu and I. Foster, “Mapping the Gnutella Network: Properties of Large-Scale Peer-to-Peer Systems and Implications for System Design,” IEEE Internet Computing J., vol. 6, pp. 50-57, 2002.
[36] S. Saroiu, P.K. Gummadi, and S.D. Gribble, “A Measurement Study of Peer-to-Peer File Sharing Systems,” Proc. Multimedia Computing and Networking (MMCN), 2002.
[37] S. Sen and J. Wong, “Analyzing Peer-to-Peer Traffic across Large Networks,” Proc. Second ACM SIGCOMM Internet Measurement Workshop (IMW), 2002.
[38] Q. Lv, P. Cao, E. Cohen, K. Li, and S. Shenker, “Search and Replication in Unstructured Peer-to-Peer Networks,” Proc. 16th ACM Int'l Conf. Supercomputing (ICS), 2002.
[39] S. Ratnasamy, S. Shenker, and I. Stoica, “Routing Algorithms for DHTs: Some Open Questions,” Proc. First Int'l Workshop Peer-to-Peer Systems (IPTPS), 2002.
[40] S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker, “A Scalable Content-Addressable Network,” Proc. ACM SIGCOMM, 2001.
[41] B.Y. Zhao et al., “Tapestry: A Resilient Global-Scale Overlay for Service Deployment,” IEEE J. Selected Areas in Comm., vol. 22, 2004.
[42] J. Kubiatowicz et al., “OceanStore: An Architecture for Global-Scale Persistent Storage,” Proc. Ninth Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2000.
[43] P. Mockapetris and K.J. Dunlap, “Development of the Domain Name System,” Proc. ACM SIGCOMM, 1988.
[44] J. Jung, E. Sit, H. Balakrishnan, and R. Morris, “DNS Performance and the Effectiveness of Caching,” IEEE/ACM Trans. Networking, vol. 10, pp. 589-603, 2002.
[45] S. Sun, L. Lannom, and B. Boesch, Handle System Overview, Internet Engineering Task Force (IETF) Request for Comments (RFC), RFC3650,, Nov. 2003.
[46] R. Kahn and R. Wilensky, “A Framework for Distributed Digital Object Services,” Int'l J. Digital Libraries, vol. 6, pp. 115-123, 2006.
[47] H. Cho, “Catalog Management in Heterogeneous Distributed Database Systems,” Proc. IEEE Pacific Rim Conf. Comm., Computers and Signal Processing (PACRIM), 1997.
[48] B.G. Lindsay, “A Retrospective of R: A Distributed Database Management System,” Proc. IEEE, vol. 75, pp. 668-673, 1987.
[49] D.M. Choy, P.G. Selinger, I. Center, and C.A. San Jose, “A Distributed Catalog for Heterogeneous Distributed Database Resources,” Proc. First Int'l Conf. Parallel and Distributed Information Systems (PDIS), 1991.
[50] Squid: Optimising Web Delivery, Squid-cache Project, http:/, 2008
[51] M. Beynon, T. Kurc, U. Catalyurek, C. Chang, A. Sussman, and J. Saltz, “Distributed Processing of Very Large Datasets with DataCutter,” Parallel Computing, vol. 27, pp. 1457-1478, 2001.
[52] N. Harvey, M. Jones, S. Saroiu, M. Theimer, and A. Wolman, “SkipNet: A Scalable Overlay Network with Practical Locality Properties,” Proc. Fourth USENIX Symp. Internet Technologies and Systems (USITS), 2003.
[53] J. Liang, R. Kumar, and K.W. Ross, “The KaZaA Overlay: A Measurement Study,” Computer Networks J., vol. 49, 2005.
[54] D. Karger, A. Sherman, A. Berkheimer, B. Bogstad, R. Dhanidina, K. Iwamoto, B. Kim, L. Matkins, and Y. Yerushalmi, “Web Caching with Consistent Hashing,” Proc. Eighth Int'l World Wide Web Conf. (WWW), 1999.
[55] D.R. Karger, E. Lehman, F.T. Leighton, R. Panigrahy, M.S. Levine, and D. Lewin, “Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web,” Proc. 29th ACM Symp. Theory of Computing (STOC), 1997.
[56] M. Rabinovich, J. Chase, and S. Gadde, “Not All Hits Are Created Equal: Cooperative Proxy Caching over a Wide-Area Network,” Proc. Third Int'l WWW Caching Workshop (WCW), 1998.

Index Terms:
Replica Location Service, replica management, grids, data management.
Ann L. Chervenak, Robert Schuler, Matei Ripeanu, Muhammad Ali Amer, Shishir Bharathi, Ian Foster, Adriana Iamnitchi, Carl Kesselman, "The Globus Replica Location Service: Design and Experience," IEEE Transactions on Parallel and Distributed Systems, vol. 20, no. 9, pp. 1260-1272, Sept. 2009, doi:10.1109/TPDS.2008.151
Usage of this product signifies your acceptance of the Terms of Use.