Collaborative Technologies and Systems, International Symposium on (2009)
Baltimore, MD, USA
May 18, 2009 to May 22, 2009
Arcot Rajasekar , University of North Carolina, Chapel Hill, USA
Reagan Moore , University of North Carolina, Chapel Hill, USA
Michael Wan , University of California at San Diego, La Jolla, USA
Wayne Schroeder , University of California at San Diego, La Jolla, USA
Large-scale Data Grid Systems (LDGS) facilitate collaborative sharing of large collections (Petabytes and100s of millions of objects) containing files, databases and data streams that are geographically distributed across heterogeneous resources and multiple administrative domains. LDGS provide a “universal view” of the distributed data, resources, users and methods and hide the idiosyncrasies and the heterogeneity of the underlying infrastructure and protocols - enhancing user collaborations. To improve transparency, an “open policy” system is needed by which data providers and administrators can describe the exact processes and policies that implement LDGS services. We consider policies and processes as the essential defining characteristics of a productive LDGS collaboration. We have implemented an LDGS, called integrated Rule-Oriented Data Systems (iRODS), which provides a universal view while enabling an open policy environment for publishing descriptions of the available services. The open policy environment is supported by a distributed workflow/rule engine. The services are encoded as rules in a high-level workflow language that transparently describes the underlying functionality. Well-defined semantics are used to control the composition of the workflow functions, called micro-services, to map to the desired client-level actions. In this paper, we describe the iRODS system from the “universal view” and “open policy” perspective and show its scalability for managing more than 10 million files.
M. Wan, W. Schroeder, A. Rajasekar and R. Moore, "Universal view and open policy: Paradigms for collaboration in data grids," Collaborative Technologies and Systems, International Symposium on(CTS), Baltimore, MD, USA, 2009, pp. 322-329.