Grids Get Closer
by George Lawton
Grid computing lets users access resources across distributed computers to provide high performance and high-capacity storage. It has become an established part of data-intensive research. However, the computing and networking worlds have traditionally evolved separately, making it difficult to set up grid applications across networks managed by different operators. In response to this problem, an international research team funded by the European Union has recently completed the Phosphorus project to help bridge grid applications across multiple networks.
Phosphorous is a tool suite that lets grid users dynamically tune networks to an application's needs rather than tailor grid applications to existing network resources, noted Artur Binczewski, professor at Poznan University and Phosphorus project coordinator. Binczewski said the tools suite has significantly enhanced e-science application capabilities by providing a unified network/grid infrastructure. "[The infrastructure] can flexibly adapt to the demands of applications having strong, combined requirements on CPU, memory, and storage resources as well as on the communication network," he said.
The project was designed to support scientific applications that have network requirements such as determinism (for example, guaranteed QoS), shared data spaces, and large data transfers. These capabilities often require dedicated optical bandwidth.
High-capacity optical networking can satisfy bandwidth and latency requirements, but grid users need software tools and frameworks for end-to-end, on-demand network service provisioning. These tools and frameworks must be able to coordinate with other resources, such as CPU and storage. They must also span multiple administrative and network-technology domains.
Phosphorus Testbed
The Phosphorus testbed involved several research networks and organizations including European national research and education networks (NRENs), Internet2, the Canadian Advanced Network and Research for Industry and Education, and the Global Lambda Integrated Facility (GLIF).
Project participants tested Phosphorus components in four applications to work out some of the bugs and demonstrate how all components would work together in a real-world project:
- The EU's Wirespeed Security Domains using Optical Monitoring (Wisdom) project used Phosphorus to send up to 27,000 small files to several grid nodes for analysis of malaria drugs.
- The KoDaVis visualization project used it to transfer approximately 90 Gbytes of data in a few files for atmospheric simulation.
- The GridFTP and Tivolie Storage Manager high-performance data transfer protocols used Phosphorus to implement DDDS (distributed data storage system) applications as part of their testing program.
- The Technology for Optical Pixel Streaming project used Phosphorus to stream ultra-high-resolution images.
Binczewski said Phosphorus technology could be useful in commercial applications, such as distributed computing, as well as in the distribution of very high quality media with huge bandwidth requirements.
The goal was to develop integration between applications, middleware, and transport networks, based on a service plane, a Network Resource Provisioning System (NRPS) plane, and a control plane. The service plane defines a set of APIs for automatically meeting resource authentication, authorization, and accounting (AAA) protocols across multiple networks. The NRPS plane was integrated with the middleware so that applications could control different networks automatically using the Harmony multidomain network service architecture framework that Phosphorus developed. The control plane is managed using Grid-Enabled Generalized Multiprotocol Label Switching (G2MPLS), which allows middleware to automatically switch optical networks.
Bridging Services
To prevent accidental or malicious misuse of networking and compute resources, a variety of AAA frameworks have emerged. These frameworks ensure that resources are used in an appropriate manner, but different organizations have implemented different frameworks. This makes it difficult for an application developed in Europe, for example, to control resources at a US facility.
"We realized that we need different security mechanisms to support policy-based access control and flexible policy enforcement in multidomain environments," said Yuri Demchenko, senior researcher at the University of Amsterdam's Advanced Internet Research Group. As a result of Phosphorus, the group proposed a generic AAA framework for network resource provisioning (GAAA-NRP).
Translating Reservations
Several research efforts have implemented NRPSs to streamline network resource scheduling, but each NRPS is limited to a particular research network. The Harmony framework provides a translation mechanism to automate scheduling across networks. The challenge is that each NRPS implements different types of functionality. Harmony addresses this challenge by accepting a minimum set of the functionalities that all systems perform and need for grid applications. The Phosphorus team has tested the mechanism across as many as 10 independent domains.
Evangelos Chaniotakis is chair of GLIF's generic network interface (GNI) API working group, which addresses interoperability between NRPSs. "Each NRPS provides some flavor of guaranteed quality, circuit-like service with advance scheduling," he said, "but they all have different (though roughly equivalent) APIs. There is no current standard API, and even though one is in the works from the OGF NSI [Open Grid Forum's Network Service Interface] working group, we're years away from an approved standard."
Grid Meets Optics
GMPLS has been widely implemented in optical networking equipment to manage optical networks and change topologies. G2MPLS is an GMPLS extension that allocates and provisions network and grid resources in a single step. Binczewski said the tight integration with the GMPLS standard promises to overcome the current limitation of grids operating as stand-alone networks with their own administrative ownership and procedures.
G2MPLS enables a real node-to-node deployment of on-demand grid services for users. Moreover, for network operators (particularly the NRENs), it creates a way to integrate grids and automated network-control-plane technologies in real operational infrastructures.
"When you think about how we evolved from grids to clouds, this is an exciting commoditization of the use case scenarios that is important to many researchers and many folks using these kinds of resources," said Francine Berman, vice president for research at Rensselaer Polytechnic Institute and coeditor of Grid Computing: Making the Global Infrastructure a Reality. "Over the last decade, there has been an unbelievable evolution between the Grid and the Web work, and now we are linking up different kinds of sources all of the time in ways that we only dreamed of over the last decade."
Table 1 summarizes the Phosphorus enhancements. The Phosphorus software and tools are ready for use and available at www.ist-phosphorus.eu.
Table 1. Phosphorus enhancements | Topic | Technical solutions before Phosphorus | Phosphorus project enhancements |
| End-to-end, multidomain, on-demand resource provisioning | Dynamic light-path provisioning through NRPSs or standard GMPLS control plane, but no correlations between grid-provisioning sessions initiated by the applications | Single-step light-path and grid-resources provisioning. Network and grid-specific resources are set up and controlled at the same time and with the same priority. A true user-to-user on-demand grid-service provisioning. |
| Service plane: Applications | Grid-specific applications with limited or no ability to place demands on network resources | Support for applications that combine requirements from network and conventional grid resources |
| Service plane: Authentication, authorization, and auditing | Several AAA components aimed at user attributes and roles, but no focus on multidomain networking | An AAA component set using available standards and focused toward multidomain optical networking. Existing AAA infrastructures integrate smoothly with Phosphorus innovations for grid middleware, NRPSs, and the GMPLS control plane. |
| Grid resources: Provisioning, management, and reservation | Through standard Grid middleware unaware of network resources | Through enhanced grid middleware empowered by G2MPLS |
| Network resources: Provisioning, management, and reservation | Different NRPSs, independent and not interoperable | Standard GMPLS control plane and optical network interfaces for dynamic light-path setup. Seamless interoperability between three NRPSs, and between the NRENs and the standard GMPLS control plane. GMPLS enhancements toward an integrated control-plane concept that works with NRPS and grid middleware and applications |
