Cluster Computing and the Grid, IEEE International Symposium on (2009)
May 18, 2009 to May 21, 2009
Grid data sharing systems usually provide a data-intensive application with either a pre-staging mechanism or an on-demand access mechanism to access shared data. Pre-staging systems simultaneously download an entire shared file from multiple data sources even when only a tiny file fragment is required. Such systems consume unnecessary data transmission time and storage space. On-demand access systems, on the other hand, download only the required fragments from a single data source. Such systems unfortunately do not fully exploit available network bandwidth. This paper presents a data sharing system, designated as the On-Demand data Co-Allocation (ODCA). ODCA facilitates an unmodified legacy applications to transparently access shared data by using native I/O system calls. ODCA transfers only the necessary fragments on user demand, thereby reducing data transmission time, avoiding wasted network bandwidth and wasted storage space. Moreover, ODCA reduces data waiting time by downloading the file fragments from multiple data sources. Experimental results show ODCA successfully reduces turnaround time in data-intensive applications.
Data Grid, Distributed Shared Memory, On-demand Access, Co-allocation, Teamster-G
Y. Zhuang, C. Shieh, T. Liang, J. Chang and P. Chen, "Memory-Mapped File Approach for On-Demand Data Co-allocation on Grids," Cluster Computing and the Grid, IEEE International Symposium on(CCGRID), Shanghai, China, 2009, pp. 300-307.