This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
2012 IEEE Fifth International Conference on Cloud Computing
Evaluating Hadoop for Data-Intensive Scientific Operations
Honolulu, HI, USA USA
June 24-June 29
ISBN: 978-1-4673-2892-0
Emerging sensor networks, more capable instruments, and ever increasing simulation scales are generating data at a rate that exceeds our ability to effectively manage, curate, analyze, and share it. Data-intensive computing is expected to revolutionize the next-generation software stack. Hadoop, an open source implementation of the MapReduce model provides a way for large data volumes to be seamlessly processed through use of large commodity computers. The inherent parallelization, synchronization and fault-tolerance the model offers, makes it ideal for highly-parallel data-intensive applications. MapReduce and Hadoop have traditionally been used for web data processing and only recently been used for scientific applications. There is a limited understanding on the performance characteristics that scientific data intensive applications can obtain from MapReduce and Hadoop. Thus, it is important to evaluate Hadoop specifically for data-intensive scientific operations -- filter, merge and reorder-- to understand its various design considerations and performance trade-offs. In this paper, we evaluate Hadoop for these data operations in the context of High Performance Computing (HPC) environments to understand the impact of the file system, network and programming modes on performance.
Index Terms:
Conferences,Cloud computing
Citation:
Zacharia Fadika, Madhusudhan Govindaraju, Richard Canon, Lavanya Ramakrishnan, "Evaluating Hadoop for Data-Intensive Scientific Operations," cloud, pp.67-74, 2012 IEEE Fifth International Conference on Cloud Computing, 2012
Usage of this product signifies your acceptance of the Terms of Use.