The Community for Technology Leaders
High Performance Computing, Networking Storage and Analysis, SC Companion: (2012)
Salt Lake City, Utah, USA
June 24, 2012 to June 29, 2012
ISBN: 978-1-4673-3049-7
pp: 1625-1628
A. Daga , Dept. of Comput. Sci., Univ. of Houston, Houston, TX, USA
R. Powell , Comput. Inst., Univ. of Chicago, Chicago, IL, USA
A. Heath , Comput. Inst., Univ. of Chicago, Chicago, IL, USA
R. Grossman , Comput. Inst., Univ. of Chicago, Chicago, IL, USA
M. Greenway , Comput. Inst., Univ. of Chicago, Chicago, IL, USA
S. Bailey , InfoBlox Inc., Santa Clara, CA, USA
S. Narayan , InfoBlox Inc., Santa Clara, CA, USA
ABSTRACT
Hadoop has emerged as an important platform for data intensive computing. The shuffle and sort phases of a MapReduce computation often saturate top of the rack switches, as well as switches that aggregate multiple racks. In addition, MapReduce computations often have "hot spots" in which the computation is lengthened due to inadequate bandwidth to some of the nodes. In principle, OpenFlow enables an application to adjust the network topology as required by the computation, providing additional network bandwidth to those resources requiring it. We describe Hadoop-OFE, which is an OpenFlow enabled version of Hadoop that dynamically modifies the network topology in order to improve the performance of Hadoop.
INDEX TERMS
MapReduce, Hadoop, OpenFlow, OpenFlow over Ethernet, data intensive computing
CITATION
A. Daga, R. Powell, A. Heath, R. Grossman, M. Greenway, S. Bailey, S. Narayan, "OpenFlow Enabled Hadoop over Local and Wide Area Clusters", High Performance Computing, Networking Storage and Analysis, SC Companion:, vol. 00, no. , pp. 1625-1628, 2012, doi:10.1109/SC.Companion.2012.340
88 ms
(Ver 3.3 (11022016))