IEEE Transactions on Cloud Computing

From the October-December 2015 issue

LsPS: A Job Size-Based Scheduler for Efficient Task Assignments in Hadoop

By Yi Yao, Jianzhe Tai, Bo Sheng, and Ningfang Mi

Featured article thumbnail imageThe MapReduce paradigm and its open source implementation Hadoop are emerging as an important standard for large-scale data-intensive processing in both industry and academia. A MapReduce cluster is typically shared among multiple users with different types of workloads. When a flock of jobs are concurrently submitted to a MapReduce cluster, they compete for the shared resources and the overall system performance in terms of job response times, might be seriously degraded. Therefore, one challenging issue is the ability of efficient scheduling in such a shared MapReduce environment. However, we find that conventional scheduling algorithms supported by Hadoop cannot always guarantee good average response times under different workloads. To address this issue, we propose a new Hadoop scheduler, which leverages the knowledge of workload patterns to reduce average job response times by dynamically tuning the resource shares among users and the scheduling algorithms for each user. Both simulation and real experimental results from Amazon EC2 cluster show that our scheduler reduces the average MapReduce job response time under a variety of system workloads compared to the existing FIFO and Fair schedulers.

download PDF View article (PDF)     csdl View issue (CSDL)      TCC Facebook Link  TCC on Facebook      TCC LinkedIn Group  TCC on Linkedin


NOTE: We seek submission of papers that present new, original and innovative ideas for the "first" time in TCC (Transactions on Cloud Computing). That means, submission of "extended versions" of already published works (e.g., conference/workshop papers) is not encouraged unless they contain significant number of "new and original" ideas/contributions along with more than 65% brand "new" material. If you are submitting an extended version, you SHOULD submit a cover letter/document detailing (1) the "Summary of Differences" between TCC paper and earlier paper, (2) a clear listing of "new and original" ideas/contributions in TCC paper (identifying sections where they are proposed/presented), and (3) confirming the percentage of new material. Otherwise, submission will be "desk" rejected without any reviews.


Editorials and Announcements

Announcements


Editorials


Guest Editorials


Reviewers List


Author Index


Call for Papers

Special Issue on Advances of Utility and Cloud Computing Technologies and Services

Extended Submission deadline: February 15, 2016. View PDF.

Computing is rapidly moving towards a model where it is provided as services that are delivered in a manner similar to traditional utilities such as water, electricity, gas, and telephony. In such a model, users access services according to their requirements, without regard to where the services are hosted or how they are delivered. Several computing architectures have evolved to realize this utility computing vision, including Grid computing, Service-Oriented Architecture (SOA) and Cloud computing, which has recently shifted into the center of attention in the ICT industry. Increasing numbers of IT vendors are promising to offer applications, storage and computation hosting services with conforming Service-Level Agreements (SLA) to ensure Quality of Services (QoS) and performance. Considering many of these services are hosted in traditional data centers, there is significant complexity involved in ensuring the scalability, availability, manageability and accessibility of applications, services and data, as the scale of the systems as well as the users grows. As a result, it is becoming important to investigate the use of cloud computing techniques and its interoperability with utility computing. This special issue focuses on principles, paradigms and applications of "Utility computing" and its practical realization especially in the context of Cloud Computing.

Special Issue on Advances of Multimedia Big Data on the Cloud

Extended Submission deadline: February 21, 2016. View PDF.

Today’s multimedia big data is becoming a part of daily life in our society, industry and academia to access different multimedia systems, services and applications in terms of internet search, social media stream, internet-of-things-based streams, video stream in surveillance, medical image or video stream, mobile phone photos or video stream, business transactions, to name a few. However, due to the challenge of managing Exabyte of such multimedia big data in terms of computations, communications, storage, and sharing, there is a growing demand of an infrastructure to have on-demand access to a shared pool of configurable computing resources (e.g., networks, storages, processing, applications and services) to collect, store, preserve, manage, analyze, and share huge quantities of multimedia data. Cloud computing is such an infrastructure providing scalability, flexibility, agility, and ubiquity in terms of massive scale multimedia data processing, storage, access and communications.

The potential of this emergent research on multimedia big data is huge because of the major challenges in dealing with uncertainty, unpredictability (e.g., data volume, velocity and heterogeneity or variety) and massiveness with regards to real-time cross-media integration, multi-scale analysis and processing, where media is originated from multiple heterogeneous media sources with a variety of modality as well as context. Moreover, provisioning of Cloud resources for multimedia big data applications further enhances many technical challenges with regards to capturing, storage, searching, correlating, transferring, sharing, analysis, and visualization of multimedia.

General Call for Papers

General call for papers. View PDF.


TCC is financially cosponsored by:

IEEE Computer SocietyIEEE Communications SocietyIEEE Power & Energy SocietyIEEE Consumer Electronics SocietyIEEE Systems Council

TCC is technically cosponsored by:

IEEE Signal Processing Society