IEEE Transactions on Computers (TC) has moved to the OnlinePlus publication model starting with 2013 issues!

From the October 2014 issue

NO2: Speeding up Parallel Processing of Massive Compute-Intensive Tasks

By Yongwei Wu, Weichao Guo, Jinglei Ren, Xun Zhao, and Weimin Zheng

Featured article thumbnail imageLarge-scale computing frameworks, either tenanted on the cloud or deployed in the high-end local cluster, have become an indispensable software infrastructure to support numerous enterprise and scientific applications. Tasks executed on these frameworks are generally classified into data-intensive and compute-intensive ones. However, most existing frameworks, led by MapReduce, are mainly suitable for data-intensive tasks. Their task schedulers assume that the proportion of data I/O reflects the task progress and state. Unfortunately, this assumption does not apply to most compute-intensive tasks. Due to biased estimation of task progress, traditional frameworks cannot timely cut off outliers and therefore largely prolong execution time when performing compute-intensive tasks. We propose a new framework designed for compute-intensive tasks. By using instrumentation and automatic instrument point selector, our framework estimates the compute-intensive task progress without resorting to data I/O. We employ a clustering method to identify outliers at runtime and perform speculative execution/aborting, speeding up task execution by up to 25%. Moreover, our improvement to bare instrumentation limits overhead within 0.1%, and the aborting-based execution only introduces 10% more average CPU usage. Low overhead and resource consumption make our framework practically usable in the production environment.

download PDF View the PDF of this article      csdl View this issue in the digital library

Editorials and Announcements


New Essential Set


Guest Editorials


Reviewers List

Annual Index

Access Recently Published TC Articles

RSS Subscribe to the RSS feed of latest TC content added to the digital library

Mail Sign up for the Transactions Connection newsletter.

Importance of Coherence Protocols with Network Applications on Multi-Core Processors


Automated Generation of Performance and Dependability Models for the Assessment of Wireless Sensor Networks


IEEE Transactions on Computers (TC) is a monthly publication that publishes research in such areas as computer organizations and architectures, digital devices, operating systems, and new and important applications and trends. 
Read the full scope of TC