This Article 
 Bibliographic References 
 Add to: 
Out-of-Core Divisible Load Processing
October 2003 (vol. 14 no. 10)
pp. 1048-1056
Maciej Drozdowski, IEEE Computer Society

Abstract—In this paper, we analyze processing divisible loads in systems with a memory hierarchy. Divisible loads are computations that can be divided into parts of arbitrary sizes and these parts can be independently processed in a distributed system. The problem is to partition the load so that the total processing time, including communications and computations, is the shortest possible. Earlier works in the divisible load theory assumed distributed systems with a flat memory model. The dependence of the processing time on the size of the assigned load was assumed to be linear. A new mathematical model relaxing the above two assumptions is proposed in this article. We study distributed systems which have both the hierarchical memory model and a piecewise linear dependence of the processing time on the size of the assigned load. Performance of such systems is modeled and evaluated. Finally, we compare the efficiency of distributed processing divisible loads in multiinstallment and out-of-core modes. Multiinstallment processing consists in sending multiple small chunks of the load to processors instead of a single chunk which needs external memory. It turns out that multiinstallment is an advantageous strategy for reasonably selected load chunks sizes.

[1] C. Bajaj, V. Pascucci, D. Thompson, and X.Y. Zhang, Parallel Accelerated Isocontouring for Out-Of-Core Visualization Proc. IEEE Parallel Visualization and Graphics Symp., pp. 97-104, , 1999.
[2] M. Berkelaar, lp_solveMixed Integer Linear Program Solver, / , 1995.
[3] V. Bharadwaj and G. Barlas, Access Time Minimization for Distributed Multimedia Applications Multimedia Tools and Applications, vol. 12, no. 2/3, pp. 235-256, 2000.
[4] V. Bharadwaj, D. Ghose, and T. Robertazzi, Divisible Load Theory: A New Paradigm for Load Scheduling in Distributed Systems Cluster Computing, vol. 6, no. 1, pp. 7-17, 2003.
[5] V. Bharadwaj, D. Ghose, and V. Mani, Multiinstallment Load Distribution in Tree Networks With Delays IEEE Trans. Aerospace and Electronic Systems, vol. 31, no. 2, pp. 555-567, 1995.
[6] V. Bharadwaj, D. Ghose, V. Mani, and T. Robertazzi, Scheduling Divisible Loads in Parallel and Distributed Systems. IEEE Computer Soc. Press, Los Alamitos, Calif., 1996.
[7] J. Blazewicz, M. Drozdowski, F. Guinand, and D. Trystram, Scheduling a Divisible Task in a 2-Dimensional Mesh Discrete Applied Math., vol. 94, nos. 1-3, pp. 35-50, 1999.
[8] J. Blazewicz, M. Drozdowski, and M. Markiewicz, Divisible Task Scheduling Concept and Verification Parallel Computing, vol. 25, pp. 87-98, 1999.
[9] P. Brezany, M. Bubak, M. Malawski, K. Zajac, Irregular and Out-of-Core Parallel Computing Proc. Conf. Parallel Processing and Applied Math., pp. 299-306, 2001.
[10] Y.C. Cheng and T.G. Robertazzi, Distributed Computation with Communication Delays IEEE Trans. Aerospace and Electronic Systems, vol. 24, no. 6, pp. 700-712, Nov. 1988.
[11] S.K. Chan, V. Bharadwaj, and D. Ghose, Experimental Study on Large Size Matrix-Vector Product Computations Using Divisible Load Paradigm on Distributed Bus Networks Math. and Computers in Simulation, vol. 58, no. 1, pp. 71-92, 2001.
[12] W.T. Corrêa, J.T. Klosowski, and C.T. Silva, Out-of-Core Sort-First Parallel Rendering for Cluster-Based Tiled Displays Proc. Eurographics Workshop Parallel Graphics and Visualization, piwalk.pdf, 2002.
[13] M. Drozdowski, Selected Problems of Scheduling Tasks in Multiprocessor Computer Systems. Poznan Univ. of Technology Press, Series: Monographs, no. 321,, 1997.
[14] M. Drozdowski and W. Glazek, Scheduling Divisible Loads in a Three-Dimensional Mesh of Processors Parallel Computing, vol. 25, pp. 381-404, 1999.
[15] M. Drozdowski and P. Wolniewicz, Experiments with Scheduling Divisible Tasks in Clusters of Workstations Proc. Euro-Par, pp. 311-319, 2000.
[16] M. Drozdowski and P. Wolniewicz, Divisible Load Scheduling in Systems with Limited Memory Cluster Computing, vol. 6, no. 1, pp. 19-29, 2003.
[17] Storage, Transformation, and Recomputation of Integrals Gaussian 98 Technical Info,http://www.gaussian.comg98.htm, 2000.
[18] K. Ko, Scheduling Data Intensive Parallel Processing in Distributed and Networked Environments PhD thesis, Dept. of Electrical and Computer Eng., State Univ. of New York at Stony Brook, 2000.
[19] X. Li, V. Bharadwaj, and C.C. Ko, Optimal Divisible Task Scheduling on Single-Level Tree Networks with Buffer Constraints IEEE Trans. Aerospace and Electronic Systems, vol. 36, no. 4, pp. 1298-1308, 2000.
[20] G.L. Nemhauser and L.A. Wolsey, Integer and Combinatorial Optimization. New York: John Wiley and Sons, 1988.
[21] Parallel Data-Intensive Algorithms and Applications Parallel Computing, D. Talia and P.K. Srimani, eds., vol. 28, no. 5, pp. 669-860, 2002.
[22] S. Toledo, A Survey of Out-of-Core Algorithms in Numerical Linear Algebra External Memory Algorithms, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Am. Math. Soc., J.M. Abello and J. Scott Vitter, eds.,http://www.math. il/~stoledo/Pubs oocsurvey.pdf, pp. 161-179, 1999.
[23] P. Wolniewicz, Multi-Installment Divisible Job Processing with Communication Startup Cost Foundations of Computing and Decision Sciences, vol. 27, no. 1, pp. 43-57, 2002.

Index Terms:
Divisible load theory, scheduling, performance evaluation, memory hierarchy, communication delays.
Maciej Drozdowski, Pawel Wolniewicz, "Out-of-Core Divisible Load Processing," IEEE Transactions on Parallel and Distributed Systems, vol. 14, no. 10, pp. 1048-1056, Oct. 2003, doi:10.1109/TPDS.2003.1239872
Usage of this product signifies your acceptance of the Terms of Use.