This Article 
 Bibliographic References 
 Add to: 
A Stochastic Model for Heterogeneous Computing and Its Application in Data Relocation Scheme Development
November 1998 (vol. 9 no. 11)
pp. 1088-1101

Abstract—In a dedicated, mixed-machine, heterogeneous computing (HC) system, an application program may be decomposed into subtasks, then each subtask assigned to the machine where it is best suited for execution. Data relocation is defined as selecting the sources for needed data items. It is assumed that multiple independent subtasks of an application program can be executed concurrently on different machines whenever possible. A theoretical stochastic model for HC is proposed, in which the computation times of subtasks and communication times for intermachine data transfers can be random variables. The optimization problem for finding the optimal matching, scheduling, and data relocation schemes to minimize the total execution time of an application program is defined based on this stochastic HC model. The global optimization criterion and search space for the above optimization problem are described. It is validated that a greedy algorithm-based approach can establish a local optimization criterion for developing data relocation heuristics. The validation is provided by a theoretical proof based on a set of common assumptions about the underlying HC system and application program. The local optimization criterion established by the greedy approach, coupled with the search space defined for choosing valid data relocation schemes, can help developers of future practical data relocation heuristics.

[1] J.A. Bondy and U.S.R. Murty, Graph Theory with Applications.New York: Elsevier Science Publishing, 1976.
[2] G. Casella and R.L. Berger, Statistical Inference.Pacific Grove, Calif.: Wadsworth and Brooks/Cole Advanced Books and Software, 1990.
[3] S. Chen, M.M. Eshaghian, A. Khokhar, and M.E. Shaaban, "A Selection Theory and Methodology for Heterogeneous Supercomputing," Proc. Workshop Heterogeneous Processing, pp. 15-22, Apr. 1993.
[4] M. Cierniak, W. Li, and M.J. Zaki, "Loop Scheduling for Heterogeneity," Proc. Fourth IEEE Int'l Symp. High-Performance Distributed Computing, pp. 78-85, Aug. 1995.
[5] T.H. Cormen,C.E. Leiserson, and R.L. Rivest,Introduction to Algorithms.Cambridge, Mass.: MIT Press/McGraw-Hill, 1990.
[6] D.E. Comer, Internetworking with TCP/IP. Prentice-Hall, 1991.
[7] H.E. Rewini and T.G. Lewis,"Scheduling parallel program tasks onto arbitrary target machines," J. Parallel and Distributed Computing, vol. 9, pp. 138-153, 1990.
[8] R.F. Freund, "Optimal Selection Theory for Superconcurrency," Proc. Supercomputing '89, pp. 699-703, 1989.
[9] R.F. Freund, M. Gherrity, S. Ambrosius, M. Campbell, M. Halderman, D. Hensgen, E. Keith, T. Kidd, M. Kussow, J.D. Lima, F. Mirabile, L. Moore, B. Rust, and H.J. Siegel, "Scheduling Resources in Multi-User, Heterogeneous, Computing Environments with SmartNet," Proc. Seventh Heterogeneous Computing Workshop, pp. 184-199, Mar. 1998.
[10] R.F. Freund and H.J. Siegel, "Heterogeneous Processing," Computer, vol. 26, no. 6, pp. 13-17, June 1993.
[11] Ghafoor and J. Yang, "A Distributed Heterogeneous Supercomputing Management System," Computer, Vol. 26, No. 6, June 1993, pp. 78-86.
[12] M.A. Iverson, F. Ozguner, and G.J. Follen, "Parallelizing Existing Applications in a Distributed Heterogeneous Environment," Proc. Fourth Heterogeneous Computing Workshop, pp. 93-100, Apr. 1995.
[13] A. Khokhar, V.K. Prasanna, M. Shaaban, and C.L. Wang, "Heterogeneous Supercomputing: Problems and Issues," Proc. Workshop Heterogeneous Processing, pp. 3-12, Mar. 1992.
[14] A.A. Khokhar, V.K. Prasanna, M.E. Shaaban,, and C.L. Wang, “Heterogeneous Computing: Challenges and Opportunities,” Computer, pp. 18-27, June 1993.
[15] L. Kleinrock, Communication Nets: Stochastic Message Flow and Delay.New York: McGraw-Hill, 1964.
[16] Y.A. Li and J.K. Antonio, "Estimating the Execution Time Distribution for a Task Graph in a Heterogeneous Computing System," Proc. Sixth Heterogeneous Computing Workshop, pp. 172-184, Apr. 1997.
[17] Y.A. Li, J.K. Antonio, H.J. Siegel, M. Tan, and D.W. Watson, "Determining the Execution Time Distribution for a Data Parallel Program in a Heterogeneous Computing Environment," J. Parallel and Distributed Computing, vol. 44, no. 1, pp. 35-52, July 1997.
[18] B. Narahari, A. Youssef, and H.A. Choi, "Matching and Scheduling in a Generalized Optimal Selection Theory," Proc. Heterogeneous Computing Workshop, pp. 3-8, Apr. 1994.
[19] H.J. Siegel, J.K. Antonio, R.C. Metzger, M. Tan, and Y.A. Li, "Heterogeneous Computing," Parallel and Distributed Computing Handbook, A.Y. Zomaya, ed., pp. 725-761.New York: McGraw-Hill, 1996.
[20] H.J. Siegel, H.G. Dietz, and J.K. Antonio, "Software Support for Heterogeneous Computing," The Computer Science and Engineering Handbook, A.B. Tucker Jr., ed., pp. 1,886-1,909.Boca Raton, Fla.: CRC Press, 1997
[21] G.C. Sih and E.A. Lee, “A Compile-Time Scheduling Heuristic for Interconnection-Constrained Heterogeneous Processor Architectures,” IEEE Trans. Parallel and Distributed Systems, vol. 4, no. 2, pp. 175-186, Feb. 1993.
[22] M. Tan, H.J. Siegel, J.K. Antonio, and Y.A. Li, “Minimizing the Application Execution Time through Scheduling of Subtasks and Communication Traffic in a Heterogeneous Computing System,” IEEE Trans. Parallel and Distributed Systems, vol. 8, no. 8, pp. 857-871, Aug. 1997.
[23] D.F. Towsley, "Allocating Programs Containing Branches and Loops within a Multiple Processor System," IEEE Trans. Software Eng., vol. 12, pp. 1,018-1,024, Oct. 1986.
[24] D.W. Watson, J.K. Antonio, H.J. Siegel, R. Gupta, and M.J. Atallah, "Static Matching of Ordered Program Segments to Dedicated Machines in a Heterogeneous Computing Environment," Proc. Fifth Heterogeneous Computing Workshop, pp. 24-37, Apr. 1996.
[25] M. Wang, S.-D. Kim, M.A. Nichols, R.F. Freund, H.J. Siegel, and W.G. Nation, "Augmenting the Optimal Selection Theory for Superconcurrency," Proc. Workshop Heterogeneous Processing, pp. 13-22, Mar. 1992.
[26] L. Wang, H.J. Siegel, V.P. Roychowdhury, and A.A. Maciejewski “, Task Matching and Scheduling in Heterogeneous Computing Environments Using a Genetic-Algorithm-Based Approach,” J. Parallel and Distributed Computing, vol. 47, no. 1, pp. 1-15, Nov. 1997.
[27] J. Yang, A. Khokhar, S. Sheikh, and A. Ghafoor, "Estimating Execution Time for Parallel Tasks in Heterogeneous Processing (HP) Environment," Proc. Heterogeneous Computing Workshop, pp. 23-28, Apr. 1994.
[28] X. Zhang and Y. Yan, "Modeling and Characterizing Parallel Computing Performance on Heterogeneous Networks of Workstations," Proc. Seventh IEEE Symp. Parallel and Distributed Processing, pp. 25-34, Oct. 1995.

Index Terms:
Data relocation, greedy algorithm, heterogeneous computing, mapping, matching, optimization, scheduling, stochastic modeling.
Min Tan, Howard Jay Siegel, "A Stochastic Model for Heterogeneous Computing and Its Application in Data Relocation Scheme Development," IEEE Transactions on Parallel and Distributed Systems, vol. 9, no. 11, pp. 1088-1101, Nov. 1998, doi:10.1109/71.735956
Usage of this product signifies your acceptance of the Terms of Use.