This Article 
 Bibliographic References 
 Add to: 
Design and Potential Performance of Goal-Oriented Job Scheduling Policies for Parallel Computer Workloads
December 2008 (vol. 19 no. 12)
pp. 1642-1656
Su-Hui Chiang, Portland State Univ, Portland
Sangsuree Vasupongayya, Portland State University, Portland
To balance multiple scheduling performance requirements on parallel computer systems, traditional job schedulers use many parameters that can be configured to define job or queue priorities. Offering many parameters seems flexible, but in reality tuning the values for the parameters is highly challenging. To simplify the task of resource management, we propose goal-oriented policies, which allow system administrators to specify high-level performance objectives, rather than tuning low-level scheduling parameters. We study the design of goal-oriented policies, including (1) appropriate multi-objective models for specifying trade-offs between objectives, (2) efficient search algorithms for searching the best schedule at each scheduling decision point, and (3) appropriate performance measures to be optimized in the objectives with respect to two common performance requirements: preventing starvation and favoring shorter jobs. We compare goal-oriented policies with widely used backfill policies. Policies are evaluated by simulation using ten monthly workloads that ran on a Linux cluster (IA-64) from NCSA. Our results show that by automatically optimizing performance according to the given objectives through search, goal-oriented policies can simultaneously outperform FCFS-backfill and LXF-backfill, which are designed in favor of the maximum wait and average slowdown, respectively.

[1] OpenPBS, http://www.openpbs.orgdocs.html, Aug. 2000.
[2] LSF Scheduler, Platform Computing, http:/, 2008.
[3] Maui Scheduler, http://www.supercluster.orgmaui/, 2008.
[4] B. Chun and D. Culler, “User-Centric Performance Analysis of Market-Based Cluster Batch Schedulers,” Proc. Second IEEE Int'l Symp. Cluster Computing and the Grid (CCGRID '02), pp.30-38, May 2002.
[5] K. Kurowski, J. Nabrzyski, A. Oleksiak, and J. Weglarz, “Scheduling Jobs on the Grid-Multicriteria Approach,” Computational Methods in Science and Technology, vol. 12, no. 2, pp. 123-138, 2006.
[6] Parallel Workloads Archive, models.html, 2008.
[7] S. Vasupongayya, S.-H. Chiang, and B. Massey, “Search-Based Job Scheduling for Parallel Computer Workloads,” Proc. IEEE Int'l Conf. Cluster Computing (CLUSTER '05), Sept. 2005.
[8] S. Vasupongayya and S.-H. Chiang, “Multi-Objective Models for Scheduling Jobs on Parallel Computer System,” Proc. IEEE Int'l Conf. Cluster Computing (CLUSTER '06), short paper, Sept. 2006.
[9] D. Lifka, “The ANL/IBM SP Scheduling System,” Proc. First Job Scheduling Strategies for Parallel Processing (JSSPP '95), Apr. 1995.
[10] S. Kannan, M. Roberts, P. Mayes, D. Brelsford, and J.F. Skovira, Workload Management with Loadleveler, IBM, Nov. 2001.
[11] D. Zotkin and P.J. Keleher, “Job-Length Estimation and Performance in Backfilling Schedulers,” Proc. Eighth IEEE Int'l Symp. High Performance Distributed Computing (HPDC '99), pp. 236-243, Aug. 1999.
[12] S.-H. Chiang and M.K. Vernon, “Production Job Scheduling for Parallel Shared Memory Systems,” Proc. 15th IEEE Int'l Parallel and Distributed Processing Symp. (IPDPS '01), Apr. 2001.
[13] S.-H. Chiang, A. Dusseau-Arpaci, and M.K. Vernon, “The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance,” Proc. Eighth Job Scheduling Strategies for Parallel Processing (JSSPP '02), pp. 103-127, July 2002.
[14] S. Srinivasan, R. Kettimuthu, V. Subramani, and P. Sadayappan, “Selective Reservation Strategies for Backfill Job Scheduling,” Proc. Eighth Job Scheduling Strategies for Parallel Processing (JSSPP'02), pp. 55-71, July 2002.
[15] A.W. Mu'alem and D.G. Feitelson, “Utilization, Predictability, Workloads, and User Runtime Estimates in Scheduling the IBM SP2 with Backfilling,” IEEE Trans. Parallel and Distributed Systems, vol. 12, no. 6, pp. 529-543, June 2001.
[16] S.-H. Chiang and C. Fu, “Re-Evaluating Reservation Policies for Backfill Scheduling on Parallel Systems,” Proc. 16th IASTED Int'l Conf. Parallel and Distributed Computing and Systems (PDCS '04), Nov. 2004.
[17] D. Talby and D.G. Feitelson, “Improving and Stabilizing Parallel Computer Performance Using Adaptive Backfilling,” Proc. 19th IEEE Int'l Parallel and Distributed Processing Symp. (IPDPS '05), Apr. 2005.
[18] B. Lawson and E. Smirni, “Self-Adaptive Scheduler Parameterization via Online Simulation,” Proc. 19th IEEE Int'l Parallel and Distributed Processing Symp. (IPDPS '05), Apr. 2005.
[19] M. Ehrgott and X. Gandibleux, “A Survey and Annotated Bibliography of Multiobjective Combinatorial Optimization,” OR Spektrum, vol. 22, pp. 425-460, 2000.
[20] J.M. Crawford, “Solving Satisfiability Problems Using a Combination of Systematic and Local Search,” Second DIMACS Challenge: Cliques, Coloring, and Satisfiability, Oct. 1993.
[21] W.D. Harvey and M.L. Ginsberg, “Limited Discrepancy Search,” Proc. 14th Int'l Joint Conf. Artificial Intelligence (IJCAI'95), Aug. 1995.
[22] R.E. Korf, “Improved Limited Discrepancy Search,” Proc. 13th Nat'l Conf. Artificial Intelligence (AAAI '96), pp. 209-215, Aug. 1996.
[23] T. Walsh, “Depth-Bounded Discrepancy Search,” Proc. 15th Int'l Joint Conf. Artificial Intelligence (IJCAI '97), vol. 2, pp. 1388-1393, Aug. 1997.
[24] R. Gangadharan and C. Rajendran, “A Simulated Heuristic for Scheduling in a Flowshop with Bicriteria,” Computers and Industrial Eng., vol. 27, nos. 1-4, pp. 473-476, 1994.

Index Terms:
Scheduling, Parallel systems, Batch processing systems, Goal-oriented policies, Multi-objective models, Backfill scheduling policies, Search algorithms
Su-Hui Chiang, Sangsuree Vasupongayya, "Design and Potential Performance of Goal-Oriented Job Scheduling Policies for Parallel Computer Workloads," IEEE Transactions on Parallel and Distributed Systems, vol. 19, no. 12, pp. 1642-1656, Dec. 2008, doi:10.1109/TPDS.2008.48
Usage of this product signifies your acceptance of the Terms of Use.