This Article 
 Bibliographic References 
 Add to: 
A Measurement-Based Model to Predict the Performance Impact of System Modifications: A Case Study
January 1995 (vol. 6 no. 1)
pp. 28-40

Abstract—This paper presents a performance case study of parallel jobs executing in real multi-user workloads. The study is based on a measurement-based model capable of predicting the completion time distribution of the jobs executing under real workloads. The model constructed is also capable of predicting the effects of system design changes on application performance. The model is a finite-state, discrete-time Markov model with rewards and costs associated with each state. The Markov states are defined from real measurements and represent system/workload states in which the machine has operated. This paper places special emphasis on choosing the correct number of states to represent the workload measured. Specifically, the performance of computationally-bound, parallel applications executing in real workloads on an Alliant FX/80 is evaluated. The constructed model is used to evaluate scheduling policies, the performance effects of multiprogramming overhead, and the scalability of the Alliant FX/80 in real workloads. The model identifies a number of available scheduling policies which would improve the response time of parallel jobs. In addition, the model predicts that doubling the number of processors in the current configuration would only improve response time for a typical parallel application by 25%. The model recommends a different processor configuration to more fully utilize extra processors. This paper also presents empirical results which validate the model created.

[1] G. Bell, “Ultracomputers: A Teraflop Before Its Time,” Comm. ACM, vol. 35, no. 8, pp. 26-47, Aug. 1992.
[2] J. Zahorjan and C. McCann,“Processor scheduling in shared memory multiprocessors,”inProc. 1990 ACM SIGM Conf. Meas., Model., Comput., Syst., May 1990, pp. 214–225.
[3] S. T. Leutenegger and M. K. Vernon,“The performance of multiprogrammed multiprocessor scheduling policies,”inProc. ACM Sigmetrics Conf., Boulder, CO, 1990, pp. 226–236.
[4] A. Gupta, A. Tucker, and S. Urushibara,“The impact of operating system scheduling policies and synchronization methods on the performance of parallel applications,”inProc. Conf. Measurement and Modeling of Comput. Syst.,1991, pp. 120–132.
[5] C. Polychronopoulos,“Multiprocessing versus multiprogramming,”inProc. Int. Conf. on Parallel Processing,1989, pp. 223–230.
[6] A. Malony,“Multiprocessor instrumentation: approaches for cedar,”inProc. Workshop on Instrumentation for Future Parallel Syst.,May, 1988.
[7] M.C. Hsueh, R.K. Iyer, and K.S. Trivedi,“Performability modeling based on real data: a case study,”IEEE Trans. Comput., vol. 37, pp. 478–484, Apr. 1988.
[8] V. Kulkarni, V. Nicola, R. Smith, and K.S. Trivedi,“Numerical evaluation of performability and job completion time in repairable fault-tolerant systems,”inProc. 16th Ann. Fault Tolerant Comput. Syst.,1986, pp. 252–257.
[9] C. Ciardo, R. Marie, B. Sericola, and K.S. Trivedi,“Performability analysis using semi-markov reward processes,”IEEE Trans. Comput.,pp. 1251–1264, Oct. 1990.
[10] Devarakonda, M.“Predictability of Process Resource Usage: A Measurement-Based Study of Unix,”Ph.D. dissertation, Univ. of Illinois at Urbana-Champaign, Oct., 1987.
[11] D. Ferrari, G. Serazzi, and A. Zeigner,Measurement and Tuning of Comput. Syst.Englewood Cliffs, NJ: Prentice-Hall, 1983.
[12] A. Goyal, P. Shahabuddin, P. Heidelberger, V. Nicola, and P. Glynn,“A Unified Framework for Simulating Markovian Models of Highly Dependable Systems,”IBM Tech. Rep. RC 14772 66141, July 1989.
[13] A. Bobbio and K. Trivedi,“Computation of the distribution of the completion time when the work requirement is a Ph random variable,”Stochastic Models,vol. 6, no. 1, pp. 133-150, 1990.
[14] R. Dimpsey and R. Iyer, "Modeling and measuring multiprogramming and system overheads on a multiprocessor: case study,"J. Parallel and Distrib. Comput.,pp. 402–414, Aug., 1991.
[15] ——,“Performance prediction and tuning on a multiprocessor,”Proc. Int. Symp. on Comput. Architecture,1991, pp. 190–199.
[16] ——,“Predicting the impact of scheduling modifications on system performance: case study,”Hawaii Int. Conf. Syst. Sci.,1992, pp. I559–I568.
[17] Alliant Computer System Corp.,“FX/Series Product Summary,”June, 1985.
[18] M. Berry, D. Chen, P. Koss, D. Kuck, S. Lo, Y. Pang, R. Roloff, et. al.,“The perfect club benchmarks: effective performance evaluation of supercomputers,”Int. J. Supercomputer Applic.,Fall 1989.
[19] SAS Institute,“SAS User's Guide: Statistics, 1985 Edition.”
[20] Artis, H.,“Workload characterization using SAS PROC FASTCLUS,”Int. Workshop on Workload Characterization of Comput. Syst. and Comput. Networks,1985, pp. 21–31.
[21] J. Neter, W. Wasserman, and M. Kutner,“Applied linear statistical models,”Irwin, IL, 1985.

Robert T. Dimpsey, Ravishankar K. Iyer, "A Measurement-Based Model to Predict the Performance Impact of System Modifications: A Case Study," IEEE Transactions on Parallel and Distributed Systems, vol. 6, no. 1, pp. 28-40, Jan. 1995, doi:10.1109/71.363413
Usage of this product signifies your acceptance of the Terms of Use.