This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Model-Based Performance Prediction in Software Development: A Survey
May 2004 (vol. 30 no. 5)
pp. 295-310

Abstract—Over the last decade, a lot of research has been directed toward integrating performance analysis into the software development process. Traditional software development methods focus on software correctness, introducing performance issues later in the development process. This approach does not take into account the fact that performance problems may require considerable changes in design, for example, at the software architecture level, or even worse at the requirement analysis level. Several approaches were proposed in order to address early software performance analysis. Although some of them have been successfully applied, we are still far from seeing performance analysis integrated into ordinary software development. In this paper, we present a comprehensive review of recent research in the field of model-based performance prediction at software development time in order to assess the maturity of the field and point out promising research directions.

[1] F. Andolfi, F. Aquilani, S. Balsamo, and P. Inverardi, Deriving Performance Models of Software Architectures from Message Sequence Charts ACM Proc. Second Int'l Workshop Software and Performance, pp. 47-57, 2000.
[2] D. Austry and G. Boudol, Algébre de Processus et Synchronization Theoretical Computer Science, vol. 30, no. 1, pp. 91-131, 1984.
[3] M. Ajmone, G. Balbo, and G. Conte, Performance Models of Multiprocessor Performance. MIT Press, 1986.
[4] F. Aquilani, S. Balsamo, and P. Inverardi, Performance Analysis at the Software Architecture Design Level Performance Evaluation, vol. 45, no. 4, pp. 205-221, 2001.
[5] L.B. Arief and N.A. Speirs, A UML Tool for an Automatic Generation of Simulation Programs ACM Proc. Second Int'l Workshop Software and Performance, pp. 71-76, 2000.
[6] P. Buchholz, A Framework for the Hierarchical Analysis of Discrete Event Dynamic Systems PhD thesis, Univ. of Dortmund, Germany, 1996.
[7] F. Baccelli, G. Balbo, R.J. Boucherie, J. Campos, and G. Chiola, Annotated Bibliography on Stochastic Petri Nets Performance Evaluation of Parallel and Distributed Systems Solution Methods, CWI Tract, 105, pp. 1-24, 1994.
[8] G. Balbo, S. Bruell, and S. Ghanta, Combining Queueing Networks and Generalized Stochastic Petri Nets for the Solution of Complex Models of System Behaviour IEEE Trans. Computers, vol. 37, pp. 1251-1268, 1988.
[9] S. Balsamo, M. Bernardo, and M. Simeoni, Combining Stochastic Process Algebras and Queueing Networks for Software Architecture Analysis ACM Proc. Int'l Workshop Software and Performance, pp. 190-202, 2002.
[10] R.J.A. Buhr and R.S. Casselman, Use CASE Maps for Object-Oriented Systems. Prentice Hall, 1996.
[11] M. Bernardo, P. Ciancarini, and L. Donatiello, ÆMPA: A Process Algebraic Description Language for the Performance Analysis of Software Architectures ACM Proc. Int'l Workshop Software and Performance, pp. 1-11, 2000.
[12] L. Bass, P. Clements, and R. Kazman, Software Architecture in Practice SEI Series in Software Eng., 1998.
[13] J. Banks, J.S. Carson II, B.L. Nelson, and D.M. Nicol, Discrete-Event System Simulation. Prentice-Hall, 1999.
[14] M. Bernardo, W.R. Cleaveland, and W.S. Stewart, TwoTowers 1.0 User Manual http://www.sti.uniurb.it/bernardotwotowers /, 2001.
[15] S. Balsamo, A. Di Marco, P. Inverardi, and M. Simeoni, Model-Based Performance Prediction in Software Development: A Survey Technical Report TR 011/2004, Dipartimento di Informatica, Univ. di L'Aquila, 2004.
[16] S. Bernardi, S. Donatelli, and J. Merseguer, From UML Sequence Diagrams and Statecharts to Analysable Petri Net Models ACM Proc. Int'l Workshop Software and Performance, pp. 35-45, 2002.
[17] M. Bernardo and R. Gorrieri, A Tutorial on EMPA: A Theory of Concurrent Processes with Nondeterminism, Priorities, Probabilities and Time Theoretical Computer Science, vol. 202, pp. 1-54, 1998.
[18] J. Bosch and P. Molin, Software Architecture Design: Evaluation and Transformation Proc. 1999 IEEE Eng. of Computer Systems Symp., pp. 4-10, Dec. 1999.
[19] H. Beilner, J. Mäter, and C. Wysocki, The Hierarchical Evaluation Tool HIT Proc. Seventh Int'l Conf. Modelling Techniques and Tools for Computer Performance Evaluation, 1994.
[20] V. Cortellessa, A. D'Ambrogio, and G. Iazeolla, Automatic Derivation of Software Performance Models from CASE Documents Performance Evaluation, vol. 45, pp. 81-105, 2001.
[21] P.S. Coe, F.W. Howell, R.N. Ibbett, and L.M. Williams, Technical Note: A Hierarchical Computer Architecture Design and Simulation Environment ACM Trans. Modelling and Computer Simulation, vol. 8, no. 4, pp. 431-446, 1998.
[22] V. Cortellessa, G. Iazeolla, and R. Mirandola, Early Generation of Performance Models for Object-Oriented Systems IEE Proc.-Software, vol. 147, no. 3 pp. 61-72, 2000.
[23] V. Cortellessa and R. Mirandola, Deriving a Queueing Network Based Performance Model from UML Diagrams ACM Proc. Int'l Workshop Software and Performance, pp. 58-70, 2000.
[24] C++Sim http:/cxxsim.ncl.ac.uk/, 1997.
[25] CSIM Performance Simulator http://www.atl.lmco.com/projcsim, 2004.
[26] M. DeMiguel, T. Lambolais, M. Hannouz, S. Betgé-Brezetz, and S. Piekarec, UML Extensions for the Specification and Evaluation of Latency Constraints in Architectural Models ACM Proc. Int'l Workshop Software and Performance, pp. 83-88, 2000.
[27] P.H. Enslow Jr., What is a‘Distributed’Data Processing System? Computer, vol. 11, no. 1, pp. 13-21, Jan. 1978.
[28] G. Franks, A. Hubbard, S. Majumdar, D.C. Petriu, J. Rolia, and C.M. Woodside, A Toolset for Performance Engineering and Software Design of Client-Server Systems Performance Evaluation, vol. 24, nos. 1-2, pp. 117-135, 1995.
[29] R. Franks and C.M. Woodside, Performance of Multi-Level Client-Server Systems with Parallel Service Operations ACM Proc. Int'l Workshop Software and Performance, pp. 120-130, 1998.
[30] S. Gilmore and J. Hillston, The PEPA Workbench: A Tool to Support a Process Algebra-Based Approach to Performance Modelling Proc. Seventh Int'l Conf. Modelling Techniques and Tools for Performance Evaluation, pp. 353-368, 1994.
[31] N. Götz, U. Herzog, and M. Rettelback, TIPP A Language for Timed Processes and Performance Evaluation Technical Report 4/92, IMMD7, Univ. of Erlangen-Nürnberg, Germany, 1992.
[32] H. Gomaa and D.A. Menascé, Design and Performance Modeling of Component Interconnection Patterns for Distributed Software Architectures ACM Proc. Int'l Workshop Software and Performance, pp. 117-126, 2000.
[33] H. Gomaa and D. Menascé, Performance Engineering of Component-Based Distributed Software Systems Performance Eng., R. Dumke et al., eds. pp. 40-55, 2001.
[34] V. Grassi and R. Mirandola, PRIMAmob-UML: A Methodology for Performance Analysis of Mobile Software Architectures ACM Proc. Int'l Workshop Software and Performance, pp. 262-274, 2002.
[35] G. Gu and D.C. Petriu, XSLT Transformation from UML Models to LQN Performance Models ACM Proc. Int'l Workshop Software and Performance, pp. 227-234, 2002.
[36] C.A.R. Hoare, Communicating Sequential Processes. Prentice-Hall, 1985.
[37] J. Hillston, PEPA Performance Enhanced Process Algebra Technical Report CSR-24-93, Dept. of Computer Science, Univ. of Edimburgh, UK, 1993.
[38] F. Hoeben, Using UML Models for Performance Calculation ACM Proc. Int'l Workshop Software and Performance, pp. 77-82, 2000.
[39] P.G. Harrison and J. Hillston, Exploiting Quasi-Reversible Structures in Markovian Process Algebra Models Computer J., vol. 38, no. 7, pp. 510-520, 1995.
[40] H. Hermanns, U. Herzog, and J.P. Katoen, Process Algebra for Performance Evaluation Theoretical Computer Science, vol. 274, nos. 1-2, pp. 43-87, 2002.
[41] U. Herzog, U. Klehmet, V. Mertsiotakis, and M. Siegle, Compositional Performance Modelling with the TIPPtool Performance Evaluation, vol. 39, nos. 1-4, pp. 5-35, 2000.
[42] J. Hillston and N. Thomas, Product Form Solution for a Class of PEPA Models Performance Evaluation, vol. 35, no. 3, pp. 171-192, 1999.
[43] J.E. Hopcroft and J.D. Ullman, Introduction to Automata Theory, Languages and Computations. Addison Wesley, 1979.
[44] ITU-Telecommunication Standardization Sector, Message Sequence Charts ITU-T Recommentation Z. 120(11/99), 1999.
[45] JavaSim,http:/javasim.ncl.ac.uk/, 1998.
[46] L. Kleinrock, Queueing Systems, Volume 1: Theory. Wiley, 1975.
[47] K. Kant, Introduction to Computer System Performance Evaluation. McGraw-Hill, 1992.
[48] P. Kähkipuro, UML-Based Performance Modeling Framework for Component-Based Distributed Systems Proc. Performance Eng. Conf., pp. 167-184, 2001.
[49] P. King and R. Pooley, Derivation of Petri Net Performance Models from UML Specifications of Communication Software Proc. 25th UK Performance Eng. Workshop, 1999.
[50] J.G. Kemeny and J.L. Snell, Finite Markov Chains. Springer, 1976.
[51] C. Lindemann, A. Thümmler, A. Klemm, M. Lohmann, and O.P. Waldhorst, Performance Analysis of Time-Enhanced UML Diagrams Based on Stochastic Processes ACM Proc. Int'l Workshop Software and Performance, pp. 25-34, 2002.
[52] E.D. Lazowska, J. Zahorjan, G.S. Graham, and K.C. Sevcik, Quantitative System Performance: Computer System Analysis Using Queueing Network Models. Prentice-Hall, 1984.
[53] R. Milner, Communication and Concurrency. Prentice-Hall, 1989.
[54] D.A. Menascé, A Framework for Software Performance Engineering of Client/Server Systems Proc. 1997 Computer Measurement Group Conf., 1997.
[55] D.A. Menascé and H. Gomaa, On a Language Based Method for Software Performance Engineering of Client/Server Systems ACM Proc. Int'l Workshop Software and Performance, pp. 63-69, 1998.
[56] D.A. Menascé and H. Gomaa, A Method for Design and Performance Modeling of Client/Server Systems IEEE Trans. Software Eng., vol. 26, no. 11, pp. 1066-1085, Nov. 2000.
[57] OPNET Manuals, Mil 3, Inc., 1999.
[58] Developer 5.1 Reference Manual, ObjecTime Ltd., Ottawa, Ont., 1998.
[59] R. Pooley, Using UML to Derive Stochastic Process Algebra Models Proc. 25th UK Performance Eng. Workshop, pp. 23-34, 1999.
[60] R. Pooley and P. King, The Unified Modeling Language and Performance Engineering Proc. IEE Software, pp. 2-10, 1999.
[61] Petri nets tools database,http://www.daimi.au.dkPetriNets, 2004.
[62] D.C. Petriu and H. Shen, Applying UML Performance Profile: Graph Grammar-Based Derivation of LQN Models from UML Specifications Proc. Seventh Int'l Conf. Modelling Techniques and Tools for Performance Evaluation, pp. 159-177, 2002.
[63] D.C. Petriu and C.M. Woodside, Software Performance Models from System Scenarios in Use Case Maps Proc. 12th Int'l Conf. Modelling Tools and Techniques for Computer and Comm. System Performance Evaluation, pp. 141-1158, 2002.
[64] D.C. Petriu and X. Wang, From UML Descriptions of High-Level Software Architectures to LQN Performance Models Proc. Applications of Graph Transformations with Industrial Relevance Workshop (AGTIVE'99), pp. 47-62, 1999.
[65] W. Reisig, Petri Nets: An Introduction EATCS Monographs on Theoretical Computer Science, vol. 4, 1985.
[66] J. Rambaugh, M. Blaha, W. Premerlani, F. Eddy, and W. Lorensen, Object-Oriented Modeling and Design. Prentice-Hall, 1991.
[67] J. Rolia and K. Sevcik, “The Method of Layers,” IEEE Tran. Software Eng., vol. 21, no. 8, pp. 689–700, Aug. 1995.
[68] C.U. Smith, Performance Engineering of Software Systems. Addison Wesley, 1990.
[69] M. Sereno, Towards a Product Form Solution for Stochastic Process Algebras Computer J., vol. 38, no. 7, pp. 622-632, 1995.
[70] C.U. Smith and L.G. Williams, Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software. Addison Wesley, 2002.
[71] C.U. Smith and L.G. Williams, Performance Engineering Evaluation of Object-Oriented Systems with SPE ED Springer LNCS 1245, pp. 135-153, 1997.
[72] K.S. Trivedi, Probability and Statistics with Reliability, Queuing, and Computer Science Applications. John Wiley and Sons, 2001.
[73] C.M. Woodside, C. Hrischuk, B. Selic, and S. Brayarov, Automated Performance Modeling of Software Generated by a Design Environment Performance Evaluation, vol. 45, pp. 107-123, 2001.
[74] C.M. Woodside, J.E. Neilson, D.C. Petriu, and S. Majumdar, “The Stochastic Rendezvous Network Model for the Performance of Synchronous Client-Server-Like Distributed Software,” IEEE Trans. Computer, vol. 44, no. 1, pp. 20–34, Jan. 1995.
[75] Proc. ACM Int'l Workshop Software and Performance, 1998.
[76] Proc. ACM Int'l Workshop Software and Performance, 2000.
[77] Proc. ACM Int'l Workshop Software and Performance, 2002.
[78] L.G. Williams and C.U. Smith, Performance Evaluation of Software Architectures Proc. ACM Int'l Workshop Software and Performance, pp. 164-177, 1998.
[79] L.G. Williams and C.U. Smith, PASA: A Method for the Performance Assesment of Software Architectures Proc. ACM Int'l Workshop Software and Performance, pp. 179-189, 2002.
[80] Unified Modeling Language (UML) version 1.5, OMG Documentation,http://www.omg.org/technology/documents/ formal uml.htm, 2003.
[81] Object Management Group, UML Profile for Schedulability, Performance, and Time OMG document ptc/2002-03-02,http://www.omg.org/cgi-bin/doc?ptc2002-03-02 , 2002.
[82] Extensible Markup Language (XML) 1.0 second ed., W3C Recommendation 6, Oct. 2000, http://www.w3.org/TR/2000REC-xml-20001006 .

Index Terms:
Software verification, performance modeling and prediction, integrated environments.
Citation:
Simonetta Balsamo, Antinisca Di Marco, Paola Inverardi, Marta Simeoni, "Model-Based Performance Prediction in Software Development: A Survey," IEEE Transactions on Software Engineering, vol. 30, no. 5, pp. 295-310, May 2004, doi:10.1109/TSE.2004.9
Usage of this product signifies your acceptance of the Terms of Use.