Subscribe

Issue No.03 - March (2013 vol.39)

pp: 292-304

Laura Carnevali , Università degli Studi di Firenze, Firenze

Lorenzo Ridi , Università degli Studi di Firenze, Firenze

Enrico Vicario , Università degli Studi di Firenze, Firenze

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TSE.2012.42

ABSTRACT

In the process of testing of concurrent timed systems, input generation identifies values of temporal parameters that let the Implementation Under Test (IUT) execute selected cases. However, when some parameters are not under control of the driver, test execution may diverge from the selected input and produce an inconclusive behavior. We formulate the problem on the basis of an abstraction of the IUT which we call partially stochastic Time Petri Net (psTPN), where controllable parameters are modeled as nondeterministic values and noncontrollable parameters as random variables with general (GEN) distribution. With reference to this abstraction, we derive the analytical form of the probability that the IUT runs along a selected behavior as a function of choices taken on controllable parameters. In the applicative perspective of real-time testing, this identifies a theoretical upper limit on the probability of a conclusive result, thus providing a means to plan the number of test repetitions that are necessary to guarantee a given probability of test-case coverage. It also provides a constructive technique for an optimal or suboptimal approach to input generation and a way to characterize the probability of conclusive testing under other suboptimal strategies.

INDEX TERMS

Stochastic processes, Timing, Real time systems, Testing, Tin, Vectors, Automata, Difference Bound Matrix, Real-time testing, input generation, Time Petri Nets, non-Markovian Stochastic Petri Nets, stochastic processes

CITATION

Laura Carnevali, Lorenzo Ridi, Enrico Vicario, "A Quantitative Approach to Input Generation in Real-Time Testing of Stochastic Systems",

*IEEE Transactions on Software Engineering*, vol.39, no. 3, pp. 292-304, March 2013, doi:10.1109/TSE.2012.42REFERENCES

- [1] R. Alur and D.L. Dill, "A Theory of Timed Automata,"
Theoretical Computer Science, vol. 126, no. 2, pp. 183-235, 1994.- [2] B. Beizer,
Black-Box Testing: Techniques for Functional Testing of Software and Systems. Wiley, 1995.- [3] S. Bernardi, J. Campos, and J. Merseguer, "Timing-Failure Risk Assessment of UML Design Using Time Petri Net Bound Techniques,"
IEEE Trans. Industrial Informatics, vol. 7, no. 1, pp. 90-104, Feb. 2011.- [4] J. Berntsen, T.O. Espelid, and A. Genz, "An Adaptive Algorithm for the Approximate Calculation of Multiple Integrals,"
ACM Trans. Math. Software, vol. 17, pp. 437-451, Dec. 1991.- [5] B. Berthomieu and M. Diaz, "Modeling and Verification of Time Dependent Systems Using Time Petri Nets,"
IEEE Trans. Software Eng., vol. 17, no. 3, pp. 259-273, Mar. 1991.- [6] G. Bucci, L. Carnevali, L. Ridi, and E. Vicario, "Oris: A Tool for Modeling Verification and Evaluation of Real-Time Systems,"
Int'l J. Software Tools for Technology Transfer, vol. 12, no. 5, pp. 391-403, 2010.- [7] L. Carnevali, L. Grassi, and E. Vicario, "A Tailored V-Model Exploiting the Theory of Preemptive Time Petri Nets,"
Proc. 13th Ada-Europe Int'l Conf. Reliable Software Technologies, pp. 87-100, 2008.- [8] L. Carnevali, L. Grassi, and E. Vicario, "State-Density Functions over DBM Domains in the Analysis of Non-Markovian Models,"
IEEE Trans. Software Eng., vol. 35, no. 2, pp. 178-194, Mar./Apr. 2009.- [9] L. Carnevali, L. Ridi, and E. Vicario, "Partial Stochastic Characterization of Timed Runs over DBM Domains,"
Proc. Ninth Int'l Workshop Performability Modeling of Computer and Comm. Systems, Sept. 2009.- [10] L. Carnevali, L. Ridi, and E. Vicario, "A Framework for Simulation and Symbolic State Space Analysis of Non-Markovian Models,"
Proc. 30th Int'l Conf. Computer Safety, Reliability, and Security, pp. 409-422, 2011.- [11] L. Carnevali, L. Ridi, and E. Vicario, "Sirio: A Framework for Simulation and Symbolic State Space Analysis of Non-Markovian Models,"
Proc. Eighth Int'l Conf. Quantitative Evaluation of Systems, pp. 153-154, 2011.- [12] L. Carnevali, L. Sassoli, and E. Vicario, "Sensitization of Symbolic Runs in Real-Time Testing Using the ORIS Tool,"
Proc. IEEE Conf. Emerging Technologies and Factory Automation, Sept. 2007.- [13] T.S. Chow, "Testing Software Design Modeled by Finite-State Machines,"
IEEE Trans. Software Eng., vol. 4, no. 3, pp. 178-187, May 1978.- [14] G.B. Dantzig and B.C. Eaves, "Fourier-Motzkin Elimination and Its Dual,"
J. Combinatorial Theory, vol. 14, no. 3, pp. 288-297, 1973.- [15] D. Dill, "Timing Assumptions and Verification of Finite-State Concurrent Systems,"
Proc. Workshop Computer Aided Verification Methods for Finite State Systems, 1989.- [16] A. En-Nouaary, R. Dssouli, and F. Khendek, "Timed WP-Method: Testing Real-Time Systems,"
IEEE Trans. on Software Eng., vol. 28, no. 11, pp. 1023-1038, Nov. 2002.- [17] S. Fujiwara, G. Bochmann, F. Khendek, M. Amalou, and A. Ghedamsi, "Test Selection Based on Finite-State Models,"
IEEE Trans. Software Eng., vol. 17, no. 2, pp. 591-603, June 1991.- [18] G. Karsai, J. Sztipanovits, A. Ledeczi, and T. Bapty, "Model-Integrated Development of Embedded Software,"
Proc. IEEE, vol. 91, no. 1, pp. 145-164, Jan. 2003.- [19] A. Hessel, K. Larsen, M. Mikucionis, B. Nielsen, P. Pettersson, and A. Skou, "Testing Real-Time Systems Using UPPAAL,"
Formal Methods and Testing, pp. 77-117, Springer, 2008.- [20] A. Hessel and P. Pettersson, "A Global Algorithm for Model-Based Test Suite Generation,"
Electronic Notes in Theoretical Computer Science, vol. 190, no. 2, pp. 47-59, 2007.- [21] R.M. Hierons, K. Bogdanov, J.P. Bowen, R. Cleaveland, J. Derrick, J. Dick, M. Gheorghe, M. Harman, K. Kapoor, P. Krause, G. Lüttgen, A.J.H. Simons, S. Vilkomir, M.R. Woodward, and H. Zedan, "Using Formal Specifications to Support Testing,"
ACM Computer Surveys, vol. 41, no. 2, pp. 1-76, Feb. 2009.- [22] C. Jard and T. Jéron, "TGV: Theory, Principles and Algorithms, a Tool for the Automatic Synthesis of Conformance Test Cases for Non-Deterministic Reactive Systems,"
Int'l J. Software Tools for Technology Transfer, vol. 6, Oct. 2004.- [23] M. Krichen and S. Tripakis, "Black-Box Conformance Testing for Real-Time Systems,"
Proc. 11th Int'l SPIN Workshop Model Checking of Software, 2004.- [24] M. Krichen and S. Tripakis, "An Expressive and Implementable Formal Framework for Testing Real-Time Systems,"
Testing of Communicating Systems, F. Khendek and R. Dssouli, eds., pp. 375-375, Springer 2005.- [25] M. Kwiatkowska, G. Norman, R. Segala, and J. Sproston, "Verifying Quantitative Properties of Continuous Probabilistic Timed Automata,"
Proc. 11th Int'l Conf. Concurrency Theory, pp. 123-137, 2000.- [26] L. Carnevali, L. Ridi, and E. Vicario, "Putting Preemptive Time Petri Nets to Work in a V-Model SW Life Cycle,"
IEEE Trans. Software Eng., vol. 37, no. 6, pp. 826-844, Nov./Dec. 2011.- [27] K.G. Larsen, M. Mikucionis, and B. Nielsen, "Online Testing of Real-Time Systems Using UPPAAL: Status and Future Work,"
Perspectives of Model-Based Testing, Dagstuhl Seminar Proc., E. Brinksma, W. Grieskamp, and J. Tretmans, eds., 2005.- [28] M.G. Merayo, M. Núñez, and I. Rodríguez, "Formal Testing from Timed Finite State Machines,"
Computer Networks, vol. 52, no. 2, pp. 432-460, 2008.- [29] H. Muccini, A. Bertolino, and P. Inverardi, "Using Software Architecture for Code Testing,"
IEEE Trans. Software Eng., vol. 30, no. 3, pp. 160-171, Mar. 2004.- [30] G.J. Myers,
The Art of Software Testing, second ed. John Wiley & Sons, Inc., 2004.- [31] J.A. Nelder and R. Mead, "A Simplex Method for Function Minimization,"
The Computer J., vol. 7, no. 4, pp. 308-313, Jan. 1965.- [32] W. Penczek and A. Polrola, "Specification and Model Checking of Temporal Properties in Time Petri Nets and Timed Automata,"
Proc. 25th Int'l Conf. Application and Theory of Petri Nets, June 2004.- [33] Radio Technical Commission for Aeronautics,
DO-178B, Software Considerations in Airborne Systems and Equipment Certification, 1992.- [34] J. Schmaltz and J. Tretmans, "On Conformance Testing for Timed Systems,"
Proc. Sixth Int'l Conf. Formal Modeling and Analysis of Timed Systems, pp. 250-264, 2008.- [35] D.C. Schmidt, "Model-Driven Engineering,"
Computer, vol. 39, no. 2, pp. 1-2, Feb. 2006.- [36] J. Springintveld, F. Vaandrager, and P. D'Argenio, "Testing Timed Automata,"
Theoretical Computer Science, vol. 254, nos. 1-2, pp. 225-257, 2001 (previously appeared as Technical Report CTIT-97-17, Univ. of Twente, 1997). - [37] J. Tretmans, "Model Based Testing with Labelled Transition Systems,"
Formal Methods and Testing, pp. 1-38, Springer, 2008.- [38] E. Vicario, "Static Analysis and Dynamic Steering of Time Dependent Systems Using Time Petri Nets,"
IEEE Trans. Software Eng., vol. 27, no. 1, pp. 728-748, Aug. 2001.- [39] E. Vicario, L. Sassoli, and L. Carnevali, "Using Stochastic State Classes in Quantitative Evaluation of Dense-Time Reactive Systems,"
IEEE Trans. Software Eng., vol. 35, no. 5, pp. 703-719, Sept./Oct. 2009.- [40] R. Wilhelm, J. Engblom, A. Ermedahl, N. Holsti, S. Thesing, D. Whalley, G. Bernat, C. Ferdinand, R. Heckmann, T. Mitra, F. Mueller, I. Puaut, P. Puschner, J. Statshulat, and P. Stenstroem, "Priority Inheritance Protocols: The Worst Case Execution-Time problem: Overview of Methods and Survey of Tools,"
ACM Trans. Embedded Computing Systems, vol. 7, no. 3, pp. 1-53, 2008.- [41] Wolfram Connection Tech nologies,
Java Toolkit: J/Link, http://www.wolfram.com/solutions/mathlink jlink/, 2012.- [42] Wolfram Research, www.wolfram.com,
Mathemathica 5.2, 2012.- [43] N. Wolovick, P. D'Argenio, and H. Qu, "Optimizing Probabilities of Real-Time Test Case Execution,"
Proc. Int'l Conf. Software Testing Verification and Validation, pp. 1-10, Apr. 2009.- [44] H.L.S. Younes and R.G. Simmons, "Solving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions,"
Proc. 19th Nat'l Conf. Artificial Intelligence, pp. 742-747, 2004. |