This Article 
 Bibliographic References 
 Add to: 
Incremental Integration Testing of Concurrent Programs
June 2002 (vol. 28 no. 6)
pp. 607-623

Abstract—We present a method for selecting test sequences for concurrent programs from labeled transitions systems (LTS). A common approach to selecting test sequences from a set of LTSs is to derive a global LTS, called the reachability graph, and then force deterministic program executions according to paths selected from the graph. However, using a reachability graph for test path selection introduces a state explosion problem. To overcome this problem, a reduced graph can be generated using incremental reachability analysis, which consists of repeatedly generating a reachability graph for a subset of LTSs, reducing this graph, and using the reduced graph in place of the original LTSs. Unfortunately, existing incremental reachability analysis techniques generate reduced graphs with insufficient information for deterministic testing. In this paper, we present an incremental approach to testing concurrent programs. Incremental testing consists of incremental reachability analysis for test path selection and deterministic testing for test execution. We define a new type of reachability graph for incremental analysis, called an annotated labeled transition system (ALTS). An ALTS is an LTS annotated with information necessary for deterministic testing. We propose practical coverage criteria for selecting tests paths from an ALTS and present an ALTS reduction algorithm. The results of several case studies are reported.

[1] Ada 95 Reference Manual, Int'l Standard ANSI/ISO/IEC-8652:1995, Jan. 1995.
[2] A. Ates and B. Sarikaya, “Test Sequence Generation and Timed Testing,” Computer Networks and ISDN Systems, vol. 29, pp. 101-131, 1996.
[3] K. Bartlett, R. Scantlebury, and P. Wilkinson, "A Note on Reliable Full-Duplex Transmission over Half-Duplex Links," Comm. ACM, vol. 12, no. 5, pp. 260-261, May 1969.
[4] R.H. Carver, "Testing Abstract Distributed Programs and Their Implementations," J. Systems and Software, special issue on Software Eng. for Distributed Computing, pp. 223-237, June 1996.
[5] J.C. Corbett, M.B. Dwyer, J. Hatcliff, S. Laubach, C.S. Pasareanu, Robby, and H. Zheng, “Bandera: Extracting Finite-State Models from Java Source Code,” Proc. 22nd Int'l Conf. Software Eng., pp. 439–448, June 2000.
[6] E.M. Clarke, M. Fujita, S.P. Rajan, T. Reps, S. Shankar, and T. Teitelbaum, “Program Slicing of Hardware Description Languages,” Technical Report CMU-CS-99-103, School of Computer Science, Carnegie Mellon Univ., Mar. 1999.
[7] S.C. Cheung and J. Kramer, "Enhancing Compositional Reachability Analysis Using Context Constraints," Proc. First ACM SIGSOFT Symp. Foundations of Software Engineering, pp. 115-125, Dec. 1993.
[8] S.C. Cheung and J. Kramer, “Compositional Reachability Analysis of Finite-State Distributed Systems with User-Specified Constraints,” Proc. Third ACM SIGSOFT. Symp. Foundations of Software Eng., pp. 141-150, 1995.
[9] E.M. Clarke, D.E. Long, and K.L. McMillan, “Compositional Model Checking,” Proc. Fourth Ann. Symp. Logic in Computer Science, June 1989.
[10] J.C. Corbett, “Constructing Compact Models of Concurrent Java Programs,” Proc. ACM Int'l Symp. Software Testing and Analysis (ISSTA'98), pp. 1-10, 1998.
[11] R. Cleaveland, J. Parrow, and B. Steffen, "The Concurrency Workbench: A Semantics-Based Tool for the Verification of Concurrent Systems," ACM Trans. Programming Languages and Systems, Jan. 1993, pp. 36-72.
[12] R.H. Carver and K.-C. Tai, Use of Sequencing Constraints for Specification-Based Testing of Concurrent Programs IEEE Trans. Software Eng., vol. 24, no. 6, pp. 471-490, 1998.
[13] J. Eloranta, “Minimizing the Number of Transitions with Respect to Observational Equivalence,” BIT 31, pp. 576-590, 1991.
[14] J. Eve and R. Kurki-Suonio, “On Computing the Transitive Closure of a Relation,” Acta Inf., vol. 8, pp. 303-314, 1977.
[15] J. Eloranta, M. Tienari, and A. Valmari, “Essential Transitions to Bisimulation Equivalences,” Theoretical Computer Science 179, pp. 397-419, 1997.
[16] S. Graf and B. Steffen, “Compositional Minimization of Finite State Systems,” Proc. Second Conf. Computer-Aided Verification, pp. 186-196, 1990.
[17] P. Godefroid and P. Wolper, "Using Partial Orders for the Efficient Verification of Deadlock Freedom and Safety Properties," Formal Methods in System Design, pp. 149-164, Apr. 1993.
[18] C.A.R. Hoare,“Communicating sequential processes,” Comm. of the ACM, vol. 21, no. 8, pp. 666-677, Aug. 1978.
[19] C.A.R. Hoare, Communicating Sequential Processes, Prentice Hall, Englewood Cliffs, N.J., 1985.
[20] G.J. Holzmann, P. Godefroid, and D. Pirottin, “Coverage Preserving Reduction Strategies for Reachability Analysis,” Proc. 12th Int'l Symp. Protocol Specification, Testing, and Verification, pp. 349-363, 1992.
[21] ITU-T, “Section 5, Draft Recommendation Q.2931 - User Network Interface Layer 3 Specification for the Basic Call/Connection Control,” ITU-Telecomm. Standardization Sector, June 1994.
[22] P.V. Koppol, R.H. Carver, and K.C. Tai, “Incremental Integration Testing of Concurrent Programs,” Technical Report TR00-07, George Mason Univ., 2000.
[23] P.V. Koppol, “Incremental Analysis and Testing of Distributed Software Systems,” PhD Thesis, Dept. of Computer Science, North Carolina State Univ., Raleigh, NC 1996.
[24] P.C. Kanellakis and S.A. Smolka, "CCS Expressions, Finite State Processes, and Three Problems of Equivalence," Information and Computation, pp. 43-68, vol. 86, 1990.
[25] B. Karaçali and K.C. Tai, “Model Checking Based on Simultaneous Reachability Analysis,” Proc. Seventh Int'l SPIN Workshop Model Checking of Software, pp. 34-53, 2000.
[26] P.V. Koppol and K.C. Tai, “Conformance Testing of Protocols Specified as Labeled Transition Systems,” Proc. Eighth Int'l Workshop Protocol Test Systems, pp. 143-158, 1995.
[27] P.V. Koppol and K.C. Tai, "An Incremental Approach to Structural Testing of Concurrent Software" ACM Int'l Symp. Software Testing and Analysis, pp. 14-23, Jan. 1996.
[28] K.K. Sabnani, A.M. Lapone, and M.U. Uyar, "An Algorithm Procedure for Checking Safety Properties of Protocols," IEEE Trans. Comm., pp. 940-948, vol. 37, no. 9, 1989.
[29] R. Milner, Communication and Concurrency, Prentice-Hall, Englewood Cliffs, N.J., 1989.
[30] E. Nuutila, “An Efficient Transitive Closure Algorithm for Cyclic Digraphs,” Information Processing Letters 52, pp. 207-213, 1994.
[31] M. Pezzè, R.N. Taylor, and M. Young, "Graph Models for Reachability Analysis of Concurrent Programs," ACM Trans. Software Eng. and Methodology, vol. 4, no. 2, pp. 171-213, Apr. 1995.
[32] K.C. Tai and R.H. Carver, “Testing of Distributed Programs,” Parallel and Distributed Computing Handbook, A.Y. Zomaya, ed., pp. 955-978, 1996.
[33] K.C. Tai, R.H. Carver, and E.E. Obaid, "Debugging Concurrent Ada Programs by Deterministic Execution," IEEE Trans. Soft. Eng., vol. 17, no. 1, pp. 45-63, Jan. 1991.
[34] K.C. Tai and P.V. Koppol, “Hierarchy-Based Incremental Analysis of Communication Protocols,” Proc. Int'l Conf. Network Protocols, pp. 318-325, 1993.
[35] Using Formal Description Techniques: An Introduction to Estelle, Lotos, and SDL, K.J. Turner ed., Wiley, 1993.
[36] R.N. Taylor, D.L. Levine, and C.D. Kelly, "Structural Testing of Concurrent Programs," IEEE Trans. Software Eng., vol. 18, no. 3, pp. 206-215, Mar. 1992.
[37] M. Weiser, “Program Slicing,” IEEE Trans. Software Eng., vol. 19, no. 4, pp. 352-357, July 1984.
[38] M. Young, R.N. Taylor, D.L. Levine, K.A. Nies, and D. Brodbeck, "A Concurrency Analysis Tool Suite: Rationale, Design, and Preliminary Experience," ACM Trans.Software Eng. and Methodology, vol. 4, no. 1, pp. 64-106, Jan. 1995.
[39] W.J. Yeh and M. Young, "Compositional Reachability Analysis Using Process Algebra," Proc. ACM SIGSOFT Symp. Testing, Analysis and Verification, pp. 49-59, Oct. 1991.
[40] W.J. Yeh and M. Young, “Hierarchical Tracing of Concurrent Programs,” Proc. Third Irvine Software Symp., pp. 73-84, 1993.

Index Terms:
Software testing, concurrent programs, structural testing, incremental testing.
Pramod V. Koppol, Richard H. Carver, Kuo-Chung Tai, "Incremental Integration Testing of Concurrent Programs," IEEE Transactions on Software Engineering, vol. 28, no. 6, pp. 607-623, June 2002, doi:10.1109/TSE.2002.1010062
Usage of this product signifies your acceptance of the Terms of Use.