This Article 
 Bibliographic References 
 Add to: 
Use of Sequencing Constraints for Specification-Based Testing of Concurrent Programs
June 1998 (vol. 24 no. 6)
pp. 471-490

Abstract—This paper presents and evaluates a specification-based methodology for testing concurrent programs. This methodology requires sequencing constraints, which specify restrictions on the allowed sequences of synchronization events. Sequencing constraints for a concurrent program can be derived from the program's formal or informal specification. Details of the proposed testing methodology based on the use of Constraints on Succeeding and Preceding Events (CSPE) are given. How to achieve coverage and detect violations of CSPE constraints for a concurrent program, according to deterministic and nondeterministic testing of this program, are described. A coverage criterion for CSPE-based testing is defined and analyzed. The results of empirical studies of CSPE-based testing for four concurrent problems are reported. These results indicate that the use of sequencing constraints for specification-based testing of concurrent programs is a promising approach.

[1] M.A. Vouk, D. McAllister, and K.C. Tai, "An Experimental Evaluation of the Effectiveness of Random Testing of Fault-Tolerant Software," Proc. ACM Software Testing Workshop, pp. 74-81, July 1986.
[2] B. Beizer, Software Testing Techniques, second ed. Boston, Mass.: Int'l Thomson Computer Press, 1990.
[3] R.N. Taylor, D.L. Levine, and C.D. Kelly, "Structural Testing of Concurrent Programs," IEEE Trans. Software Eng., vol. 18, no. 3, pp. 206-215, Mar. 1992.
[4] R.D. Yang and C.G. Chung, "Path Analysis Testing of Concurrent Programs," Information and Software Technology, vol. 34. no. 1, pp. 43-56, Jan. 1992.
[5] S.K. Damodaran-Kamal and J.M. Francioni, "Nondeterminancy: Testing and Debugging in Message Passing Parallel Programs," Proc. ACM/ONR Workshop Parallel and Distributed Debugging, ACM SIGPLAN Notices, vol. 28, no. 12, pp. 118-128, Dec. 1993.
[6] G.H. Hwang, K.C. Tai, and T.L. Huang, "Reachability Testing: An Approach to Testing Concurrent Software," Proc. Int'l J. Software Eng. and Knowledge Eng., vol. 5, no. 4, pp. 493-510, Dec. 1995.
[7] K.C. Tai, "Reachability Testing of Asynchronous Message-Passing Programs," Proc. IEEE Int'l Workshop Software Eng. for Parallel and Distributed Systems, pp. 50-61, May 1997.
[8] G.J. Holzmann, Design and Validation of Computer Protocols, Prentice Hall, 1991.
[9] G. v. Bochmann and A. Petrenko, "Protocol Testing: Review of Methods and Relevance for Software Testing," Proc. ACM Symp. Software Testing and Analysis, pp. 109-124 1994.
[10] L.K. Dillon, G. Kutty, L.E. Moser, P.M. Melliar-Smith, and Y.S. Ramakrishna, "Graphical Specifications for Concurrent Software Systems," Proc. 14th Int'l Conf. Software Eng., pp. 214-224, 1992.
[11] D. Helmbold and D. Luckham, "TSL: Task Sequencing Language," Proc. Ada Int'l Conf., pp. 255-274, 1985.
[12] L.K. Dillon and Q. Yu, “Oracles for Checking Temporal Properties of Concurrent Systems,” Proc. Symp. Foundations of Software Eng., pp. 140–153, Dec. 1994.
[13] D. Helmbold and D. Luckham, "Debugging Ada Tasking Programs," IEEE Software, vol. 2, no. 2, pp. 47-57, Mar. 1985.
[14] K.C. Tai and R.H. Carver, "Testing of Distributed Programs," ch. 33 of Handbook of Parallel and Distributed Computing, A. Zomaya, ed., pp. 955-978. McGraw-Hill, 1996.
[15] R.H. Carver and K.C. Tai, "Replay and Testing for Concurrent Programs," IEEE Software, pp. 66-74, Mar. 1991.
[16] K.C. Tai, R.H. Carver, and E.E. Obaid, "Debugging Concurrent Ada Programs by Deterministic Execution," IEEE Trans. Soft. Eng., vol. 17, no. 1, pp. 45-63, Jan. 1991.
[17] T.J. LeBlanc and J.M. Mellor-Crummey, "Debugging Parallel Programs with Instant Replay," IEEE Trans. Computers, vol. 36, no. 4, pp. 471-482, Apr. 1987.
[18] R.H. Carver and K.C. Tai, "Static Analysis of Concurrent Software for Deriving Synchronization Constraints," Proc. IEEE Int'l Conf. Distributed Computing Systems, pp. 544-551, May 1991.
[19] R.H. Carver, "Testing Abstract Distributed Programs and Their Implementations," J. Systems and Software, special issue on Software Eng. for Distributed Computing, pp. 223-237, June 1996.
[20] G. Bruns, Distributed Systems Analysis with CCS. Prentice Hall, 1997.
[21] K.C. Tai and R.H. Carver, "A Specification-Based Methodology for Testing Concurrent Programs," Proc. Europe Software Engineering Conf., Lecture Notes in Computer Science 989, W. Schafer and P. Botella, eds., pp. 154-172. Springer-Verlag, 1995.
[22] R.H. Carver and J. Chen, "Incremental Conformance Testing Using Lotos Specifications," Proc. Fifth Int'l Conf. Computer Comm. and Networks, pp. 42-47, Oct. 1996.
[23] P.V. Koppol and K.C. Tai, "An Incremental Approach to Structural Testing of Concurrent Software" ACM Int'l Symp. Software Testing and Analysis, pp. 14-23, Jan. 1996.
[24] A.J. Offutt, C.Z. Rothermel, and C. Zapf, "An Experimental Evaluation of Selective Mutation," Proc. Int'l Conf. Software Eng., pp. 100-107, 1993.
[25] K.C. Tai, "A Graphical Representation of Rendezvous Sequences of Concurrent Ada Programs," ACM Ada Letters, vol. VI, no. 1, pp. 94-103, Jan./Feb. 1986.
[26] Using Formal Description Techniques: An Introduction to Estelle, Lotos, and SDL, K.J. Turner, ed., John Wiley&Sons, 1993.
[27] R.H. Carver and R. Durham, "Integrating Formal Methods and Testing for Concurrent Programs," Proc. 10th IEEE Conf. Computer Assurance, pp. 25-33, June 1995.
[28] K.C. Tai and Y.C. Young, "Synchronizable Test Sequences of Finite State Machines," 1998. to appear in Computer Network and ISDN Systems
[29] P. Tripathy and K. Naik, "Generation of Adaptive Tests from Nondeterministic Finite State Models," Proc. Protocol Test Systems, V, pp. 309-320, 1992.
[30] J. Arjjo, "On the Existence and Production of State Identification Machines for Labeled Transition Systems," Proc. Formal Description Techniques, VI, pp. 351-366, 1993.
[31] P.V. Koppol and K.C. Tai, "Conformance Testing of Protocols Specified as Labeled Transitions Systems," Proc. IFIP Eighth Int'l Workshop on Protocol Test Systems, A. Cavalli and S. Budkowski, eds., pp. 135-150, Chapman and Hall, 1996.
[32] G. v. Bochmann, R. Dssouli, and J. Zhao, "Trace Analysis for Conformance and Arbitration Testing," IEEE Trans. Software. Eng., vol. 15, no. 11, pp. 1,347-1,356, Nov. 1989.
[33] Y.C. Young and K.C. Tai, "Observation Inaccuracy in Conformance Testing with Multiple Testers," Proc. IEEE First Workshop on Application-Specific Software Eng. and Technology, pp. 77-82, Mar. 1998.
[34] D. Rosenblum, "Specifying Concurrent Systems with TSL," IEEE Software, pp. 52-61, May 1991.
[35] E.J. Weyuker, T. Goradia, and A. Singh, "Automatically Generating Test Data from Boolean Specification," IEEE Trans. Software Eng., vol. 20, no. 5, pp. 353-363, May 1994.
[36] A.M. Pardkar and K.C. Tai, "Test Generation for Boolean Expressions," Proc. Int'l Symp. Software Reliability Eng., pp. 106-115, 1995.
[37] A.M. Paradkar, K.C. Tai, and M.A. Vouk, "Automatic Test Generation for Predicates," IEEE Trans. Reliability, pp. 515-530, Dec. 1996.
[38] K.C. Tai, "Theory of Fault-Based Predicate Testing for Computer Programs," IEEE Trans. Software Eng., vol. 22, no. 8, pp. 552-562, Aug. 1996.
[39] P.C. Bates and J.C. Wileden, "High Level Debugging of Distributed Systems: The Behavior Abstraction Approach," J. Software&Systems, vol. 3, pp. 255-264, 1984.
[40] W.S. Cheng and V.E. Wallentine, "DEBL: A Knowledge-Based Language for Specifying and Debugging Distributed Programs," Comm. ACM, vol. 32, no. 9, pp. 1,079-1,084, Sept. 1989.
[41] C. Stirling, "An Introduction to Modal and Temporal Logics for CCS," Lecture Notes in Computer Science 491, pp. 1-20. Springer-Verlag, 1991.
[42] R. Cleaveland, J. Parrow, and B. Steffen, "The Concurrency Workbench: A Semantics-Based Tool for the Verification of Concurrent Systems," ACM Trans. Programming Languages and Systems, Jan. 1993, pp. 36-72.

Index Terms:
Software testing, specification-based testing, concurrent programs, sequencing constraints.
Richard H. Carver, Kuo-Chung Tai, "Use of Sequencing Constraints for Specification-Based Testing of Concurrent Programs," IEEE Transactions on Software Engineering, vol. 24, no. 6, pp. 471-490, June 1998, doi:10.1109/32.689403
Usage of this product signifies your acceptance of the Terms of Use.