This Article 
 Bibliographic References 
 Add to: 
Trace Analysis for Conformance and Arbitration Testing
November 1989 (vol. 15 no. 11)
pp. 1347-1356

The authors explore a testing approach where the concern for selecting the appropriate test input provided to the implementation under test (IUT) is separated as much as possible from the analysis of the observed output. Particular emphasis is placed on the analysis of the observed interactions of the IUT in order to determine whether the observed input/output trace conforms to the IUT's specification. The authors consider this aspect of testing with particular attention to testing of communication protocol implementations. Various distributed test architectures are used for this purpose, where partial input/output traces are observable by local observers at different interfaces. The error-detection power of different test configurations is determined on the basis of the partial trace visible to each local observer and their global knowledge about the applied test case. The automated construction of trace analysis modules from the formal specification of the protocol is also discussed. Different transformations of the protocol specification may be necessary to obtain the reference specification, which can be used by a local or global observer for checking the observed trace. Experience with the construction of an arbiter for the OSI (open systems interconnection) transport protocol is described.

[1] A. V. Aho and J. D. Ullman,Principles of Compiler Design. Reading, MA: Addison-Wesley, 1977.
[2] J. M. Ayache, P. Azema, and M. Diaz, "Observer: A concept for on-line detection for control errors in concurrent systems," inProc. 9th Int. Symp. FTC, Madison, WI, June 1979.
[3] G.v. Bochmann, E. Cerny, M. Maksud, and B. Sarikaya, "Testing of transport protocol implementations," inProc. CIPS Conf., Ottawa, 1983, pp. 123-129.
[4] G. v. Bochmann, "Usage of protocol development tools: The result of a survey," inProtocol Specification, Testing and Verification VII. Amsterdam, The Netherlands: North-Holland, 1987, pp. 147-170.
[5] G. von Bochmann, G. Gerber, and J. M. Serre, "Semi-automatic implementation of communication protocols,"IEEE Trans. Software Eng., vol. 13, no. 9, pp. 989-1000, Sept. 1987.
[6] G.v. Bochmann, C. He, D. Ouimet, and R. Zhao, "Protocol testing using automatic trace analysis," inProc. IEEE Canadian Conf. Electrical and Computer Engineering, Sept. 1989.
[7] G. v. Bochmann, "Protocol specification for OSI,"Computer Networks ISDN Syst., vol. 18, pp. 167-184, 1989/1990.
[8] S. S. Lam and A. U. Shankar, "Protocol verification via projections,"IEEE Trans. Software Eng., vol. SE-10, no. 4, pp. 325-342, July 1984.
[9] G.v. Bochmann and O. Bellal, "Test result analysis in respect to formal specifications," submitted for publication.
[10] I. C. Davidson, "The NCC protocol testing service," inProc. Work shop Introduction of High Level Protocol Standards for OSI (BNI/AFNOR), Paris, June 1983, pp. 273-279.
[11] R. Dssouli and G. v. Bochmann, "Conformance testing with multiple observers," inVI IFIP WG6.1 Workshop, G. v. Bochmann and B. Sarikaya, Eds., Gray Rocks, Montreal, June 1986.
[12] R. Dssouli and G.v. Bochmann, "Conformance testing with multiple observers," inProc. IFIP Workshop Protocol Specification, Testing, and Validation, 1986, pp. 217-229.
[13] R. Dssouli, "Etude des methodes de test pour les implantations de protocoles de communication basees sur les specifications formelles," Ph.D. dissertation, Univ. Montreal, 1986.
[14] ISO IS9074, "Estelle: A formal description technique based on an extended state transition model," 1989.
[15] C. Jard and G.v. Bochmann, "An approach to testing specifications,"J. Syst. Software, vol. 3, no. 4, pp. 315-323, Dec. 1983.
[16] L. Lamport, "Time, clocks, and the ordering of events in a distributed system,"Commun. ACM, vol. 21, no. 7, pp. 558-565, July 1978.
[17] L. Logrippo, A. Obaid, J. P. Braind, and M. C. Fehri, "An interpreter for LOTOS, A specification language for distributed systems,"Software--Practice and Exper., vol. 18, pp. 365-385, Apr. 1988.
[18] ISO IS8807, "LOTOS: A formal description technique," 1989.
[19] P. Merlin and G. Bochmann, "On the construction of submodule specifications and communication protocols,"ACM TOPLAS, vol. 5, no. 1, pp. 1-25, 1983.
[20] R. Molva, M. Diaz, and J. M. Ayache, "Observer: A run time checking tool for local area networks,"5th IFIP Workshop Protocol Specification, Testing and Verification, Toulouse, 1985.
[21] ISO TC97/SC21, DIS 9646/1, and DIS 9646/2, "OSI Conformance testing methodology and framework, Part 1: General Concepts, Part 2: Abstract Test Suite Specification," 1989.
[22] ISO TC97/SC6, IS 8073, "OSI--Connection oriented transport protocol specification."
[23] D. Rafiq, R. Castanet, C. Chraibi, J. P. Goursaud, J. Haddad, and X. Perdu, "Towards and environment for testing OSI protocols," inProc. IFIP Workshop Protocol Specification, Verification and Testing, Toulouse, 1985.
[24] D. Rayner, "A system for testing protocol implementations,"Comput. Networks, vol. 6, no. 6, Dec. 1982.
[25] CCITT SG X1, Recommendation Z.100, 1987.
[26] B. Sarikaya and G.v. Bochmann, "Synchronization and specification issues in protocol testing,"IEEE Trans. Commun., vol. COM-32, pp. 389-395, Apr. 1984.
[27] B. Sarikayaet al., "A test design methodology for protocol testing,"IEEE Trans. Software Eng., vol. SE-13, no. 5, pp. 518-531, May 1987.
[28] H. Ural and R. L. Probert, "Automated testing of protocol specifications and their implementations," inProc. ACM SIGCOMM Symp., 1984.
[29] C. Vissers, "Formal description techniques for OSI," inProc. IFIP Congress, Information Processing '86. Amsterdam, The Netherlands: North-Holland, 1986.
[30] H. X. Zeng and D. Rayner, "The impact of the ferry concept on protocol testing," inProtocol Specification, Testing and Verification (IFIP Workshop), M. Diaz, Ed. Amsterdam, The Netherlands: North-Holland, 1986, pp. 533-544.

Index Terms:
conformance testing; arbitration testing; implementation under test; IUT; communication protocol implementations; distributed test architectures; partial input/output traces; local observers; error-detection power; global knowledge; automated construction; trace analysis modules; formal specification; reference specification; OSI; open systems interconnection; transport protocol; conformance testing; open systems; program testing; protocols
G.V. Bochmann, R. Dssouli, J.R. Zhao, "Trace Analysis for Conformance and Arbitration Testing," IEEE Transactions on Software Engineering, vol. 15, no. 11, pp. 1347-1356, Nov. 1989, doi:10.1109/32.41328
Usage of this product signifies your acceptance of the Terms of Use.