The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan.-March (2012 vol.5)
pp: 116-130
Hong Zhu , Oxford Brookes University, Oxford
Yufeng Zhang , The National University of Defense Technology, Changsha
ABSTRACT
Software testers are confronted with great challenges in testing Web Services (WS) especially when integrating to services owned by other vendors. They must deal with the diversity of implementation techniques used by the other services and to meet a wide range of test requirements. However, they are in lack of software artifacts, the means of control over test executions and observation on the internal behavior of the other services. An automated testing technique must be developed to be capable of testing on-the-fly nonintrusively and nondisruptively. Addressing these problems, this paper proposes a framework of collaborative testing in which test tasks are completed through the collaboration of various test services that are registered, discovered, and invoked at runtime using the ontology of software testing STOWS. The composition of test services is realized by using test brokers, which are also test services but specialized in the coordination of other test services. The ontology can be extended and updated through an ontology management service so that it can support a wide open range of test activities, methods, techniques, and types of software artifacts. The paper presents a prototype implementation of the framework in semantic WS and demonstrates the feasibility of the framework by running examples of building a testing tool as a test service, developing a service for test executions of a WS, and composing existing test services for more complicated testing tasks. Experimental evaluation of the framework has also demonstrated its scalability.
INDEX TERMS
Software engineering, software testing, distributed/internet based software engineering tools and techniques, testing tools, ontology, web services, semantic web services, service composition.
CITATION
Hong Zhu, Yufeng Zhang, "Collaborative Testing of Web Services", IEEE Transactions on Services Computing, vol.5, no. 1, pp. 116-130, Jan.-March 2012, doi:10.1109/TSC.2010.54
REFERENCES
[1] F. McCabe, E. Newcomer, M. Champion, C. Ferris, and D. Orchard, Web Services Architecture, W3C Working Group Note, http://www.w3.org/TRws-arch, 2004.
[2] M. Stal, "Web Services: Beyond Component-Based Computing," Comm. ACM, vol. 45, no. 10, pp. 71-76, Oct. 2002.
[3] G. Canfora and M. Penta, "Service-Oriented Architectures Testing: A Survey," Software Eng.: Int'l Summer Schools (ISSSE 2006-2008), Revised Tutorial Lectures, A. Lucia and F. Ferrucci, eds., pp. 78-105, Springer-Verlag, 2009.
[4] M. Bozkurt, M. Harman, and Y. Hassoun, "Testing Web Services: A Survey," Technical Report TR-10-01, Dept. of Computer Science, King's College London, Jan. 2010.
[5] X. Bai, W. Dong, W. Tsai, and Y. Chen, "WSDL-Based Automatic Test Case Generation for Web Services Testing," Proc. IEEE Int'l Workshop Service Oriented System Eng. (SOSE '05), pp. 215-220, Oct. 2005.
[6] W. Tsai, R. Paul, W. Song, and Z. Cao, "Coyote: An XML-Based Framework for Web Services Testing," Proc. IEEE Int'l Symp. High Assurance Systems Eng. (HASE '02), pp. 173-174, Oct. 2002.
[7] N. Looker, M. Munro, and J. Xu, "WS-FIT: A Tool for Dependability Analysis of Web Services," Proc. 28th Ann. Int'l Computer Software and Applications Conf. (COMPSAC '04), pp. 120-123, Sept. 2004.
[8] J. Offutt and W. Xu, "Generating Test Cases for Web Services Using Data Perturbation," SIGSOFT Software Eng. Notes, vol. 29, no. 5, pp. 1-10, Sept. 2004.
[9] S.C. Lee and J. Offutt, "Generating Test Cases for XML-Based Web Component Interactions Using Mutation Analysis," Proc. 12th Int'l Symp. Software Reliability Eng. (ISSRE '01), pp. 200-209, Nov. 2001.
[10] W. Xu, J. Offutt, and J. Luo, "Testing Web Services by XML Perturbation," Proc. IEEE 16th Int'l Symp. Software Reliability Eng. (ISSRE '05), pp. 257-266, Nov. 2005.
[11] M.P. Emer, S.R. Vergilio, and M. Jino, "A Testing Approach for XML Schemas," Proc. 29th Ann. Int'l Conf. Computer Software and Applications Conf. (COMPSAC '05), pp. 57-62, July 2005.
[12] A. Bertolino, J. Gao, and E. Marchetti, "XML Every-Flavor Testing," Proc. Second Int'l Conf. Web Information Systems and Technologies (WEBIST '06), pp. 268-273, Apr. 2006.
[13] A. Bertolino, J. Gao, E. Marchetti, and A. Polini, "Systematic Generation of XML Instances to Test Complex Software Applications," Rapid Integration of Software Engineering Techniques, N. Guelfi, et al., eds., pp. 114-129, Springer, 2007.
[14] A. Bertolino, J. Gao, E. Marchetti, and A. Polini, "Automatic Test Data Generation for XML Schema-Based Partition Testing," Proc. Second Int'l Workshop Automation of Software Test (AST '07), p. 4, May 2007.
[15] J.B. Li and J. Miller, "Testing the Semantics of W3C XML Schema," Proc. 29th Ann. Int'l Computer Software and Applications Conf. (COMPSAC '05), pp. 443-448, July 2005.
[16] L.F. de Almeida and S.R. Vergilio, "Exploring Perturbation Based Testing for Web Services," Proc. IEEE Int'l Conf. Web Services (ICWS '06), pp. 717-726, Sept. 2006.
[17] S. Hanna and M. Munro, "An Approach for WSDL-Based Automated Robustness Testing of Web Services," Information Systems Development: Challenges in Practice, Theory, and Education, C. Barry, et al., eds., vol. 2, pp. 493-504, Springer, 2009.
[18] J. García-Fanjul, J. Tuya, and C. de la Riva, "Generating Test Cases Specifications for BPEL Compositions of Web Services Using SPIN," Proc. Int'l Workshop Web Services Modeling and Testing (WS-MaTe), 2006.
[19] C. Bartolini, A. Bertolino, E. Marchetti, and I. Parissis, "Data Flow-Based Validation of Web Services Compositions: Perspectives and Examples," Architecting Dependable Systems, R.V. Lemos, et al., eds., pp. 298-325, Springer-Verlag, 2008.
[20] C. Bartolini, A. Bertolino, and E. Marchetti, "Introducing Service-Oriented Coverage Testing," Proc. IEEE/ACM 23rd Int'l Conf. Automated Software Eng. (ASE '08), pp. 57-64, 2008.
[21] K. Kaschner and N. Lohmann, "Automatic Test Case Generation for Services," Proc. Fourth Int'l Workshop Eng. Service-Oriented Applications: Analysis and Design (WESOA '08), Dec. 2008.
[22] M. Lallali, F. Zaidi, A. Cavalli, and I. Hwang, "Automatic Timed Test Case Generation for Web Services Composition," Proc. IEEE Sixth European Conf. Web Services (ECOWS '08), pp. 53-62, 2008.
[23] Z. Li, W. Sun, Z.B. Jiang, and X. Zhang, "BPEL4WS Unit Testing: Framework and Implementation," Proc. IEEE Int'l Conf. Web Services (ICWS '05), pp. 103-110, July 2005.
[24] P. Mayer, "Design and Implementation of a Framework for Testing BPEL Compositions," master's thesis, Leibnitz Univ., Germany 2006.
[25] L. Mei, W.K. Chan, and T.H. Tse, "Data Flow Testing of Service-Oriented Workflow Applications," Proc. ACM/IEEE 30th Int'l Conf. Software Eng. (ICSE '08), pp. 371-380, 2008.
[26] Y. Zheng, J. Zhou, and P. Krause, "An Automatic Test Case Generation Framework for Web Services," J. Software, vol. 2, no. 3, pp. 64-77, 2007.
[27] H. Huang, W. Tsai, R. Paul, and Y. Chen, "Automated Model Checking and Testing for Composite Web Services," Proc. IEEE Eighth Int'l Symp. Object-Oriented Real-Time Distributed Computing (ISORC '05), pp. 300-307, May 2005.
[28] Y. Wang, X. Bai, J. Li, and R. Huang, "Ontology-Based Test Case Generation for Testing Web Services," Proc. IEEE Eighth Int'l Symp. Object-Oriented Real-Time Distributed Computing (ISADS '07), pp. 43-50, 2007.
[29] X. Bai, S. Lee, W.T. Tsai, and Y. Chen, "Ontology-Based Test Modeling and Partition Testing of Web Services," Proc. IEEE Int'l Conf. Web Services (ICWS '08), pp. 465-472, 2008.
[30] A. Bertolino and A. Polini, "The Audition Framework for Testing Web Services Interoperability," Proc. 31st EUROMICRO Conf. Software Eng. and Advanced Applications, pp. 134-142, Aug. 2005.
[31] F. Belli and M. Linschulte, "Event-Driven Modeling and Testing of Web Services," Proc. IEEE 32nd Ann. Int'l Computer Software and Applications Conf. (COMPSAC '08), pp. 1168-1173, 2008.
[32] J. Magee, J. Kramer, S. Uchitel, and H. Foster, "LTSA-WS: A Tool for Model-Based Verification of Web Service Compositions and Choreography," Proc. 28th Int'l Conf. Software Eng. (ICSE '06), pp. 771-774, 2006.
[33] R. Heckel and M. Lohmann, "Towards Contract-Based Testing of Web Services," Electronic Notes in Theoretical Computer Science, vol. 82, no. 6, pp. 145-156, 2004.
[34] R. Heckel and L. Mariani, "Automatic Conformance Testing of Web Services," Proc. Int'l Conf. Fundamental Approaches to Software Eng. (FASE '05), pp. 34-48, 2005.
[35] W. Tsai, X. Wei, Y. Chen, R. Paul, and X. Bai, "Swiss Cheese Test Case Generation for Web Services Testing," IEICE - Trans. Information and Systems, vol. 88, no. 12, pp. 2691-2698, Dec. 2005.
[36] A. Bertolino, G. De Angelis, L. Frantzen, and A. Polini, "Model-Based Generation of Testbeds for Web Services," Proc. Int'l Conf. Testing of Software and Communicating Systems (TestCom/FATES '08), pp. 266-282, 2008.
[37] H. Huang, H. Liu, Z. Li, and J. Zhu, "Surrogate: A Simulation Apparatus for Continuous Integration Testing in Service Oriented Architecture," Proc. IEEE Int'l Conf. Services Computing (SCC '08), vol. 2, pp. 223-230, 2008.
[38] W.K. Chan, S.C. Cheung, and K.R.P.H. Leung, "A Metamorphic Testing Approach for Online Testing of Service-Oriented Software Applications," Int'l J. Web Services Research, vol. 4, no. 2, pp. 61-81, Apr. 2007.
[39] W. Tsai, X. Zhou, Y. Chen, and X. Bai, "On Testing and Evaluating Service-Oriented Software," Computer, vol. 41, no. 8, pp. 40-46, Aug. 2008.
[40] W. Tsai, Y. Chen, R. Paul, N. Liao, and H. Huang, "Cooperative and Group Testing in Verification of Dynamic Composite Web Services," Proc. 28th Ann. Int'l Computer Software and Applications Conf. (COMPSAC '04), vol. 2, pp. 170-173, Sept. 2004.
[41] A. Bertolino, J. Gao, E. Marchetti, and A. Polini, "TAXI-A Tool for XML-Based Testing," Proc. 29th Int'l Conf. Software Eng. (ICSE '07 Companion), pp. 53-54, May 2007.
[42] A. Bertolino, G. De Angelis, L. Frantzen, and A. Polini, "The PLASTIC Framework and Tools for Testing Service-Oriented Applications," Proc. Int'l Summer School on Software Eng. (ISSSE '08), pp. 106-139, 2008.
[43] G. Canfora and M. Penta, "Testing Services and Service-Centric Systems: Challenges and Opportunities," IT Professional, vol. 8, no. 2, pp. 10-17, 2006.
[44] H. Zhu, "A Framework for Service-Oriented Testing of Web Services," Proc. 30th Ann. Int'l Computer Software and Applications Conf. (COMPSAC '06), pp. 679-691, Sept. 2006.
[45] Insurance Industry Committee on Motor Vehicle Administration, Model User Guide for Implementing Online Insurance Verification, version 4, http://www.iicmva.comIICMVAPublications.html , Aug. 2010.
[46] K. Sycara, M. Paolucci, A. Ankolekar, and N. Srinivasan, "Automated Discovery, Interaction and Composition of Semantic Web Services," J. Web Semantics, vol. 1, no. 1, pp. 27-46, Dec. 2003.
[47] M. Uschold and M. Gruninger, "Ontologies: Principles, Methods, and Applications," Knowledge Eng. Rev., vol. 11, no. 2, pp. 93-155, 1996.
[48] T.R. Gruber, "A Translation Approach to Portable Ontology Specification," Knowledge Acquisition, vol. 5, pp. 199-220, 1993.
[49] OWL Web Ontology Language Reference, http://www.w3.org/TR/2004REC-owl-ref-20040210 , 2004.
[50] OWL-S: Semantic Markup for Web Services, http://www.w3.org/ SubmissionOWL-S, 2004.
[51] N. Srinivasan, M. Paolucci, and K. Sycara, "Adding OWL-S to UDDI, Implementation and Throughput," Proc. First Int'l Workshop Semantic Web Services and Web Process Composition, pp. 169-182, 2004.
[52] T. Kawamura, J-A. De Blasio, T. Hasegawa, M. Paolucci, and K. Sycara, "A Preliminary Report of a Public Experiment of a Semantic Service Matchmaker Combined with a UDDI Business Registry," Proc. First Int'l Conf. Service Oriented Computing (ICSOC '03), pp. 208-224, Dec. 2003.
[53] "Protégé-OWL API Programmer's Guide," http://protege. stanford.edu/plugins/owl/ apiguide.html, 2011.
[54] K. Sycara, M. Paolucci, J. Soudry, and N. Srinivasan, "Dynamic Discovery and Coordination of Agent-Based Semantic Web Services," IEEE Internet Computing, vol. 8, no. 3, pp. 66-73, May/June 2004.
[55] Mindswap OWL-S API, http://www.mindswap.org/2004/ owl-sapi, 2011.
[56] B. Yu, L. Kong, Y. Zhang, and H. Zhu, "Testing Java Components Based on Algebraic Specifications," Proc. First Int'l Conf. Software Testing Verification and Validation (ICST '08), pp. 190-199, 2008.
[57] C. Cadar, D. Dunbar, and D. Engler, "KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs," Proc. Eighth USENIX Conf. Operating Systems Design and Implementation (OSDI), 2008.
[58] S.C. Edmund, E. Clarke, A. Groce, S. Jha, and T. Vienna, "Modular Verification of Software Components in C," IEEE Trans. Software Eng., vol. 30, no. 6, pp. 388-402, June 2004.
[59] JavaNCSS, http:/javancss.codehaus.org, 2011.
[60] D. Hovemeyer and W. Pugh, "Finding More Null Pointer Bugs, but Not Too Many," Proc. Seventh ACM SIGPLAN-SIGSOFT Workshop Program Analysis for Software Tools and Eng. (PASTE '07), pp. 9-14, 2007.
[61] PMD, http:/pmd.sourceforge.net, 2011.
[62] L. Shan and H. Zhu, "Generating Structurally Complex Test Cases by Data Mutation," The Computer J., vol. 52, pp. 571-588, 2009.
[63] W. Tsai, R. Paul, L. Yu, A. Saimi, and Z. Cao, "Scenario-Based Web Services Testing with Distributed Agents," IEICE Trans. Information and Systems, vol. E86-D, no. 10, pp. 2130-2144, Oct. 2003.
[64] W. Tsai, R. Paul, Z. Cao, L. Yu, A. Saimi, and X. Bai, "Verification of Web Services Using an Enhanced UDDI Server," Proc. Eighth Int'l Workshop Object-Oriented Real-Time Dependable Systems (WORDS '03), pp. 131-138, 2003.
[65] X. Bai, Y. Wang, G. Dai, W.T. Tsai, and Y. Chen, "A Framework of Contract-Based Collaborative Verification and Validation of Web Services," Proc. 10th Int'l Conf. Component-Based Software Eng. (CBSE '07), pp. 256-271, July 2007.
[66] X. Bai, S. Lee, R. Liu, W.T. Tsai, and Y. Chen, "Collaborative Web Services Monitoring with Active Service Broker," Proc. IEEE 32nd Ann. Int'l Computer Software and Applications Conf. (COMPSAC '08), pp. 84-91, 2008.
[67] A. Bertolino and A. Polini, "SOA Test Governance: Enabling Service Integration Testing across Organization and Technology Borders," Proc. Int'l Conf. Software Testing, Verification, and Validation Workshops (ICSTW '09), Apr. 2009.
[68] W. Tsai, R. Paul, Y. Wang, C. Fan, and D. Wang, "Extending WSDL to Facilitate Web Services Testing," Proc. IEEE Seventh Int'l Symp. High Assurance Systems Eng. (HASE '02), pp. 171-172, 2002.
[69] H. Zhu and Q. Huo, "Developing A Software Testing Ontology in UML for a Software Growth Environment of Web-Based Applications," Software Evolution with UML and XML, H. Yang, ed., pp. 263-295, IDEA Group, Inc., 2005.
[70] H. Zhu, Q. Huo, and S. Greenwood, "A Multi-Agent Software Environment for Testing Web-Based Applications," Proc. 27th Ann. Int'l Conf. Computer Software and Applications Conf. (COMPSAC '03), pp. 210-215, Nov. 2003.
[71] Y. Zhang and H. Zhu, "Ontology for Service Oriented Testing of Web Services," Proc. IEEE Int'l Symp. Service Oriented System Eng. (SOSE '08), Dec. 2008.
[72] T. Berners-Lee, J. Hendler, and O. Lassila, "The Semantic Web," Scientific Am., vol. 284, no. 5, pp. 34-43, 2001.
6 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool