The Community for Technology Leaders
RSS Icon
Issue No.10 - Oct. (2013 vol.39)
pp: 1444-1465
Chunyang Ye , University of Toronto, Toronto
Hans-Arno Jacobsen , University of Toronto, Toronto
Whitening the testing of service-oriented applications can provide service consumers confidence on how well an application has been tested. However, to protect business interests of service providers and to prevent information leakage, the implementation details of services are usually invisible to service consumers. This makes it challenging to determine the test coverage of a service composition as a whole and design test cases effectively. To address this problem, we propose an approach to whiten the testing of service compositions based on events exposed by services. By deriving event interfaces to explore only necessary test coverage information from service implementations, our approach allows service consumers to determine test coverage based on selected events exposed by services at runtime without releasing the service implementation details. We also develop an approach to design test cases effectively based on event interfaces concerning both effectiveness and information leakage. The experimental results show that our approach outperforms existing testing approaches for service compositions with up to 49 percent more test coverage and an up to 24 percent higher fault-detection rate. Moreover, our solution can trade off effectiveness, efficiency, and information leakage for test case generation.
events, Web service composition, white-box testing, event interface,
Chunyang Ye, Hans-Arno Jacobsen, "Whitening SOA Testing via Event Exposure", IEEE Transactions on Software Engineering, vol.39, no. 10, pp. 1444-1465, Oct. 2013, doi:10.1109/TSE.2013.20
[1] "ODE BPEL Engine," Apache, http:/, 2013.
[2] L. Ardissono, L. Console, A. Goy, G. Petrone, C. Picardi, M. Segnan, and D.T. Dupre, "Enhancing Web Services with Diagnostic Capabilities," Proc. Third European Conf. Web Services, p. 182, 2005.
[3] X. Bai, W. Dong, W. Tsai, and Y. Chen, "WSDL-Based Automatic Test Case Generation for Web Services Testing," Proc. IEEE Int'l Workshop Service-Oriented System Eng., pp. 207-212, 2005.
[4] C. Bartolini, A. Bertolino, S. Elbaum, and E. Marchetti, "Whitening SOA Testing," Proc. Seventh Joint Meeting of the European Software Eng. Conf. and the ACM SIGSOFT Symp. The Foundations of Software Eng., pp. 161-170, 2009.
[5] C. Bartolini, A. Bertolino, S. Elbaum, and E. Marchetti, "Bringing White-Box Testing to Service Oriented Architectures through a Service Oriented Approach," J. Systems and Software, vol. 84, pp. 655-668, Apr. 2011.
[6] C. Bartolini, A. Bertolino, F. Lonetti, and E. Marchetti, "Approaches to Functional, Structural and Security SOA Testing," Performance and Dependability in Service Computing: Concepts, Techniques and Research Directions, to be published, 2011.
[7] C. Bartolini, A. Bertolino, E. Marchetti, and A. Polini, "Towards Automated WSDL-Based Testing of Web Services," Proc. Sixth Int'l Conf. Service-Oriented Computing, pp. 524-529, 2008.
[8] L. Bentakouk, P. Poizat, and F. Zaïdi, "A Formal Framework for Service Orchestration Testing Based on Symbolic Transition Systems," Proc. 21st IFIP WG 6.1 Int'l Conf. Testing of Software and Comm. Systems and Ninth Int'l FATES Workshop, pp. 16-32, 2009.
[9] A. Bertolino, G. Angelis, L. Frantzen, and A. Polini, "Model-Based Generation of Testbeds for Web Services," Proc. 20th IFIP TC 6/WG 6.1 Int'l Conf. Testing of Software and Communicating Systems: Eighth Int'l Workshop, pp. 266-282, 2008.
[10] A. Bertolino and A. Polini, "SOA Test Governance: Enabling Service Integration Testing Across Organization and Technology Borders," Proc. IEEE Int'l Conf. Software Testing, Verification, and Validation Workshops, pp. 277-286, 2009.
[11] S. Beydeda and V. Gruhn, "An Integrated Testing Technique for Component-Based Software," Proc. ACS/IEEE Int'l Conf. Computer Systems and Applications, p. 328, 2001.
[12] D. Beyer, A. Chakrabarti, and T.A. Henzinger, "Web Service Interfaces," Proc. Int'l Conf. World Wide Web, pp. 148-159, 2005.
[13] R.V. Binder, Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Longman Publishing Co., Inc., 1999.
[14] L.C. Briand, Y. Labiche, and M.M. Sówka, "Automated, Contract-Based User Testing of Commercial-Off-the-Shelf Components," Proc. 28th Int'l Conf. Software Eng., pp. 92-101, 2006.
[15] L.C. Briand, Y. Labiche, and Y. Wang, "A Comprehensive and Systematic Methodology for Client-Server Class Integration Testing," Proc. 14th Int'l Symp. Software Reliability Eng., p. 14, 2003.
[16] A. Bucchiarone, H. Melgratti, and F. Severoni, "Testing Service Composition," Proc. Eighth Argentine Symp. Software Eng., 2007.
[17] G. Canfora and M. Di Penta, "Testing Services and Service-Centric Systems: Challenges and Opportunities," IT Professional, vol. 8, no. 2, pp. 10-17, Mar./Apr. 2006.
[18] G. Canfora and M. Di Penta, "Service-Oriented Architectures Testing: A Survey," Software Eng., pp. 78-105, 2009.
[19] T.-D. Cao, T.-T. Phan-Quang, P. Felix, and R. Castanet, "Automated Runtime Verification for Web Services," Proc. IEEE Int'l Conf. Web Services, pp. 76-82, 2010.
[20] W. Chan, S. Cheung, and K. Leung, "A Metamorphic Testing Approach for Online Testing of Service-Oriented Software Applications," Int'l J. Web Services Research, vol. 4, no. 2, pp. 61-81, 2007.
[21] K. Chandy, "Event-Driven Applications: Costs, Benefits and Design Approaches," Proc. Gartner Application Integration and Web Services Summit Conf., 2006.
[22] K. Chatzikokolakis, T. Chothia, and A. Guha, "Statistical Measurement of Information Leakage," Proc. Tools and Algorithms for the Construction and Analysis of Systems Conf., pp. 390-404, 2010.
[23] L.A. Clarke, "A System to Generate Test Data and Symbolically Execute Programs," IEEE Trans. Software Eng., vol. 2, no. 3, pp. 215-222, Sept. 1976.
[24] J.C. Corbett, M.B. Dwyer, J. Hatcliff, S. Laubach, C.S. Păsăreanu, and H. Zheng, "Bandera: Extracting Finite-State Models from Java Source Code," Proc. Int'l Conf. Software Eng., pp. 439-448, 2000.
[25] G. Dai, X. Bai, Y. Wang, and F. Dai, "Contract-Based Testing for Web Services," Proc. 31st Ann. Int'l Computer Software and Applications Conf., vol. 1, pp. 517-526, 2007.
[26] L. de Alfaro and T.A. Henzinger, "Automata," Proc. Eighth European Software Eng. Conf., pp. 109-120, 2001.
[27] R. De Nicola and M. Hennessy, "Testing Equivalence for Processes," Proc. Int'l Colloquium Automata Languages and Programming, pp. 548-560, 1983.
[28] R.A. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," Computer, vol. 11, pp. 34-41, Apr. 1978.
[29] D. Dranidis, E. Ramollari, and D. Kourtesis, "Run-Time Verification of Behavioural Conformance for Conversational Web Services," Proc. IEEE Seventh European Conf. Web Services, pp. 139-147, 2009.
[30] M. Emmi, D. Giannakopoulou, and C.S. Păsăreanu, "Assume-Guarantee Verification for Interface Automata," Proc. 15th Int'l Symp. Formal Methods, pp. 116-131, 2008.
[31] I. Epifani, C. Ghezzi, and G. Tamburrelli, "Change-Point Detection for Black-Box Services," Proc. 18th ACM SIGSOFT Int'l Symp. Foundations of Software Eng. pp. 227-236, 2010.
[32] H. Foster, S. Uchitel, J. Magee, and J. Kramer, "Model-Based Verification of Web Service Compositions," Proc. 18th IEEE Int'l Conf. Automated Software Eng., pp. 152-161, 2003.
[33] X. Fu, T. Bultan, and J. Su, "Analysis of Interacting Bpel Web Services," Proc. 13th Int'l Conf. World Wide Web, pp. 621-630, 2004.
[34] W. Gaaloul, S. Bhiri, and M. Rouached, "Event-Based Design and Runtime Verification of Composite Service Transactional Behavior," IEEE Trans. Services Computing, vol. 3, no. 1, pp. 32-45, Jan.-Mar. 2010.
[35] V. Ganesh and D.L. Dill, "A Decision Procedure for Bit-Vectors and Arrays," Proc. 19th Int'l Conf. Computer Aided Verification, pp. 519-531, 2007.
[36] S. Hallé, T. Bultan, G. Hughes, M. Alkhalaf, and R. Villemaire, "Runtime Verification of Web Service Interface Contracts," Computer, vol. 43, no. 3, pp. 59-66, Mar. 2010.
[37] J.E. Hanson, P. Nandi, and S. Kumaran, "Conversation Support for Business Process Integration," Proc. Sixth Int'l Enterprise Distributed Object Computing Conf., pp. 65-74, 2002.
[38] J. Hodges Jr and E. Lehmann, "Estimates of Location Based on Rank Tests," The Annals of Math. Statistics, vol. 34, pp. 598-611, 1963.
[39] M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, "Experiments of the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria," Proc. 16th Int'l Conf. Software Eng., pp. 191-200, 1994.
[40] "Loan Approval," IBM, libraryws-bpelcol5/, 2013.
[41] Z. Jin and A.J. Offutt, "Coupling-Based Criteria for Integration Testing," The J. Software Testing, Verification, and Reliability, vol. 8, pp. 133-154, 1998.
[42] K. Kaschner and N. Lohmann, "Automatic Test Case Generation for Interacting Services," Proc. Int'l Conf. Service-Oriented Computing Workshops, pp. 66-78, 2008.
[43] G. Li, V. Muthusamy, and H.-A. Jacobsen, "A Distributed Service-Oriented Architecture for Business Process Execution," ACM Trans. Web, vol. 4, no. 1, pp. 1-33, 2010.
[44] L. Li, W. Chou, and W. Guo, "Control Flow Analysis and Coverage Driven Testing for Web Services," Proc. IEEE Int'l Conf. Web Services, pp. 473-480, 2008.
[45] Z. Li, W. Sun, Z.B. Jiang, and X. Zhang, "BPEL4WS Unit Testing: Framework and Implementation," Proc. IEEE Int'l Conf. Web Services, pp. 103-110, 2005.
[46] Z.J. Li, H.F. Tan, H.H. Liu, J. Zhu, and N.M. Mitsumori, "Business-Process-Driven Gray-Box SOA Testing," IBM Systems J., vol. 47, no. 3, pp. 457-472, 2008.
[47] D.C. Luckham, The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison-Wesley, 2001.
[48] C. Ma, C. Du, T. Zhang, F. Hu, and X. Cai, "WSDL-Based Automated Test Data Generation for Web Service," Proc. Int'l Conf. Computer Science and Software Eng., vol. 2, pp. 731-737, 2008.
[49] L. Mariani, "Behavior Capture and Test for Verifying Evolving Component-Based Systems," Proc. Int'l Conf. Software Eng., pp. 78-80, 2004.
[50] L. Mariani, S. Papagiannakis, and M. Pezze, "Compatibility and Regression Testing of Cots-Component-Based Software," Proc. 29th Int'l Conf. Software Eng., pp. 85-95, 2007.
[51] L. Mather and E. Oswald, "Pinpointing Side-Channel Information Leaks in Web Applications," J. Cryptographic Eng., vol. 2, pp. 131-177, 2012.
[52] J. Mayer and C. Schneckenburger, "An Empirical Analysis and Comparison of Random Testing Techniques," Proc. ACM/IEEE Int'l Symp. Empirical Software Eng., pp. 105-114, 2006.
[53] T. McCabe, "A Complexity Measure," IEEE Trans. Software Eng., vol. 2, no. 4, pp. 308-320, Dec. 1976.
[54] P. McMinn, "Search-Based Software Test Data Generation: A Survey: Research Articles," Software Testing, Verification, and Reliability, vol. 14, no. 2, pp. 105-156, June 2004.
[55] L. Mei, W. Chan, and T. Tse, "Data Flow Testing of Service-Oriented Workflow Applications," Proc. 30th Int'l Conf. Software Eng., pp. 371-380, 2008.
[56] L. Mei, W.K. Chan, and T.H. Tse, "Data Flow Testing of Service Choreography," Proc. Seventh Joint Meeting of the European Software Eng. Conf. and the ACM SIGSOFT Symp. The Foundations of Software Eng., pp. 151-160, 2009.
[57] L. Mei, W.K. Chan, T.H. Tse, and R.G. Merkel, "Tag-Based Techniques for Black-Box Test Case Prioritization for Service Testing," Proc. Ninth Int'l Conf. Quality Software, pp. 21-30, 2009.
[58] L. Mei, Z. Zhang, W.K. Chan, and T.H. Tse, "Test Case Prioritization for Regression Testing of Service-Oriented Business Applications," Proc. 18th Int'l Conf. World Wide Web, pp. 901-910, 2009.
[59] H. Muccini, A. Bertolino, and P. Inverardi, "Using Software Architecture for Code Testing," IEEE Trans. Software Eng., vol. 30, no. 3, pp. 160-171, Mar. 2004.
[60] S. Noikajana and T. Suwannasart, "Web Service Test Case Generation Based on Decision Table (Short Paper)," Proc. Eighth Int'l Conf. Quality Software, pp. 321-326, 2008.
[61] "Book Ordering," Oracle, products/bpelindex.html, 2013.
[62] A. Orso, M.J. Harrold, and D.S. Rosenblum, "Component Metadata for Software Engineering Tasks," Proc. Revised Papers from the Second Int'l Workshop Eng. Distributed Objects, pp. 129-144, 2001.
[63] "SCA Event Processing," OSOA, http:/, 2013.
[64] C. Peltz, "Web Services Orchestration and Choreography," Computer, vol. 36, no. 10, pp. 46-52, Oct. 2003.
[65] M.J. Rutherford, A. Carzaniga, and A.L. Wolf, "Evaluating Test Suites and Adequacy Criteria Using Simulation-Based Models of Distributed Systems," IEEE Trans. Software Eng., vol. 34, no. 4, pp. 452-470, July 2008.
[66] C.E. Shannon, "A Mathematical Theory of Communication," ACM SIGMOBILE Mobile Computing and Comm. Rev., vol. 5, no. 1, pp. 3-55, Jan. 2001.
[67] J. Tretmans, "Test Generation with Inputs, Outputs, and Quiescence," Proc. Second Int'l Workshop Tools and Algorithms for Construction and Analysis of Systems, pp. 127-146, 1996.
[68] "Web Services Description Language," W3C, http://www.w3. org/TRwsdl, 2013.
[69] "Ws-Eventing, Owl-s," W3C, http:/, 2013.
[70] L. Wang and S.D. Stoller, "Runtime Analysis of Atomicity for Multi-Threaded Programs," IEEE Trans. Software Eng., vol. 32, no. 2, pp. 93-110, Feb. 2006.
[71] Y. Wang, X. Bai, J. Li, and R. Huang, "Ontology-Based Test Case Generation for Testing Web Services," Proc. Eighth Int'l Symp. Autonomous Decentralized Systems, pp. 43-50, 2007.
[72] E.J. Weyuker, "Axiomatizing Software Test Data Adequacy," IEEE Trans. Software Eng., vol. 12, no. 12, pp. 1128-1138, Dec. 1986.
[73] F. Wilcoxon, "Individual Comparisons by Ranking Methods," Biometrics Bull., vol. 1, no. 6, pp. 80-83, 1945.
[74] "Supply Chain Management," WS-I., deliverablesworkinggroup.aspx?wg=sampleapps , 2013.
[75] G. Wu, J. Wei, C. Ye, X. Shao, H. Zhong, and T. Huang, "Runtime Monitoring of Data-Centric Temporal Properties for Web Services," Proc. IEEE Int'l Conf. Web Services, pp. 161-170, 2011.
[76] G. Wu, J. Wei, C. Ye, H. Zhong, and T. Huang, "Detecting Data Inconsistency Failure of Composite Web Services through Parametric Stateful Aspect," Proc. IEEE Int'l Conf. Web Services, pp. 68-75, 2010.
[77] W. Xu, J. Offutt, and J. Luo, "Testing Web Services by XML Perturbation," Proc. IEEE 16th Int'l Symp. Software Reliability Eng., pp. 257-266, 2005.
[78] C. Ye, S.C. Cheung, and W.K. Chan, "Publishing and Composition of Atomicity-Equivalent Services for B2B Collaboration," Proc. 28th Int'l Conf. Software Eng., pp. 351-360, 2006.
[79] C. Ye, S.C. Cheung, W.K. Chan, and C. Xu, "Atomicity Analysis of Service Composition across Organizations," IEEE Trans. Software Eng., vol. 35, no. 1, pp. 2-28, Jan./Feb. 2009.
[80] C. Ye and H.-A. Jacobsen, "White-Box Testing of Service Compositions via Event Interfaces," technical report, Univ. of Toronto, 2010.
[81] Y. Yuan, Z. Li, and W. Sun, "A Graph-Search Based Approach to BPEL4Ws Test Generation," Proc. Int'l Conf. Software Eng. Advances, p. 14, 2006.
[82] Y. Zheng, J. Zhou, and P. Krause, "An Automatic Test Case Generation Framework for Web Services," J. Software, vol. 2, no. 3, pp. 64-77, 2007.
[83] H. Zhu, P.A.V. Hall, and J.H.R. May, "Software Unit Test Coverage and Adequacy," ACM Computing Surveys, vol. 29, no. 4, pp. 366-427, 1997.
[84] H. Zhu and Y. Zhang, "Collaborative Testing of Web Services," IEEE Trans. Services Computing, vol. 5, no. 1, pp. 116-130, Jan.-Mar. 2012.
8 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool