This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Assessing Software Service Quality and Trustworthiness at Selection Time
July/August 2010 (vol. 36 no. 4)
pp. 559-574
Noura Limam, POSTECH-Pohang University of Science and Technology, Pohang
Raouf Boutaba, University of Waterloo, Waterloo
Online Only: Erratum(PDF)
The integration of external software in project development is challenging and risky, notably because the execution quality of the software and the trustworthiness of the software provider may be unknown at integration time. This is a timely problem and of increasing importance with the advent of the SaaS model of service delivery. Therefore, in choosing the SaaS service to utilize, project managers must identify and evaluate the level of risk associated with each candidate. Trust is commonly assessed through reputation systems; however, existing systems rely on ratings provided by consumers. This raises numerous issues involving the subjectivity and unfairness of the service ratings. This paper describes a framework for reputation-aware software service selection and rating. A selection algorithm is devised for service recommendation, providing SaaS consumers with the best possible choices based on quality, cost, and trust. An automated rating model, based on the expectancy-disconfirmation theory from market science, is also defined to overcome feedback subjectivity issues. The proposed rating and selection models are validated through simulations, demonstrating that the system can effectively capture service behavior and recommend the best possible choices.

[1] J. Li, R. Conradi, O.P. Slyngstad, M. Torchiano, M. Morisio, and C. Bunse, "A State-of-the-Practice Survey of Risk Management in Development with Off-the-Shelf Software Components," IEEE Trans. Software Eng., vol. 34, no. 2, pp. 271-286, Mar./Apr. 2008.
[2] J. Anselmi, D. Ardagna, and P. Cremonesi, "A QoS-Based Selection Approach of Autonomic Grid Services," Proc. Workshop Service-Oriented Computing Performance: Aspects, Issues, and Approaches, 2007.
[3] L. Zeng, B. Benatallah, A.H. Ngu, M. Dumas, J. Kalagnanam, and H. Chang, "QoS-Aware Middleware for Web Services Composition," IEEE Trans. Software Eng., vol. 30, no. 5, pp. 311-327, May 2004.
[4] L.-H. Vu, M. Hauswirth, and K. Aberer, "QoS-Based Service Selection and Ranking with Trust and Reputation Management," Proc. 13th Conf. Cooperative Information Systems, 2005.
[5] J. Skene, F. Raimondi, and W. Emmerich, "Service-Level Agreements for Electronic Services," IEEE Trans. Software Eng., vol. 36, no. 2, pp. 288-304, Mar./Apr. 2010, http://doi.ieeecomputer society.org/10.1109 TSE.2009.55.
[6] J.R. Douceur, "The Sybil Attack," Proc. First Int'l Workshop Peer-to-Peer Systems, 2002.
[7] J. Skene, A. Skene, J. Crampton, and W. Emmerich, "The Monitorability of Service-Level Agreements for Application-Service Provision," Proc. Sixth Int'l Workshop Software and Performanc, pp. 3-14, 2007.
[8] BelGOnet, "Service Level Agreement," http://www.belgonet. com/website/ UKService_Level_Agreement_UK.pdf, 2008.
[9] EZSM, "EZSM Service Level Agreement," http://www.easy servermanagement.comsla.php , 2008.
[10] Amazon, "Amazon s3 Service Level Agreement," http://aws. amazon.coms3-sla/, 2007.
[11] Intacct, "Intacct Buy with Confidence," http://us.intacct.com/downloads/08datasheets DS_Buy_with_Confidence.pdf, 2008.
[12] H. Ludwig, R.P. King, and A. Keller, "Web Service Level Agreements," http://www.research.ibm.com/wslasample- outsourced.wsla , 2002.
[13] Y. Wang and J. Vassileva, "A Review on Trust and Reputation for Web Service Selection," Proc. 27th Int'l Conf. Distributed Computing Systems Workshops, 2007.
[14] T. SaaS, "Trust Saas: Putting the Trust in Software as a Service (SaaS)," http:/trustsaas.com/, 2008.
[15] R.L. Oliver, "A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions," J. Marketing Research, vol. 17, pp. 460-469, Nov. 1980.
[16] J. Xiao and R. Boutaba, "Assessing Network Service Profitability: Modeling from Market Science Perspective," IEEE/ACM Trans. Networking, vol. 15, no. 6, pp. 1307-1320, Dec. 2007.
[17] V. Poladian, J.P. Sousa, D. Garlan, and M. Shaw, "Dynamic Configuration of Resource-Aware Services," Proc. 26th Int'l Conf. Software Eng., 2004.
[18] P. Brockwell and R. Davis, Introduction to Time Series and Forecasting. Springer-Verlang, 2002.
[19] V. Tosic, K. Patel, and B. Pagurek, "Wsol—Web Service Offerings Language," Proc. Workshop Web Services, e-Business, and the Semantic Web, 2002.
[20] A. Keller and H. Ludwig, "The WSLA Framework: Specifying and Monitoring Service Level Agreements for Web Services," J. Network and System Management, vol. 11, no. 1, pp. 57-81, 2003.
[21] C. Zhou, L.-T. Chia, and B.-S. Lee, "DAML-QoS Ontology for Web Services," Proc. IEEE Int'l Conf. Web Services, 2004.
[22] M. Burstein, J. Hobbs, O. Lassila, D. Mcdermott, S. Mcilraith, S. Narayanan, M. Paolucci, B. Parsia, T. Payne, E. Sirin, N. Srinivasan, and K. Sycara, "OWL-S: Semantic Markup for Web Services," http://www.w3.org/Submission/2004SUBM-OWL-S-20041122 /, 2004.
[23] K. Kyriakos and P. Dimitris, "A Semantic QoS-Based Web Service Discovery Algorithm for Over-Constrained Demands," Proc. Third Int'l Conf. Next Generation Web Services Practices, 2007.
[24] N. Limam and R. Boutaba, "Assessing Service Quality and Trustworthiness at Selection Time," Proc. 11th IEEE/IFIP Network Operations and Management Symp., 2008.
[25] T. Yu, Y. Zhang, and K.-J. Lin, "Efficient Algorithms for Web Services Selection with End-to-End QoS Constraints," ACM Trans. Web, vol. 1, no. 1, 2007.
[26] G. Wu, J. Wei, X. Qiao, and L. Li, "A Bayesian Network Based QoS Assessment Model for Web Services," Proc. IEEE Int'l Conf. Services Computin, 2007.
[27] S.J.H. Yang, J.S.F. Hsieh, B.C.W. Lan, and J.-Y. Chung, "Composition and Evaluation of Trustworthy Web Services," Proc. IEEE Int'l Workshop Business Services Networks, 2005.
[28] A.S. Ali, S. Majithia, O.F. Rana, and D.W. Walker, "Reputation-Based Semantic Service Discovery," Proc. 13th IEEE Int'l Workshops Enabling Technologies: Infrastructure for Collaborative Enterprises, 2004.
[29] R. Wishart, R. Robinson, J. Indulska, and A. Josang, "Superstringrep: Reputation-Enhanced Service Discovery," Proc. 28th Australasian Conf. Computer Science, 2005.
[30] F. Raimondi, J. Skene, and W. Emmerich, "Efficient Online Monitoring of Web-Service SLAs," Proc. 16th ACM SIGSOFT Int'l Symp. Foundations of Software Eng., pp. 170-180, 2008.

Index Terms:
Software as a service (SaaS), software selection, service utility, review and rating, trust and reputation, risk management, SLA monitoring.
Citation:
Noura Limam, Raouf Boutaba, "Assessing Software Service Quality and Trustworthiness at Selection Time," IEEE Transactions on Software Engineering, vol. 36, no. 4, pp. 559-574, July-Aug. 2010, doi:10.1109/TSE.2010.2
Usage of this product signifies your acceptance of the Terms of Use.