The Community for Technology Leaders
RSS Icon
Issue No.06 - November/December (2008 vol.34)
pp: 813-825
Bo Yang , University of Electronic Science and Technology of China, Chengdu
Huajun Hu , University of Electronic Science and Technology of China, Chengdu
Lixin Jia , Xi'an Jiaotong University, Xi'an
For a development software project, management often faces the dilemma of when to stop testing the software and release it for operation, which requires careful decision-making as it has great impact on both software reliability and project cost. In most existing research on optimal software release problem, the cost considered was the expected cost (EC) of the project. However, what management concerns is the actual cost (AC) of the project rather than the EC. Treatment (such as minimization) of the EC may not ensure a desired low level of the AC, due to the uncertainty (variability) involved in the AC. In this paper, we study the uncertainty in software cost and its impact on optimal software release time in detail. The uncertainty is quantified by the variance of the AC and several risk functions. A risk-control approach to optimal software release problem is proposed. New formulations of the problem which are extensions of current formulations are developed, and solution procedures are established. Several examples are presented. Results reveal that it seems crucial to take account of the uncertainty in software cost in optimal software release problem, otherwise unsafe decision may be reached which could be a false dawn to management.
Cost estimation, Time estimation, Project control and modeling, Reliability
Bo Yang, Huajun Hu, Lixin Jia, "A Study of Uncertainty in Software Cost and Its Impact on Optimal Software Release Time", IEEE Transactions on Software Engineering, vol.34, no. 6, pp. 813-825, November/December 2008, doi:10.1109/TSE.2008.47
[1] S. Yamada, T. Ichimori, and M. Nishiwaki, “Optimal Allocation Policies for Testing-Resource Based on a Software Reliability Growth Model,” Math. and Computer Modelling, vol. 22, nos. 10-12, pp. 295-301, 1995.
[2] M. Xie, Software Reliability Modelling. World Scientific, 1991.
[3] K. Okumoto and A.L. Goel, “Optimum Release Time for Software Systems, Based on Reliability and Cost Criteria,” J. Systems and Software, vol. 1, no. 4, pp. 315-318, 1980.
[4] H.S. Koch and P. Kubat, “Optimal Release Time for Computer Software,” IEEE Trans. Software Eng., vol. 9, no. 3, pp. 323-327, May 1983.
[5] S.M. Ross, “Software Reliability: The Stopping Rule Problem,” IEEE Trans. Software Eng., vol. 11, no. 12, pp. 1472-1476, Dec. 1985.
[6] S. Yamada and S. Osaki, “Cost-Reliability Optimal Release Policies for Software Systems,” IEEE Trans. Reliability, vol. 34, no. 5, pp.422-424, Dec. 1985.
[7] N.D. Singpurwalla, “Determining an Optimal Time Interval for Testing and Debugging Software,” IEEE Trans. Software Eng., vol. 17, no. 4, pp. 313-319, Apr. 1991.
[8] Y.W. Leung, “Optimum Software Release Time with a Given Cost Budget,” J. Systems and Software, vol. 17, no. 3, pp. 233-242, 1992.
[9] H. Pham, “A Software Cost Model with Imperfect Debugging, Random Life Cycle and Penalty Cost,” Int'l J. Systems Science, vol. 27, no. 5, pp. 455-463, 1996.
[10] R.H. Hou, S.Y. Kuo, and Y.P. Chang, “Optimal Release Times for Software Systems with Scheduled Delivery Time Based on the HGDM,” IEEE Trans. Computers, vol. 46, no. 2, pp. 216-221, Feb. 1997, doi:10.1109/12.565602.
[11] H. Pham and X. Zhang, “A Software Cost Model with Warranty and Risk Costs,” IEEE Trans. Computers, vol. 48, no. 1, pp. 71-75, Jan. 1999, doi:10.1109/12.743412.
[12] M. Kimura, T. Toyota, and S. Yamada, “Economic Analysis of Software Release Problems with Warranty Cost And Reliability Requirement,” Reliability Eng. and System Safety, vol. 66, no. 1, pp.49-55, 1999.
[13] M. Xie and B. Yang, “A Study of the Effect of Imperfect Debugging on Software Development Cost,” IEEE Trans. Software Eng., vol. 29, no. 5, pp. 471-473, May 2003, doi:10.1109/TSE.2003. 1199075.
[14] N. Morali and R. Soyer, “Optimal Stopping in Software Testing,” Naval Research Logistics, vol. 50, no. 1, pp. 88-104, 2003.
[15] H. Pham, “Software Reliability and Cost Models: Perspectives, Comparison, and Practice,” European J. Operational Research, vol. 149, no. 3, pp. 475-489, 2003.
[16] P.J. Boland and H. Singh, “A Birth-Process Approach to Moranda's Geometric Software-Reliability Model,” IEEE Trans. Reliability, vol. 52, no. 2, pp. 168-174, June 2003, doi:10.1109/TR.2003.813166.
[17] Y.C. Chang and W.L. Hung, “Software Release Policies on a Shot-Noise Process Model,” Applied Math. and Computation, vol. 171, no. 2, pp. 746-759, 2005.
[18] C.Y. Huang, “Cost-Reliability-Optimal Release Policy for Software Reliability Models Incorporating Improvements in Testing Efficiency,” J. Systems and Software, vol. 77, no. 2, pp. 139-155, 2005.
[19] C.Y. Huang and M.R. Lyu, “Optimal Release Time for Software Systems Considering Cost, Testing-Effort, and Test Efficiency,” IEEE Trans. Reliability, vol. 54, no. 4, pp. 583-591, Dec. 2005, doi:10.1109/TR.2005.859230.
[20] C.T. Liu and Y.C. Chang, “A Reliability-Constrained Software Release Policy Using a Non-Gaussian Kalman Filter Model,” Probability in the Eng. and Informational Sciences, vol. 21, no. 2, pp.301-314, 2007.
[21] P.J. Bolanda and N.N. Chuiv, “Optimal Times for Software Release When Repair Is Imperfect,” Statistics and Probability Letters, vol. 77, no. 12, pp. 1176-1184, 2007.
[22] B. Yang, H. Hu, and J. Zhou, “Optimal Software Release Time Determination with Risk Constraint,” Proc. 54th Ann. Reliability and Maintainability Symp., pp. 393-398, Jan. 2008.
[23] M. Jørgensen, “Realism in Assessment of Effort Estimation Uncertainty: It Matters How You Ask,” IEEE Trans. Software Eng., vol. 30, no. 4, pp. 209-217, Apr. 2004, doi:10.1109/TSE.2004. 1274041.
[24] M. Jørgensen, “Evidence-Based Guidelines for Assessment of Software Development Cost Uncertainty,” IEEE Trans Software Eng., vol. 31, no. 11, pp. 942-954, Nov. 2005, doi:10.1109/TSE. 2005.128.
[25] W. Farr, “Software Reliability Software Survey,” Handbook of Software Reliability Engineering, M.R. Lyu, ed., chapter 3, McGraw-Hill, 1996.
[26] J.D. Musa, A. Iannino, and K. Okumoto, Software Reliability: Measurement, Prediction, Application. McGraw-Hill, 1987.
[27] J. Juran and F. Gryna, Quality Control Handbook, fourth ed. McGraw-Hill, 1988.
[28] S.A. Slaughter, E.D. Harter, and M.S. Krishnan, “Evaluating the Cost of Software Quality,” Comm. ACM, vol. 41, no. 8, pp. 67-73, 1998.
[29] A.L. Goel and K. Okumoto, “Time Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures,” IEEE Trans. Reliability, vol. 28, no. 3, pp. 206-211, 1979.
[30] S. Yamada, M. Ohba, and S. Osaki, “S-Shaped Software Reliability Growth Models and Their Applications,” IEEE Trans. Reliability, vol. 33, no. 4, pp. 289-292, Oct. 1984.
[31] M. Ohba, “Software Reliability Analysis Models,” IBM J. Research and Development, vol. 28, no. 4, pp. 428-443, 1984.
[32] B. Yang and M. Xie, “A Study of Operational and Testing Reliability in Software Reliability Analysis,” Reliability Eng. and System Safety, vol. 70, no. 3, pp. 323-329, 2000.
[33] M. Xie and G.Y. Hong, “A Study of the Sensitivity of Software Release Time,” J. Systems and Software, vol. 44, no. 2, pp. 163-168, 1998.
[34] M. Xie and B. Yang, “Optimal Testing-Time Allocation for Modular Systems,” Int'l J. Quality and Reliability Management, vol. 18, no. 8, pp. 854-863, 2001.
[35] C.Y. Huang and M.R. Lyu, “Optimal Testing Resource Allocation, and Sensitivity Analysis in Software Development,” IEEE Trans. Reliability, vol. 54, no. 4, pp. 592-603, Dec. 2005, doi:10.1109/TR.2005.858099.
[36] M. Xie, B. Yang, and O. Gaudoin, “Sensitivity Analysis in Optimal Software Release Time Problems,” OPSEARCH, vol. 41, no. 4, pp.250-263, 2004.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool