This Article 
 Bibliographic References 
 Add to: 
Orthogonal Defect Classification-A Concept for In-Process Measurements
November 1992 (vol. 18 no. 11)
pp. 943-956

Orthogonal defect classification (ODC), a concept that enables in-process feedback to software developers by extracting signatures on the development process from defects, is described. The ideas are evolved from an earlier finding that demonstrates the use of semantic information from defects to extract cause-effect relationships in the development process. This finding is leveraged to develop a systematic framework for building measurement and analysis methods. The authors define ODC and discuss the necessary and sufficient conditions required to provide feedback to a developer; illustrate the use of the defect type distribution to measure the progress of a product through a process; illustrate the use of the defect trigger distribution to evaluate the effectiveness and eventually the completeness of verification processes such as inspection or testing; provides sample results from pilot projects using ODC; and open the doors to a wide variety of analysis techniques for providing effective and fast feedback based on the concepts of ODC.

[1] J. Gray, "A Census of Tandem System Availability, 1985-1990,"IEEE Trans. Reliability, Vol. 39, No. 4, Oct. 1990, pp. 409-418.
[2] IEEE Software, "Theme articles: Reliability measurement,"IEEE Soft ware, vol. 9, July 1992.
[3] J. D. Musaet al., Software Reliability Measurement, Prediction, Application. New York: McGraw-Hill International, 1987.
[4] C. V. Ramamoorthy and F. B. Bastani, "Software reliability--Status and perspectives,"IEEE Trans. Software Eng., vol. 8, pp. 354-371, 1982.
[5] A.L. Goel, "Software reliability models: Assumptions, limitations, and applicability,"IEEE Trans. Software Eng., vol. 11, pp. 1411-1423, 1985.
[6] R. Mays, C. Jones. G. Holloway, and D. Studinski, "Experiences with defect prevention,"IBM Syst. J., vol. 29, 1990.
[7] W. S. Humphrey,Managing the Software Process. Reading, MA: Addison-Wesley, 1989.
[8] B. Littlewood and I. L. Verrall, "A Bayesian reliability growth model for computer software,"J. Royal Statistical Society, vol. 22, pp. 332-346, 1973.
[9] M. Ohba, "Software reliability analysis models,"IBM J. Res. Develop., vol. 21, no. 4, pp. 428-443, July 1984.
[10] IEEE Standards, "Standard for software quality metrics methodology,"Software Engineering Standards Subcommittee, vol. Standard P-1061/D21-Unapproved Draft, Apr. 1990.
[11] IEEE Software, "Theme articles: Metrics,"IEEE Software, vol. 7, Mar. 1990.
[12] V. R. Basili and B. T. Perricone, "Software errors and complexity: An empirical investigation,"Commun. ACM, vol. 27, no. 1, p. 42-52, Jan. 1984.
[13] V.R. Basili and H.D. Rombach, "The Tame Project: Towards Improvement-Oriented Software Environments,"IEEE Trans. Software Eng., Vol. SE-14, No. 6, June 1988, pp. 758-773.
[14] IEEE Standards, "A standard classification of software errors, faults and failures,"Technical Committee on Software Engineering, vol. Standard P-1044/D3-Unapproved Draft, Dec. 1987.
[15] R. Chillarege, W.-L Kao, and R. G. Condit, "Defect type and its impact on the growth curve," inProc. 13th Int. Conf. Software Engineering, 1991.
[16] IBM MHVPL PTM Work Group, "Requirements for program trouble memoranda," IBM Mid-Hudson Valley Programming Lab, Wappingers Falls, NY, 1992.
[17] A. Dooley, "Specifying a change management process and tool,"In Preparation, IBM Mid-Hudson Valley Programming Lab, Wappingers Falls, NY, 1992.
[18] IBM Research,Working Document, Orthogonal Defect Classilication-Data Definitions and Requirements, Request Document: Author' s address, 1992.
[19] R. Chillarege and D. P. Siewiorek, "Experimental evaluation of computer systems reliability,"IEEE Trans. Reliability, pp. 403-408, vol. 39, Oct. 1990.
[20] R. A. Radice and R. W. Phillips,Software Engineering: An Industrial Approach. Englewood Cliffs, NJ: Prentice Hall, 1988.
[21] B. Ray, I. Bhandari, and R. Chillarege, "Reliability growth for typed defects," inProc. IEEE Reliability and Maintainability Symp., pp. 327-336, 1992.
[22] M. Sullivan and R. Chillarege, "Software defects and their impact on system availability-A study of field failures in operating systems,"Digest of Papers: The 21st Int. Symp. Fault-Tolerant Computing, pp. 2-9, 1991.
[23] I. Bhandari, M-Y. Wong, R. Chillarege, B. Ray, and D. Choi, "An inference structure for process feedback: Technique and implementation," IBM Research,RC 17869, 1992.
[24] M. Sullivan and R. Chillarege, "A comparison of software defects in database management systems and operating systems,"Digest of Papers: 22nd Int. Symp. Fault-Tolerant Computing, pp. 475-484, 1992.
[25] J.K. Chaar, M.J. Halliday, I.S. Bhandari, and R. Chillarege, "In-process metrics for software inspection and test evaluations,"Tech. Rep. 80725, IBM Research, 1992.

Index Terms:
orthogonal defect classification; software development; in-process measurements; feedback; semantic information; cause-effect relationships; measurement and analysis methods; necessary and sufficient conditions; defect trigger distribution; completeness; verification processes; inspection; testing; software quality; software reliability
R. Chillarege, I.S. Bhandari, J.K. Chaar, M.J. Halliday, D.S. Moebus, B.K. Ray, M.-Y. Wong, "Orthogonal Defect Classification-A Concept for In-Process Measurements," IEEE Transactions on Software Engineering, vol. 18, no. 11, pp. 943-956, Nov. 1992, doi:10.1109/32.177364
Usage of this product signifies your acceptance of the Terms of Use.