This Article 
 Bibliographic References 
 Add to: 
Evaluating Software Complexity Measures
September 1988 (vol. 14 no. 9)
pp. 1357-1365

A set of properties of syntactic software complexity measures is proposed to serve as a basis for the evaluation of such measures. Four known complexity measures are evaluated and compared using these criteria. This formalized evaluation clarifies the strengths and weaknesses of the examined complexity measures, which include the statement count, cyclomatic number, effort measure, and data flow complexity measures. None of these measures possesses all nine properties, and several are found to fail to possess particularly fundamental properties; this failure calls into question their usefulness in measuring synthetic complexity.

[1] A. L. Baker and S. H. Zweben, "A comparison of measures of control flow complexity,"IEEE Trans. Software Eng., vol. SE-6, no. 6, pp. 506-512, Nov. 1980.
[2] V. R. Basili and E. E. Katz, "Metrics of interest in Ada development," inProc. 1983 IEEE Computer Society Workshop Software Engineering Technology Transfer, 1983, pp. 22-29.
[3] V. R. Basili, R. W. Selby, and T.-Y. Phillips, "Metric analysis and data validation across Fortran projections,"IEEE Trans. Software Eng., vol. SE-9, no. 6, pp. 652-663, Nov. 1983.
[4] J. M. Bieman and W. R. Edwards, "Measuring data dependency complexity," Dep. Comput. Sci., Univ. Southwestern Louisiana, Tech. Rep. 83-5-3, July 1983.
[5] M. Blum, "On the size of machines,"Inform. Contr., vol. 11, pp. 257-265, 1967.
[6] B. Curtis, S. B. Sheppard, P. Milliman, M. A. Borst, and T. Love, "Measuring the psychological complexity of software maintenance tasks with the Halstead and McCabe metrics,"IEEE Trans. Software Eng., vol. SE-5, no. 2, pp. 96-104, Mar. 1979.
[7] M. D. Davis and E. J. Weyuker,Computability, Complexity, and Languages: Fundamentals of Theoretical Computer Science. New York: Academic Press, 1983.
[8] W. M. Evangelist, "Software complexity metric sensitivity to program structuring rules,"J. Syst. Software, vol. 3, pp. 231-243, 1982.
[9] R. D. Gordon, "Measuring improvements in program clarity,"IEEE Trans. Software Eng., vol. SE-5, no. 2, pp. 79-90, Mar. 1979.
[10] M. Halstead,Elements of Software Science. New York: Elsevier-North Holland, 1977.
[11] W. J. Hansen, "Measurement of program complexity by the pair (cyclomatic number, operator count)," ACM SIGPLAN Notices, vol. 13, no. 3, pp. 29-33, Mar. 1978.
[12] M. Hecht,Flow Analysis of Computer Programs. New York: Elsevier North-Holland, 1977.
[13] P. M. Herman, "A data flow analysis approach to program testing,"Australian Comput. J., vol. 8, no. 3, pp. 92-96, Nov. 1976.
[14] A. Iannino, J. D. Musa, K. Okumoto, and B. Littlewood, "Criteria for software reliability model comparisons,"IEEE Trans. Software Eng., vol. SE-10, no. 6, pp. 687-691, Nov. 1984.
[15] S. S. Iyengar, N. Parameswaran, and J. Fuller, "A measure of logical complexity of programs,"Comput. Lang., vol. 7, pp. 147-160, 1982.
[16] J. L. Knox and K. C. Tai, "An empirical evaluation of program complexity metrics," North Carolina State Univ., Tech. Rep. TR-84-06, 1984.
[17] J. W. Laski and B. Korel, "A data flow oriented program testing strategy,"IEEE Trans. Software Eng., vol. SE-9, no. 3, pp. 347- 354, May 1983.
[18] T. J. McCabe, "A complexity measure,"IEEE Trans. Software Eng., vol. SE-2, No. 4, pp. 308-320, Dec. 1976.
[19] G. J. Myers, "An extension to the cyclomatic measure of program complexity,"SIGPLAN Notices, vol. 12, no. 10, pp. 61-64, 1977.
[20] S. C. Ntafos, "On required element testing,"IEEE Trans. Software Eng., vol. SE-10, no. 6, pp. 795-803, Nov. 1984.
[21] E. I. Oviedo, "Control flow, data flow and program complexity," inProc. IEEE COMPSAC, Chicago, IL, Nov. 1980, pp. 146-152.
[22] P. Piwowarski, "A nesting level complexity measure,"ACM SIGPLAN Notices, vol. 17, no. 9, pp. 44-50, 1982.
[23] R. E. Prather, "An axiomatic theory of software complexity measure,"Comput. J., vol. 27, no. 4, pp. 340-346, 1984.
[24] S. Rapps and E. J. Weyuker, "Data flow analysis techniques for program test data selection," inProc. 6th Int. Conf. Software Eng., Sept. 1982, pp. 272-278.
[25] S. Rapps and E. J. Weyuker, "Selecting software test data using data flow information,"IEEE Trans. Software Eng., vol. SE-11, no. 4, pp. 367-375, Apr. 1985.
[26] M. Schaeffer,A Mathematical Theory of Global Program Optimization. Englewood Cliffs, NJ: Prentice-Hall, 1973.
[27] T. Sunohara, A. Takano, K. Uehara, and T. Ohkawa, "Program complexity measure for software development management," inProc. 5th Int. Conf. on Software Eng., 1981, pp. 100-106.
[28] K.-C. Tai, "A program complexity metric based on data flow information in control graphs," inProc. 7th Int. Conf. Software Engineering, Orlando, FL, Mar. 1984, pp. 239-248.
[29] E. J. Weyuker, "The applicability of program schema results to programs,"Int. J. Comput. Inform. Sci., vol. 8, no. 5, Nov. 1979.
[30] E. J. Weyuker, "Axiomatizing software test data adequacy,"IEEE Trans. Software Eng., vol. SE-12, no. 12, pp. 1128-1138, Dec. 1986.
[31] S. N. Woodfield, V. Y. Shen, and H. E. Dunsmore, "A study of several metrics for programming effort,"J. Syst. Software., vol. 2, pp. 97-103, 1981.
[32] M. R. Woodward, M. A. Hennell, and D. Hedley, "A measure of control flow complexity in program text,"IEEE Trans. Software Eng., vol. SE-5, no. 1, pp. 45-50, Jan. 1979.
[33] S. H. Zweben and K-C. Fung, "Exploring software science relations in COBOL and APL," inProc. IEEE COMPSAC, Chicago, IL, Nov. 1979, pp. 702-207.

Index Terms:
software engineering; software complexity measures; syntactic software complexity; statement count; cyclomatic number; effort measure; data flow complexity; synthetic complexity; software engineering
E.J. Weyuker, "Evaluating Software Complexity Measures," IEEE Transactions on Software Engineering, vol. 14, no. 9, pp. 1357-1365, Sept. 1988, doi:10.1109/32.6178
Usage of this product signifies your acceptance of the Terms of Use.