This Article 
 Bibliographic References 
 Add to: 
Towards a Framework for Software Measurement Validation
December 1995 (vol. 21 no. 12)
pp. 929-944
In this paper we propose a framework for validating software measurement. We start by defining a measurement structure model that identifies the elementary component of measures and the measurement process, and then consider five other models involved in measurement: unit definition models, instrumentation models, attribute relationship models, measurement protocols and entity population models. We consider a number of measures from the viewpoint of our measurement validation framework and identify a number of shortcomings; in particular we identify a number of problems with the construction of function points. We also compare our view of measurement validation with ideas presented by other researchers and identify a number of areas of disagreement. Finally, we suggest several rules that practitioners and researchers can use to avoid measurement problems, including the use of measurement vectors rather than artificially contrived scalars.

[1] A. Albrecht,“Measuring application development productivity,” Proc. IBM Applications Development Joint SHARE/GUIDE Symp.,Monterey, Calif., pp.83-92, 1979.
[2] B. Boehm, Software Engineering Economics, Prentice Hall, Upper Saddle River, N.J., 1981, pp. 533-535.
[3] S.R. Chidamber and C.F. Kemerer, "A Metrics Suite for Object Oriented Design," IEEE Trans. Software Eng., vol. 20, no. 6, pp. 476-493, 1994.
[4] N. Fenton, "Software Measurement: A Necessary Scientific Bias," IEEE Trans. Software Eng., vol. 20, pp. 199-206, Mar. 1994.
[5] N.E. Fenton, Software Metrics, A Rigorous Approach. Chapman&Hall, 1991.
[6] N. Fenton and B. Kitchenham,“Validating software measures,” J. of Software Technology, Verification and Reliability, vol. 1, no. 2, pp. 27-42, 1991.
[7] N. Fenton,R. Whitty,, and A. Kaposi,“A generalised mathematical theory of structured programming,” Theoretical Computer Science, vol. 36, pp. 145-171, 1985.
[8] J. Fleiss,“Measuring nominal scale agreement among many raters,” Psychological Bull., vol. 76, pp. 378-382, 1971.
[9] M.H. Halstead, Elements of Software Science. North-Holland, 1977.
[10] S. Henry and D. Kafura,“Software structure metrics based on information flow,” IEEE Transactions on Software Engineering, vol. 7, no. 5, pp. 510-518, May 1981.
[11] B.A. Kitchenham, L.M. Pickard, and S.J. Linkman, “An Evaluation of Some Design Metrics,” Software Eng. J., vol 5, no. 1, pp. 50–58, 1990.
[12] B. Kitchenham,“Using function points for software cost estimation,” N. Fenton, R. Whitty, and Y. Iizuka, eds., Software Quality Assurance and Measurement. International Thomson Press, 1995, pp. 266-280.
[13] A.C. Melton et al., "Mathematical Perspective of Software Measures Research," Software Eng. J., Vol. 5, No. 5, 1990, pp. 246-254.
[14] S. Pfleeger,“Experimental design and analysis in software engineering,” Annals of Software Eng., vol. 1, no. 1, pp. 219-253, Jan. 1995.
[15] K. Popper,The Logic of Scientific Discovery, 10th impression (revised). London: Hutchinson Co. Publishers Ltd., 1980, chapter 4, pp. 78-92.
[16] N. Schneidewind, "Methodology for Validating Software Metrics," IEEE Trans. Software Eng., vol. 18, pp. 410-421, May 1992.
[17] M. Shepperd, "Design Metrics: An Empirical Analysis," Software Engineering J., vol. 5, pp. 3-10, 1990.
[18] S. Siegel and N. Castellan, Jr.,Nonparametric Statistics for the Behavioral Sciences, 2nd edition. New York: McGraw-Hill Book Company, 1988, chapter 9, pp. 224-312.
[19] C.R. Symons, “Function Point Analysis: Difficulties and Improvements,” IEEE Trans. on Software Eng., vol. 14, no.1, Jan. 1988.
[20] E.J. Weyuker, "Evaluating Software Complexity Measures," IEEE Trans. Software Eng., Vol. 14, No. 9, 1988, pp. 1357-1365.
[21] H. Zuse, "Support of Experimentation by Measurement Theory," Experimental Software Engineering Issues (LNCS vol. 706), H.D. Rombach, V.R. Basili, and R.W. Selby, eds., pp. 137-140. Springer-Verlag, 1993.

Index Terms:
Measurement theory, software measurement, software metrics validation.
Barbara Kitchenham, Shari Lawrence Pfleeger, Norman Fenton, "Towards a Framework for Software Measurement Validation," IEEE Transactions on Software Engineering, vol. 21, no. 12, pp. 929-944, Dec. 1995, doi:10.1109/32.489070
Usage of this product signifies your acceptance of the Terms of Use.