This Article 
 Bibliographic References 
 Add to: 
An Operational Process for Goal-Driven Definition of Measures
December 2002 (vol. 28 no. 12)
pp. 1106-1125

Abstract—We propose an approach (GQM/MEDEA) for defining measures of product attributes in software engineering. The approach is driven by the experimental goals of measurement, expressed via the GQM paradigm, and a set of empirical hypotheses. To make the empirical hypotheses quantitatively verifiable, GQM/MEDEA supports the definition of theoretically valid measures for the attributes of interest based on their expected mathematical properties. The empirical hypotheses are subject to experimental verification. This approach integrates several research contributions from the literature into a consistent, practical, and rigorous approach.

[1] V. Barnett and T. Price, Outliers in Statistical Data, Third ed., John Wiley&Sons, 1995.
[2] R. Van Solingen, “The Goal/Question/Metric Approach,” Encyclopedia of Software Engineering—2 Volume Set, pp. 578-583, 2002.
[3] V.R. Basili, “The Experience Factory and its Relationship to Other Improvement Paradigms,” Proc. Fourth European Software Eng. Conf. (ESEC), Sept. 1993.
[4] V.R. Basili, G. Caldiera, and D.H. Rombach, “The Experience Factory,” Encyclopedia of Software Eng.—2 Volume Set, pp. 511-519, 2002.
[5] V.R. Basili and H.D. Rombach, "The TAME Project: Towards Improvement-Oriented Software Environments," IEEE Trans. Software Eng., Vol. 14, No. 6, 1988, pp. 758-773.
[6] V.R. Basili and D. Weiss, “A Methodology for Collecting Valid Software Engineering Data,” IEEE Trans. Software Eng., vol. 10, no. 11, pp. 758-773, Nov. 1984.
[7] V.R. Basili, M. Zelkowitz, F. McGarry, J. Page, S. Waligora, and R. Pajerski, “Special Report: SEL's Software Process-Improvement Program,” IEEE Software, vol. 12, no. 6, pp. 83-87, Nov. 1995.
[8] J. Bieman, A. Baker, P. Clites, D. Gustafson, and A. Melton, “A Standard Representation of Imperative Language Programs for Data Collection and Software Measures Specification,” J. Systems and Software, vol. 8, pp. 13-37, 1988.
[9] L.C. Briand,V.R. Basili,, and C.J. Hetmanski,“Developing interpretable models with optimized set reduction for identifying high-risk software components,” IEEE Transactions on Software Engineering, vol. 19, no. 11, pp. 1,028-1,044, Nov. 1993.
[10] L.C. Briand, V.R. Basili, and Y.-M. Kim, “Change Analysis Process to Characterize Software Maintenance Projects,” Proc. Int'l Conf. Software Maintenance, pp. 38-49, Victoria, B.C., Canada, Sept. 1994.
[11] L.C. Briand, V.R. Basili, and W.M. Thomas, "A Pattern Recognition Approach for Software Engineering Data Analysis," IEEE Trans. Software Eng., vol. 18, no. 11, pp. 931-942, 1992.
[12] L. Briand and J. Wuest, “Empirical Studies of Quality Models in Object-Oriented Systems,” Advances in Computers, Academic Press, 2002.
[13] L. Briand, C. Differding, and D. Rombach, “Practical Guidelines for Measurement-Based Process Improvement,” Software Process Improvement and Practice, 1997, also available as Technical Report ISERN-96-05, Fraunhofer Inst. for Experimental Software Eng., Germany, 1996.
[14] L. Briand, K. El Emam, and S. Morasca, “Theoretical and Empirical Validation of Software Product Measures,” Technical Report ISERN-95-03, Fraunhofer Inst. for Experimental Software Eng., Germany, 1995.
[15] L. Briand, K. El Emam, and S. Morasca, “On the Application of Measurement Theory to Software Engineering,” Empirical Software Eng.–An Int'l J., vol. 1, no. 1, 1996.
[16] L. Briand and S. Morasca, “Software Measurement and Formal Methods: A Case Study Centered on TRIO+ Specifications,” Proc. First Int'l Conf. Formal Eng. Methods (ICFEM '97), Nov. 1997.
[17] L. Briand, S. Morasca, and V. Basili, "Measuring and Assessing Maintainability at the End of High-Level Design," Proc. IEEE Conf. Software Maintenance,Montreal, Sept. 1993.
[18] L.C. Briand, S. Morasca, and V.R. Basili, "Property-Based Software Engineering Measurement," IEEE Trans. Software Eng., vol. 22, no. 1, pp. 68-85, Jan. 1996.
[19] L. Briand, S. Morasca, and V. Basili, "Defining and Validating High-Level Design Metrics," Technical Report CS-TR 3301, Univ. of Maryland, 1994; to be published in IEEE Trans. Software Eng.
[20] L. Briand, J. Wüst, S. Ikonomovski, and H. Lounis, “A Comprehensive Investigation of Quality Factors in Object-Oriented Designs: An Industrial Case Study,” Proc. Int'l Conf. Software Eng. (ICSE '99), 1999.
[21] L. Briand, J. Wuest, J. Daly, and V. Porter, “Exploring the Relationships between Design Measures and Software Quality in Object-Oriented Systems,” J. Systems and Software, vol. 51, 2000.
[22] T. De Marco, Structured Analysis and System Specification. Yourdon Press Computing Series, 1978.
[23] W. Dillon and M. Goldstein, Multivariate Analysis: Methods and Applications. Wiley&Sons, 1984.
[24] N. Fenton, "Software Measurement: A Necessary Scientific Bias," IEEE Trans. Software Eng., vol. 20, pp. 199-206, Mar. 1994.
[25] N. Fenton and A. Melton,“Deriving structurally based software measures,” J. Systems and Software, vol. 12, no. 3, pp. 177-187, 1990.
[26] N. Fenton and S. Pfleeger, Software Metrics, A Practical and Rigorous Approach. Thomson Computer Press, 1996.
[27] D.A. Gustafson et al., “Software Measure Specification,” Proc.SIGSOFT Software Eng. Notes, vol. 18, no. 5, pp. 163-168, 1993.
[28] P.G. Hoel, Introduction to Mathematical Statistics. Wiley and Sons, 1984.
[29] D. Ince and M. Sheppard, "System Design Metrics: A Review and Perspective," Proc. IEE/BCS Conf. Software Eng. '88, Liverpool Univ., pp. 23-27, July 1988.
[30] ISO/IEC 9126-1, “Software Product Quality,” Part 1: Quality Model, 1991.
[31] B. Kitchenham, “An Evaluation of Software Structure Metrics,” Proc. 12th Int'l Computer Software and Applications Conf. (COMPSAC `88), 1988.
[32] M. Lewis-Beck, Applied Regression. SAGE Publications, 1980.
[33] A.C. Melton et al., "Mathematical Perspective of Software Measures Research," Software Eng. J., Vol. 5, No. 5, 1990, pp. 246-254.
[34] S. Morasca and L. Briand, “Towards a Theoretical Framework for Measuring Software Attributes,” Proc. IEEE Symp. Software Metrics 1997.
[35] A. Morzenti and P. San Pietro, “Object-Oriented Logic Specifications of Time Critical Systems,” ACM Trans. Software Eng. and Methodology, vol. 3, no. 1, pp. 56-98, Jan. 1994.
[36] L. Moser, “Data Dependency Graphs for Ada Programs,” IEEE Trans. Software Eng., vol. 16, no. 5, pp. 498-509, May 1990.
[37] E.I. Oviedo, “Control Flow, Data Flow, and Program Complexity,” Proc. Fourth Int'l Computer Software and Applications Conf. (COMPSAC '80), pp. 146-152, Nov. 1980.
[38] N. Schneidewind, "Methodology for Validating Software Metrics," IEEE Trans. Software Eng., vol. 18, pp. 410-421, May 1992.
[39] P. Spector, Research Designs. SAGE Publications, 1981.
[40] J. Tian and M.V. Zelkowitz,“A formal model of program complexity and its application,” J. Systems and Software, vol. 17, no. 3, pp. 253-266, 1992.
[41] Specification of the Unified Modeling Language (UML), version 1.4,http:/, 2001.
[42] E.J. Weyuker, "Evaluating Software Complexity Measures," IEEE Trans. Software Eng., Vol. 14, No. 9, 1988, pp. 1357-1365.
[43] H. Zuse, Software Complexity: Measures and Methods. deGruyter, 1990.

Index Terms:
Software measurement, software quality, goal-question-metric paradigm.
Lionel C. Briand, Sandro Morasca, Victor R. Basili, "An Operational Process for Goal-Driven Definition of Measures," IEEE Transactions on Software Engineering, vol. 28, no. 12, pp. 1106-1125, Dec. 2002, doi:10.1109/TSE.2002.1158285
Usage of this product signifies your acceptance of the Terms of Use.