The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - September (2005 vol.31)
pp: 733-753
Dag I.K. Sj?berg , IEEE Computer Society
ABSTRACT
The classical method for identifying cause-effect relationships is to conduct controlled experiments. This paper reports upon the present state of how controlled experiments in software engineering are conducted and the extent to which relevant information is reported. Among the 5,453 scientific articles published in 12 leading software engineering journals and conferences in the decade from 1993 to 2002, 103 articles (1.9 percent) reported controlled experiments in which individuals or teams performed one or more software engineering tasks. This survey quantitatively characterizes the topics of the experiments and their subjects (number of subjects, students versus professionals, recruitment, and rewards for participation), tasks (type of task, duration, and type and size of application) and environments (location, development tools). Furthermore, the survey reports on how internal and external validity is addressed and the extent to which experiments are replicated. The gathered data reflects the relevance of software engineering experiments to industrial practice and the scientific maturity of software engineering research.
INDEX TERMS
Index Terms- Controlled experiments, survey, research methodology, empirical software engineering.
CITATION
Dag I.K. Sj?berg, Jo E. Hannay, Ove Hansen, Vigdis By Kampenes, Amela Karahasanovic, Nils-Kristian Liborg, Anette C. Rekdal, "A Survey of Controlled Experiments in Software Engineering", IEEE Transactions on Software Engineering, vol.31, no. 9, pp. 733-753, September 2005, doi:10.1109/TSE.2005.97
5 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool