The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - July/August (2008 vol.25)
pp: 82-88
Les Hatton , Kingston University, London
ABSTRACT
Checklists are an important part of code and design inspections. Ideally, they aim to increase the number of faults found per inspection hour by highlighting known areas of previous failure. In practice, although some researchers have quantified checklists' benefits, the conclusions' statistical robustness hasn't been as well represented. The author subjects checklists' effectiveness to formal statistical testing, using data from 308 inspections by industrial engineers over a three-year period. The results showed no evidence that checklists significantly improved these inspections. Further analysis revealed that individual inspection performance varied by a factor of 10 in terms of faults found per unit time, and individuals found on average about 53 percent of the faults. Two-person teams found on average 76 percent of the faults.
INDEX TERMS
software failure, code inspections, testing
CITATION
Les Hatton, "Testing the Value of Checklists in Code Inspections", IEEE Software, vol.25, no. 4, pp. 82-88, July/August 2008, doi:10.1109/MS.2008.100
REFERENCES
1. The Challenge of Complex IT Projects, tech. report, Royal Academy of Eng., 2004, www.bcs.org/upload/pdfcomplexity.pdf.
2. M. Fagan, "Design and Code Inspections to Reduce Errors in Program Development," IBM Systems J., vol. 15, no. 3, 1976, pp. 182–211.
3. L. Hatton, "Some Empirical Test Observations in Client/Server Systems," Computer, vol. 40, no. 5, 2007, pp. 24–29.
4. B. Brykczynski, "A Survey of Software Inspection Checklists," ACM SIGSOFTSoftware Eng. Notes, vol. 24, no. 1, 1999, pp. 82–89.
5. T. Gilb and D. Graham, Software Inspection, Addison-Wesley, 1993.
6. W. Humphrey, A Discipline of Software Engineering, Addison-Wesley, 1995.
7. E. Adams, "Optimising Preventive Service of Software Products," IBM J. Research and Development, vol. 1, no. 28, 1984, pp. 2–14.
8. B. Boehm, Software Engineering Economics, Prentice-Hall, 1981.
9. V.R. Basili et al., "The Empirical Investigation of Perspective-Based Reading," Empirical Software Eng., vol. 1, no. 2, 1996, pp. 133–164.
10. Z.E. Schnabel, "The Estimation of the Total Fish Population in a Lake," Am. Math. Monthly, vol. 45, no. 6, 1938, pp. 348–352.
11. L. Hatton, Safer C: Developing Software in High-Integrity and Safety-Critical Systems, McGraw-Hill, 1995.
12. L. Hatton, "EC-a Measurement-Based Safer Subset of ISO C Suitable for Embedded System Development," Information and Software Technology, vol. 47, no. 3, 2005, pp. 181–187.
13. L. Hatton, "Safer Language Subsets: An Overview and a Case History, MISRAC," Information and Software Technology, vol. 46, no. 7, 2004, pp. 465–472.
14. M.R. Spiegel and L.J. Stephens, Schaum's Outline of Statistics, 3rd ed., McGraw-Hill, 1999.
15. H. Sackman, W.J. Erikson, and E.E. Grant, "Exploratory Experimental Studies Comparing Online and Offline Programming Performance," Comm. ACM, vol. 11, no. 1, 1968, pp. 3–11.
16. L. Prechelt, "Technical Opinion: Comparing Java vs. C/C++, Efficiency Differences to Interpersonal Differences," Comm. ACM, vol. 42, no. 10, 1999, pp. 109–112.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool