This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Testing the Value of Checklists in Code Inspections
July/August 2008 (vol. 25 no. 4)
pp. 82-88
Les Hatton, Kingston University, London
Checklists are an important part of code and design inspections. Ideally, they aim to increase the number of faults found per inspection hour by highlighting known areas of previous failure. In practice, although some researchers have quantified checklists' benefits, the conclusions' statistical robustness hasn't been as well represented. The author subjects checklists' effectiveness to formal statistical testing, using data from 308 inspections by industrial engineers over a three-year period. The results showed no evidence that checklists significantly improved these inspections. Further analysis revealed that individual inspection performance varied by a factor of 10 in terms of faults found per unit time, and individuals found on average about 53 percent of the faults. Two-person teams found on average 76 percent of the faults.

1. The Challenge of Complex IT Projects, tech. report, Royal Academy of Eng., 2004, www.bcs.org/upload/pdfcomplexity.pdf.
2. M. Fagan, "Design and Code Inspections to Reduce Errors in Program Development," IBM Systems J., vol. 15, no. 3, 1976, pp. 182–211.
3. L. Hatton, "Some Empirical Test Observations in Client/Server Systems," Computer, vol. 40, no. 5, 2007, pp. 24–29.
4. B. Brykczynski, "A Survey of Software Inspection Checklists," ACM SIGSOFTSoftware Eng. Notes, vol. 24, no. 1, 1999, pp. 82–89.
5. T. Gilb and D. Graham, Software Inspection, Addison-Wesley, 1993.
6. W. Humphrey, A Discipline of Software Engineering, Addison-Wesley, 1995.
7. E. Adams, "Optimising Preventive Service of Software Products," IBM J. Research and Development, vol. 1, no. 28, 1984, pp. 2–14.
8. B. Boehm, Software Engineering Economics, Prentice-Hall, 1981.
9. V.R. Basili et al., "The Empirical Investigation of Perspective-Based Reading," Empirical Software Eng., vol. 1, no. 2, 1996, pp. 133–164.
10. Z.E. Schnabel, "The Estimation of the Total Fish Population in a Lake," Am. Math. Monthly, vol. 45, no. 6, 1938, pp. 348–352.
11. L. Hatton, Safer C: Developing Software in High-Integrity and Safety-Critical Systems, McGraw-Hill, 1995.
12. L. Hatton, "EC-a Measurement-Based Safer Subset of ISO C Suitable for Embedded System Development," Information and Software Technology, vol. 47, no. 3, 2005, pp. 181–187.
13. L. Hatton, "Safer Language Subsets: An Overview and a Case History, MISRAC," Information and Software Technology, vol. 46, no. 7, 2004, pp. 465–472.
14. M.R. Spiegel and L.J. Stephens, Schaum's Outline of Statistics, 3rd ed., McGraw-Hill, 1999.
15. H. Sackman, W.J. Erikson, and E.E. Grant, "Exploratory Experimental Studies Comparing Online and Offline Programming Performance," Comm. ACM, vol. 11, no. 1, 1968, pp. 3–11.
16. L. Prechelt, "Technical Opinion: Comparing Java vs. C/C++, Efficiency Differences to Interpersonal Differences," Comm. ACM, vol. 42, no. 10, 1999, pp. 109–112.

Index Terms:
software failure, code inspections, testing
Citation:
Les Hatton, "Testing the Value of Checklists in Code Inspections," IEEE Software, vol. 25, no. 4, pp. 82-88, July-Aug. 2008, doi:10.1109/MS.2008.100
Usage of this product signifies your acceptance of the Terms of Use.