This Article 
 Bibliographic References 
 Add to: 
A Controlled Experiment to Assess the Benefits of Procedure Argument Type Checking
April 1998 (vol. 24 no. 4)
pp. 302-312

Abstract—Type checking is considered an important mechanism for detecting programming errors, especially interface errors. This report describes an experiment to assess the defect-detection capabilities of static, intermodule type checking. The experiment uses ANSI C and Kernighan&Ritchie (K&R) C. The relevant difference is that the ANSI C compiler checks module interfaces (i.e., the parameter lists calls to external functions), whereas K&R C does not. The experiment employs a counterbalanced design in which each of the 40 subjects, most of them CS PhD students, writes two nontrivial programs that interface with a complex library (Motif). Each subject writes one program in ANSI C and one in K&R C. The input to each compiler run is saved and manually analyzed for defects. Results indicate that delivered ANSI C programs contain significantly fewer interface defects than delivered K&R C programs. Furthermore, after subjects have gained some familiarity with the interface they are using, ANSI C programmers remove defects faster and are more productive (measured in both delivery time and functionality implemented).

[1] V.R. Basili and B.T. Perricone,“Software errors and complexity: An empirical investigation,” Comm. ACM, vol. 27, no. 1, pp. 42-52, Jan. 1984.
[2] B. Beizer, Software Testing Techniques, second ed. Boston, Mass.: Int'l Thomson Computer Press, 1990.
[3] K. Bruce, "Typing in Object-Oriented Languages: Achieving Expressibility and Safety," ACM Computing Surveys? 1998, to appear. seehttp:/
[4] L.B. Christensen, Experimental Methodology. Allyn and Bacon, Needham Heights, Mass., sixth edition, 1994.
[5] C.R. Cook, J.C. Scholtz, and J.C. Spohrer, eds.. Empirical Studies of Programmers: Fifth Workshop,Palo Alto, Calif., Ablex Publishing Corp., Dec. 1993
[6] A. Ebrahimi, "Novice Programmer Errors: Language Constructs and Plan Composition," Intl. J. Human-Computer Studies, vol. 41, pp. 457-480, 1994.
[7] M. Eisenstadt, "Tales of Debugging from the Front Lines," [5], pp. 86-112, 1993.
[8] P.G. Frankl and S.N Weiss,“An experimental comparison of the effectiveness of branch testing and data flow testing,” IEEE Transactions on Software Engineering, vol. 19, no. 8, Oct. 1993, pp. 774-787.
[9] J.D. Gannon, "An Experimental Evaluation of Data Type Conventions," Comm. ACM, 1977.
[10] R.B. Grady, "Practical Results from Measuring Software Quality," Comm. ACM, vol. 36, no. 11, pp. 62-68, Nov. 1993.
[11] P. Hudak and M.P. Jones, "Haskell vs. Ada vs. C++ vs. Awk vs. ... An Experiment in Software Prototyping Productivity," technical report, Dept. of Computer Science, Yale Univ., New Haven, Conn., July 1994.
[12] W.S. Humphrey, A Discipline for Software Engineering, Addison-Wesley, Reading, Mass., 1995.
[13] M. Nanja and C.R. Cook, "An Analysis of the On-Line Debugging Process," [14], pp. 172-184, 1987.
[14] G.M. Olson, S. Sheppard, and E. Soloway, eds., "Empirical Studies of Programmers: Second Workshop,"Washington, D.C., Ablex Publishing Corp., Dec. 1987.
[15] L. Prechelt and W.F. Tichy, "A Controlled Experiment Measuring the Impact of Procedure Argument Type Checking on Programmer Productivity," Technical Report CMU/SEI-96-TR-014, Software Engineering Inst., Carnegie Mellon Univ., Pittsburgh, Penn., June 1996.
[16] B.A. Sheil, "The Psychological Study of Programming," ACM Computing Surveys, 1981.
[17] E. Soloway, and S. Iyengar, eds., Empirical Studies of Programmers.Norwood, N.J.: Ablex Publishing Corp., June 1986. (The papers of the First Workshop on Empirical Studies of Programmers, Washington D.C.).
[18] J.G. Spohrer and E. Soloway, "Analyzing the High Frequency Bugs in Novice Programs," [17], pp. 230-251, 1986.
[19] W. Stacy and J. MacMillian, "Cognitive Bias in Software Engineering," Comm. ACM, vol. 38, no. 6, pp. 57-63, June 1995.
[20] B. Teasley, L.M. Leventhal, and D.S. Rohlman, "Positive Test Bias in Software Testing by Professionals: What's Right and What's Wrong," Empirical Studies of Programmers: Proc. Fifth Workshop, pp. 206-221.Palo Alto, Calif.: Ablex Publishing Corp., Dec. 1993.
[21] N. Wirth, "Gedanken zur Software-Explosion," Informatik Spektrum, vol. 17, no. 1, pp. 5-20, Feb. 1994.
[22] C. Wohlin and P. Runeson, "Certification of Software Components," IEEE Trans. Software Eng., vol. 20, no. 6, pp. 494-499, June 1994.

Index Terms:
Type checking, defects, quality, productivity, controlled experiment.
Lutz Prechelt, Walter F. Tichy, "A Controlled Experiment to Assess the Benefits of Procedure Argument Type Checking," IEEE Transactions on Software Engineering, vol. 24, no. 4, pp. 302-312, April 1998, doi:10.1109/32.677186
Usage of this product signifies your acceptance of the Terms of Use.