This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Coding Scheme to Support Systematic Analysis of Software Comprehension
July/August 1999 (vol. 25 no. 4)
pp. 526-540

Abstract—Protocol Analysis is a valuable tool for gaining qualitative data from observations of programmer behavior during software maintenance. However, there are some major drawbacks with Protocol Analysis as it is currently practiced. Firstly, Protocol Analysis requires a daunting amount of effort at each stage of analysis. Secondly, the results from one Protocol Analysis are often difficult to compare with results from another. This paper describes a coding scheme, AFECS, designed to reduce the effort required to perform Protocol Analysis and to resolve the problem of noncomparable results. AFECS uses codes that consist of expandable and flexible segments. This allows AFECS to be tailored to the requirements of a variety of research studies, while maintaining a degree of consistency. Explicit segmentation also makes AFECS easy to use. An example shows AFECS' use and ability to adapt to diverse research questions.

[1] B. Adelson and E. Soloway, “The Role of Domain Experience in Software Design,” IEEE Trans. Software Eng., vol. 11, Nov. 1985.
[2] J.R. Anderson, R. Farrell, and R. Sauers, “Learning to Program in LISP,” Cognitive Science, vol. 8, pp. 87–129, 1984.
[3] D. Bergantz and J. Hassell, “Information Relationships in PROLOG Programs: How Do Programmers Comprehend Functionality?” Int'l J. Man-Machine Studies, vol. 35, pp. 313–328, 1991.
[4] R. Brooks, “Towards a Theory of the Comprehension of Computer Programs,” Int'l J. Man-Machine Studies, vol. 18, pp. 543–554, 1983.
[5] F. Detienne, “Reasoning from a Schema and from an Analogy in Software Code Reuse,” J. Koenemann-Belliveau, T.G. Mohrer, and S.P. Robertson, eds., Proc. Fourth Workshop Empirical Studies of Programmers, pp. 5–21, Norwood,N.J.: Ablex Publishing, 1992.
[6] K. Ericsson and H.A. Simon, Protocol Analysis: Verbal Reports as Data, second edition, Cambridge, Mass: MIT Press, 1993.
[7] C. Fisher, “Advancing the Study of Programming with Computer-Aided Protocol Analysis,” G.M. Olson, S. Sheppard, and E. Soloway, eds., Proc. Second Workshop Empirical Studies of Programmers, pp. 198–216, Norwood,N.J.: Ablex Publishing, 1987.
[8] A.E. Howe and P.R. Cohen, “Detecting and Explaining Dependencies in Execution Traces,” Selecting Models from Data: Artificial Intelligence and Statistics IV, P. Cheeseman and R.W. Oldford, eds., pp. 161–182, Springer-Verlag, 1994.
[9] A.E. Howe and G. Somlo, “Modeling Discrete Event Sequences as State Transition Diagrams,” Proc. Second Int'l Symp. Intelligent Data Analysis, Aug. 1997.
[10] A. Lee and N. Pennington, "The Effects of Paradigm on Cognitive Activities in Design," Int'l J. Man-Machine Studies, vol. 40, pp. 577-601, 1994.
[11] S. Letovsky, “Cognitive Processes in Program Comprehension,” Proc. First Workshop Empirical Studies of Programmers, Ablex Publishing, Norwood, N.J., 1986, pp. 58-79.
[12] D. Littman, J. Pinto, S. Letovsky, and E. Soloway, Mental Models and Software Maintenance Empirical Studies of Programmers, Proc. First Workshop, pp. 80-98, 1996.
[13] N. Pennington, “Comprehension Strategies in Programming,” Proc. Second Workshop Empirical Studies of Programmers, Ablex Publishing, Norwood, N.J., 1987, pp. 100-112.
[14] N. Pennington, “Stimulus Structures and Mental Representations in Expert Comprehension of Computer Programs,” Cognitive Psychology, vol. 19, pp. 295–341, 1987.
[15] N. Pennington, A.Y. Lee, and B. Rehder, “Cognitive Activities and Levels of Abstraction in Procedural and Object-Oriented Design,” Human-Computer Interaction, vol. 10, pp. 171-226, 1995.
[16] R.S. Rist, “System Structure and Design,” Proc. Sixth Workshop Empirical Studies of Programmers, Norwood,N.J.: Ablex Publishing pp. 163–194, 1996.
[17] J. Scholtz and S. Wiedenbeck, “Learning New Programming Languages: An Analysis of the Process and Problems Encountered,” Behavior and Information Technology, vol. 11, no. 4, pp. 199–215, 1992.
[18] T.M. Shaft and I. Vessey, “Computer Program Comprehension Processes: the Effect of Application Domain Knowledge,” W.D. Gray and D.A. Boehm-Davis, eds., Proc. Sixth Workshop Empirical Studies of Programmers, Norwood,N.J.: Ablex Publishing, pp. 277–278, 1996.
[19] E. Soloway, “What to Do Next: Meeting the Challenge of Programming-in-the-Large,” E. Soloway and S. Iyengar, eds., Empirical Studies of Programmers, Norwood,N.J.: Ablex Publishing pp. 263–268, 1986.
[20] E. Soloway, B. Adelson, and K. Ehrlich, “Knowledge and Processes in the Comprehension of Computer Programs,” M. Chi, R. Glaser, and M. Farr, eds., The Nature of Expertise, pp. 129–152, Hillsdale, N.J.: Lawrence Erlbaum Assoc., 1988.
[21] L. Suchman, Plans and Situated Actions. The Problem of Human-Machine Communication, Cambridge Univ. Press, Cambridge, 1987.
[22] H.J. van Zuylen, “Views and Representations for Reverse Engineering,” D.J. Gilmore, R.L. Winder, and F. Détienne, eds., User-Centered Requirements for Software Eng. Environments, F: Computer&System Science, vol. 123, pp. 41–56, Berlin: Springer-Verlag, 1994.
[23] I. Vessey, “Expertise in Debugging Computer Programs: A Process Analysis,” Int'l J. Man-Machine Studies, vol. 23, pp. 459–494, 1985.
[24] A. von Mayrhauser and A. Vans, "From Program Comprehension to Tool Requirements for an Industrial Environment," Proc. Second Workshop on Program Comprehension,Capri, Italy, pp. 78-86, July 1993.
[25] A. von Mayrhauser and A. Vans, "From Code Understanding Needs to Reverse Engineering Tool Capabilities," Proc. Sixth Int'l Workshop on Computer-Aided Software Engineering (CASE93),Singapore, pp. 230-239, July 1993.
[26] A. von Mayrhauser and A. Vans, "Dynamic Code Cognition Behaviors For Large Scale Code," Proc. Third Workshop on Program Comprehension,Washington D.C., pp. 74-81, Nov.14-15, 1994.
[27] A. von Mayrhauser and A. Vans, "Industrial experience with an integrated code comprehension model," IEE Software Engineering J., pp. 171-182, Sept. 1995.
[28] A. von Mayrhauser and A.M. Mans, “Identification of Dynamic Comprehension Processes During Large Scale Maintenance,” IEEE Trans. Software Eng., vol. 22, no. 6, pp. 424–437, 1996.
[29] A.V. Mayrhauser and A. Vans, “On the Role of Hypotheses During Opportunistic Understanding while Porting Large Scale Code,” Proc. Fourth Workshop Program Comprehension., pp. 68–77, Mar. 1996.
[30] A. von Mayrhauser, A.M. Vans, A.E. Howe, “Program Understanding Behavior during Enhancement of Large-Scale Software,” Software Maintenance: Research and Practice vol. 9, pp. 299–327, 1997.
[31] A. von Mayrhauser and A.M. Vans, “Hypothesis-Driven Understanding Processes During Corrective Maintenance of Large Scale Software,” Int'l Conf. Software Maintenance'97, Bari, Italy, pp. 12–20, Oct. 1997.
[32] A. von Mayrhauser and S. Lang, “Evaluating Software Maintenance Support Tools for their Support of Program Comprehension,” Proc. IEEE Aerospace Conf., Snowmass, Colo.., Mar. 1998.

Index Terms:
Software maintenance, program comprehension, protocol analysis.
Citation:
Anneliese von Mayrhauser, Stephen Lang, "A Coding Scheme to Support Systematic Analysis of Software Comprehension," IEEE Transactions on Software Engineering, vol. 25, no. 4, pp. 526-540, July-Aug. 1999, doi:10.1109/32.799950
Usage of this product signifies your acceptance of the Terms of Use.