This Article 
 Bibliographic References 
 Add to: 
Asking and Answering Questions during a Programming Change Task
July/August 2008 (vol. 34 no. 4)
pp. 434-451
Jonathan Sillito, University of Calgary, Calgary
Gail C. Murphy, University of British Columbia, Vancouver
Kris De Volder, University of British Columbia, Vancouver
Little is known about the specific kinds of questions programmers ask when evolving a code base and how well existing tools support those questions. To better support the activity of programming, answers are needed to three broad research questions: (1) what does a programmer need to know about a code base when evolving a software system? (2) how does a programmer go about finding that information? and (3) how well do existing tools support programmer?s in answering those questions? We undertook two qualitative studies of programmers performing change tasks to provide answers to these questions. In this paper, we report on an analysis of the data from these two user studies. This paper makes three key contributions. The first contribution is a catalog of 44 types of questions programmers ask during software evolution tasks. The second contribution is a description of the observed behavior around answering those questions. The third contribution is a description of how existing deployed and proposed tools do, and do not, support answering programmers' questions.

[1] B. De Alwis and G.C. Murphy, “Using Visual Momentum to Explain Disorientation in the Eclipse IDE,” Proc. IEEE Symp. Visual Languages and Human Centric Computing, pp. 51-54, 2006.
[2] E. Baniassad and G. Murphy, “Conceptual Module Querying for Software Engineering,” Proc. Int'l Conf. Software Eng., pp. 64-73, 1998.
[3] V.R. Basili and R.W. Selby, “Comparing the Effectiveness of Software Testing Strategies,” IEEE Trans. Software Eng., vol. 13, no. 12, pp. 1278-1296, Dec. 1987.
[4] T.J. Biggerstaff, B.G. Mitbander, and D.E. Webster, “Program Understanding and the Concept Assignment Problem,” Comm. ACM, vol. 37, no. 5, pp. 72-82, 1994.
[5] R. Brooks, “Towards a Theory of the Comprehension of Computer Programs,” Int'l J. Man-Machine Studies, vol. 18, no. 6, pp. 543-554, 1983.
[6] M.H. Brown, Algorithm Animation. MIT Press, 1988.
[7] M.H. Brown, “Zeus: A System for Algorithm Animation and Multi-View Editing,” Proc. IEEE Workshop Visual Languages, pp. 4-9, 1991.
[8] K. Chen and V. Rajlich, “Case Study of Feature Location Using Dependence Graph,” Proc. 10th Int'l Workshop Program Comprehension, pp. 241-249, 2000.
[9] R. DeLine, A. Khella, M. Czerwinski, and G. Robertson, “Towards Understanding Programs through Wear-Based Filtering,” Proc. ACM Symp. Software Visualization, pp. 183-192, 2005.
[10] T. Eisenbarth and R. Koschke, “Locating Features in Source Code,” IEEE Trans. Software Eng., vol. 29, no. 3, pp. 210-224, Mar. 2003.
[11] A. Erdem, W.L. Johnson, and S. Marsella, “Task Oriented Software Understanding,” Proc. Automated Software Eng., pp.230-239, 1998.
[12] K. Erdos and H.M. Sneed, “Partial Comprehension of Complex Programs (Enough to Perform Maintenance),” Proc. Sixth Int'l Workshop Program Comprehension, pp. 98-105, 1998.
[13] N.V. Flor and E.L. Hutchins, “Analyzing Distributed Cognition in Software Teams: A Case Study of Team Programming during Perfective Software Maintenance,” Proc. Fourth Workshop Empirical Studies of Programmers, pp. 36-64, 1991.
[14] C. Flynt, Tcl/Tk: A Developer's Guide, second ed. Morgan Kaufmann, 2003.
[15] M.J. Fyson and C. Boldyreff, “Using Application Understanding to Support Impact Analysis,” J. Software Maintenance Research and Practice, vol. 10, no. 2, pp. 93-110, Dec. 1998.
[16] B.G. Glaser and A.L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine Publishing, 1967.
[17] N. Gold, “Hypothesis-Based Concept Assignment to Support Software Maintenance,” Proc. IEEE Int'l Conf. Software Maintenance, pp. 545-548, 2001.
[18] A. Goldberg, Smalltalk-80: The Interactive Programming Environment. Addison-Wesley, 1984.
[19] M.T. Harandi and J.Q. Ning, “Knowledge-Based Program Analysis,” IEEE Software, vol. 7, no. 1, pp. 74-81, Jan. 1990.
[20] M. Harman, N. Gold, R. Hierons, and D. Binkley, “Code Extraction Algorithms which Unify Slicing and Concept Assignment,” Proc. IEEE Working Conf. Reverse Eng., pp. 11-21, 2002.
[21] A. Hejlsberg, S. Wiltamuth, and P. Golde, The C# Programming Language, second ed. Addison Wesley Professional, 2006.
[22] S. Henninger, “Retrieving Software Objects in an Example-Based Programming Environment,” Proc. 14th Int'l ACM SIGIR Conf. Automated Software Eng., pp. 408-418, 1991.
[23] J.D. Herbsleb and E. Kuwana, “Preserving Knowledge in Design Projects: What Designers Need to Know,” Proc. Human Factors in Computing Systems, pp. 7-14, 1993.
[24] R. Holmes and G.C. Murphy, “Using Structural Context to Recommend Source Code Examples,” Proc. Int'l Conf. Software Eng., pp. 117-125, 2005.
[25] S. Ibrahim, H.B. Idris, and A. Deraman, “Case Study: Reconnaissance Techniques to Support Feature Location Using Recon2,” Proc. Asia-Pacific Software Eng. Conf., pp. 371-378, 2003.
[26] D. Jackson, “Aspect: An Economical Bug Detector,” Proc. Int'l Conf. Software Eng., pp. 13-22, 1994.
[27] D. Jackson and E.J. Rollins, “A New Model of Program Dependences for Reverse Engineering,” Proc. SIGSOFT Foundations of Software Eng. Conf., pp. 2-10, 1994.
[28] D. Janzen and K. De Volder, “Navigating and Querying Code without Getting Lost,” Proc. Int'l Conf. Aspect-Oriented Software Development, pp. 178-187, 2003.
[29] W.L. Johnson and A. Erdem, “Interactive Explanation of Software Systems,” Proc. Knowledge-Based Software Eng., pp. 155-164, 1995.
[30] T. Kamiya, S. Kusumoto, and K. Inoue, “Ccfinder: A Multilinguistic Token-Based Code Clone Detection System for Large Scale Source Code,” IEEE Trans. Software Eng., vol. 28, no. 7, pp.654-670, July 2002.
[31] A.J. Ko, R. DeLine, and G. Venolia, “Information Needs in Collocated Software Development Teams,” Proc. Int'l Conf. Software Eng., 2007.
[32] A.J. Ko and B.A. Myers, “Designing the Whyline: A Debugging Interface for Asking Questions about Program Failures,” Proc. Conf. Computer Human Interaction, pp. 151-158, 2004.
[33] A.J. Ko, B.A. Myers, M.J. Coblenz, and H.H. Aung, “An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks,” IEEE Trans. Software Eng., vol. 32, no. 12, pp. 971-987, Dec. 2006.
[34] J. Koenemann and S.P. Robertson, “Expert Problem Solving Strategies for Program Comprehension,” Proc. SIGCHI Conf. Human Factors in Computer Systems: Reaching through Technology, pp. 125-130, 1991.
[35] W. Kozaczynski, S. Letovsky, and J. Ning, “A Knowledge-Based Approach to Software System Understanding,” Proc. Knowledge-Based Software Eng. Conf., pp. 162-170, 1991.
[36] S. Letovsky, “Cognitive Processes in Program Comprehension,” Proc. Conf. Empirical Studies of Programmers, pp. 80-98, 1986.
[37] S. Letovsky, “Cognitive Processes in Program Comprehension,” J.Systems and Software, vol. 7, no. 4, pp. 325-339, Dec. 1987.
[38] D. Littman, J. Pinto, S. Letovsky, and E. Soloway, “Mental Models and Software Maintenance,” Proc. Conf. Empirical Studies of Programmers, pp. 80-98, 1986.
[39] S. Mancoridis, B. Mitchell, C. Rorres, Y.-F. Chen, and E. Gansner, “Using Automatic Clustering to Produce High-Level System Organizations of Source Code,” Proc. Int'l Workshop Program Comprehension, pp. 45-53, 1998.
[40] R. Mays, “Power Programming with the Lemma Code Viewer,” technical report, IBM TRP Networking Laboratory, 1996.
[41] N. Miyake, “Constructive Interaction and the Iterative Process of Understanding,” Cognitive Science, vol. 10, no. 2, pp. 151-177, 1986.
[42] H.A. Muller, M.A. Orgun, S.R. Tilley, and J.S Uhl, “A Reverse Engineering Approach to Subsystem Structure Identification,” Software Maintenance: Research and Practice, vol. 5, no. 4, pp. 181-204, 1993.
[43] B.A. Myers, R. Chandhok, and A. Sareen, “Automatic Data Visualization for Novice Pascal Programmers,” Proc. IEEE Workshop Visual Languages, pp. 192-198, 1988.
[44] P.D. O'Brien, D.C. Halbert, and M.F. Kilian, “The Trellis Programming Environment,” Proc. Conf. Object-Oriented Programming, Systems, and Applications, pp. 91-102, 1987.
[45] N. Pennington, “Stimulus Structures and Mental Representations in Expert Comprehension of Computer Programs,” Cognitive Psychology, vol. 19, pp. 295-341, 1987.
[46] S.P. Reiss, “Connecting Tools Using Message Passing in the Field Environment,” IEEE Software, vol. 7, no. 4, pp. 57-66, July 1990.
[47] X. Ren, F. Shah, F. Tip, B.G. Ryder, and O. Chesley, “Chianti: A Tool for Change Impact Analysis of Java Programs,” Proc. Object-Oriented Systems, Languages, Programming, and Applications, pp.432-448, 2004.
[48] M.P. Robillard, W. Coelho, and G.C. Murphy, “How Effective Developers Investigate Source Code: An Exploratory Study,” IEEE Trans. Software Eng., vol. 30, no. 12, pp. 889-903, Dec. 2004.
[49] M.P. Robillard and G.C. Murphy, “FEAT: A Tool for Locating, Describing, and Analyzing Concerns in Source Code,” Proc. Int'l Conf. Software Eng., pp. 822-823, 2003.
[50] B. Shneiderman, Software Psychology: Human Factors in Computer and Information Systems. Winthrop, 1980.
[51] B. Shneiderman and R. Mayer, “Syntactic/Semantic Interactions in Programmer Behavior: A Model and Experimental Results,” Int'l J. Computer and Information Sciences, vol. 8, no. 3, pp. 219-238, 1979.
[52] J. Sillito, G.C. Murphy, and K. De Volder, “Questions Programmers Ask during Software Evolution Tasks,” Proc. SIGSOFT Foundations of Software Eng. Conf., pp. 23-34, 2006.
[53] J. Sillito, K. De Volder, B. Fisher, and G. Murphy, “Managing Software Change Tasks: An Exploratory Study,” Proc. Int'l Symp. Empirical Software Eng., 2005.
[54] J. Singer, T. Lethbridge, N. Vinson, and N. Anquetil, “An Examination of Software Engineering Work Practices,” Proc. IBM Centre for Advanced Studies Conf., pp. 209-223, 1997.
[55] V. Sinha, D. Karger, and R. Miller, “Relo: Helping Users Manage Context during Interactive Exploratory Visualization of Large Codebases,” Proc. Visual Languages and Human-Centric Computing Conf., pp. 187-194, 2006.
[56] E. Soloway and K. Ehrlich, “Empirical Studies of Programming Knowledge,” IEEE Trans. Software Eng., vol. 10, no. 5, pp. 595-609, 1984.
[57] R.M. Stallman, “Emacs the Extensible, Customizable Self-Documenting Display Editor,” Proc. ACM SIGPLAN SIGOA Symp. Text Manipulation, pp. 147-156, 1981.
[58] J. Steffen, “Interactive Examination of a C Program with CSCOPE,” Proc. Usenix Winter Conf., pp. 170-175, 1985.
[59] J. Steven, P. Chandra, B. Fleck, and A. Podgurski, “Jrapture: A Capture/Replay Tool for Observation-Based Testing,” Proc. ACM SIGSOFT Int'l Symp. Software Testing and Analysis, pp. 158-167, 2000.
[60] M.-A.D. Storey, F.D. Fracchia, and H.A. Muller, “Cognitive Design Elements to Support the Construction of a Mental Model During Software Visualization,” Proc. Int'l Workshop Program Comprehension, pp. 17-28, 1997.
[61] M.-A.D. Storey, H.A. Muller, and K. Wong, “Manipulating and Documenting Software Structures,” Software Visualization, pp. 244-263, 1996.
[62] M.-A.D. Storey, K. Wong, and H.A. Muller, “How Do Program Understanding Tools Affect How Programmers Understand Programs?” Science of Computer Programming, vol. 36, nos. 2-3, pp.183-207, 2000.
[63] A.L. Strauss and J. Corbin, Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications, 1998.
[64] B. Stroustrup, The C++ Programming Language, second ed. Addison Wesley Longman, 1991.
[65] W. Teitelman and L. Masinter, “The Interlisp Programming Environment,” Computer, vol. 14, no. 4, pp. 25-34, Apr. 1981.
[66] D. Tidwell, XSLT, first ed. O'Reilly Media, 2001.
[67] M.W. van Someren, Y.F. Barnard, and J.A.C. Sandberg, The Think Aloud Method; A Practical Guide to Modelling Cognitive Processes. Academic Press, 1994.
[68] A. von Mayrhauser and A.M. Vans, “From Code Understanding Needs to Reverse Engineering Tool Capabilities,” Proc. Computer-Aided Software Eng., pp. 230-239, 1993.
[69] A. Walenstein, “Cognitive Support in Software Engineering Tools: A Distributed Cognition Framework,” PhD dissertation, Simon Fraser Univ., 2002.
[70] A. Walenstein, “Theory-Based Cognitive Support Analysis of Software Comprehension Tools,” Proc. Int'l Workshop Program Comprehension, pp. 75-84, 2002.
[71] M. Weiser, “Program Slicing,” Proc. Int'l Conf. Software Eng., pp.439-449, 1981.
[72] N. Wilde and C. Casey, “Early Field Experience with the Software Reconnaissance Technique for Program Comprehension,” Proc. Working Conf. Reverse Eng., pp. 270-276, 1996.
[73] L. Williams, R.R. Kessler, W. Cunningham, and R. Jeffries, “Strengthening the Case for Pair-Programming,” IEEE Software, vol. 17, no. 4, pp. 19-25, July/Aug. 2000.
[74] D.D. Woods, “Visual Momentum: A Concept to Improve the Cognitive Coupling of Person and Computer,” Int'l J. Man-Machine Studies, vol. 21, pp. 229-244, 1984.
[75] S.G. Woods, A.E. Quilici, and Q. Yang, Constraint-Based Design Recovery for Software Engineering: Theory and Experiments. Springer, 1997.
[76] X. Zhang and R. Gupta, “Cost Effective Dynamic Program Slicing,” Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 94-106, 2004.

Index Terms:
Software psychology, Programming Environments/Construction Tools, Enhancement
Jonathan Sillito, Gail C. Murphy, Kris De Volder, "Asking and Answering Questions during a Programming Change Task," IEEE Transactions on Software Engineering, vol. 34, no. 4, pp. 434-451, July-Aug. 2008, doi:10.1109/TSE.2008.26
Usage of this product signifies your acceptance of the Terms of Use.