The Community for Technology Leaders
RSS Icon
Issue No.02 - Feb. (2013 vol.39)
pp: 197-215
Joseph Lawrance , Wentworth Institute of Technology, Boston
Christopher Bogart , Oregon State University, Corvallis
Margaret Burnett , Oregon State University, Corvallis
Rachel Bellamy , IBM TJ Watson Research Center, Hawthorne
Kyle Rector , Oregon State University, Corvallis
Scott D. Fleming , Oregon State University, Corvallis
Many theories of human debugging rely on complex mental constructs that offer little practical advice to builders of software engineering tools. Although hypotheses are important in debugging, a theory of navigation adds more practical value to our understanding of how programmers debug. Therefore, in this paper, we reconsider how people go about debugging in large collections of source code using a modern programming environment. We present an information foraging theory of debugging that treats programmer navigation during debugging as being analogous to a predator following scent to find prey in the wild. The theory proposes that constructs of scent and topology provide enough information to describe and predict programmer navigation during debugging, without reference to mental states such as hypotheses. We investigate the scope of our theory through an empirical study of 10 professional programmers debugging a real-world open source program. We found that the programmers' verbalizations far more often concerned scent-following than hypotheses. To evaluate the predictiveness of our theory, we created an executable model that predicted programmer navigation behavior more accurately than comparable models that did not consider information scent. Finally, we discuss the implications of our results for enhancing software engineering tools.
Debugging, Navigation, Topology, Programming environments, Predictive models, Approximation methods, empirical software engineering, Information foraging theory, debugging, software maintenance, programmer navigation, information scent
Joseph Lawrance, Christopher Bogart, Margaret Burnett, Rachel Bellamy, Kyle Rector, Scott D. Fleming, "How Programmers Debug, Revisited: An Information Foraging Theory Perspective", IEEE Transactions on Software Engineering, vol.39, no. 2, pp. 197-215, Feb. 2013, doi:10.1109/TSE.2010.111
[1] J.R. Anderson, "A Spreading Activation Theory of Memory," Verbal Learning and Verbal Behavior, vol. 22, pp. 261-295, 1983.
[2] J.R. Anderson, The Adaptive Character of Thought. Lawrence Erlbaum Assoc., 1990.
[3] R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval. Addison Wesley Longman, 1999.
[4] R. Brooks, "Towards a Theory of the Comprehension of Computer Programs," Int'l J. Man-Machine Studies, vol. 18, pp. 543-554, 1983.
[5] R. Brooks, "Toward a Theory of the Cognitive Processes in Computer Programming," Int'l J. Human-Computer Studies, vol. 51, pp. 197-211, 1999.
[6] E. Chi, P. Pirolli, K. Chen, and J. Pitkow, "Using Information Scent to Model User Information Needs and Actions on the Web," Proc. ACM Conf. Human Factors in Computing Systems, pp. 490-497, 2001.
[7] E. Chi, A. Rosien, G. Supattanasiri, A. Williams, C. Royer, C. Chow, E. Robles, B. Dalal, J. Chen, and S. Cousins, "The Bloodhound Project: Automating Discovery of Web Usability Issues Using the InfoScent Simulator," Proc. ACM Conf. Human Factors in Computing Systems, pp. 505-512, 2003.
[8] F. Crestani, "Application of Spreading Activation Techniques in Information Retrieval," Artificial Intelligence. Rev., vol. 11, no. 6, pp. 453-482, 1997.
[9] J. Corbin and A. Strauss, Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications, 2008.
[10] T. Cover and J. Thomas, Elements of Information Theory. John Wiley & Sons, 1991.
[11] D. Cubranic, G. Murphy, J. Singer, and K. Booth, "Hipikat: A Project Memory for Software Development," IEEE Trans. Software Eng., vol. 31, no. 6, pp. 446-465, June 2005.
[12] R. DeLine, M. Czerwinski, and G. Robertson, "Easing Program Comprehension by Sharing Navigation Data," Proc. IEEE Symp. Visual Languages and Human-Centric Computing, pp. 241-248, 2005.
[13] M. Eisenstadt, "Tales of Debugging from the Front Lines," Proc. Empirical Studies of Programmers: Fifth Workshop, pp. 86-112, 1993.
[14] K.A. Ericsson and H.A. Simon, Protocol Analysis: Verbal Reports as Data. MIT Press, 1993.
[15] P. Fritzson, T. Gyimóthy, M. Kamkar, N. Shahmehri, "Generalized Algorithmic Debugging and Testing," Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 317-326, 1991.
[16] D.J. Gilmore, "Models of Debugging," Acta Psychologica, vol. 78, pp. 151-172, 1991.
[17] J. Herlocker, J. Konstan, L. Terveen, and J. Riedl, "Evaluating Collaborative Filtering Recommender Systems," ACM Trans. Information Systems, vol. 22, no. 1, pp. 5-53, 2004.
[18] E. Hill, L. Pollock, and K. Vijay-Shanker, "Exploring the Neighborhood with Dora to Expedite Software Maintenance," Proc. Int'l Conf. Automated Software Eng., pp. 14-23, 2007.
[19] J. Hollan, E. Hutchins, and D. Kirsh, "Distributed Cognition: Toward a New Foundation for Human-Computer Interaction Research," ACM Trans. Computer-Human Interaction, vol. 7, pp. 174-196, 2000.
[20] I.R. Katz and J.R. Anderson, "Debugging: An Analysis of Bug-Location Strategies," Human-Computer Interaction, vol 3, pp. 351-399, 1988.
[21] M. Kersten and G. Murphy, "Mylar: A Degree of Interest Model for IDEs," Proc. Aspect-Oriented Software Development Conf., 2005.
[22] A.J. Ko, H. Aung, and B.A. Myers, "Eliciting Design Requirements for Maintenance-Oriented IDEs: A Detailed Study of Corrective and Perfective Maintenance Tasks," Proc. Int'l Conf. Software Eng., pp 126-135, 2005.
[23] A.J. Ko, B.A. Myers, M.J. Coblenz, and H.H. Aung, "An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks," IEEE Trans. Software Eng., vol. 32, no. 12, pp. 971-987, Dec. 2006.
[24] T.K. Landauer and S.T. Dumais, "A Solution to Plato's Problem: The Latent Semantic Analysis Theory of the Acquisition, Induction, and Representation of Knowledge," Psychological Rev., vol. 104, pp. 211-240, 1997.
[25] J. Lawrance, R. Bellamy, and M. Burnett, "Scents in Programs: Does Information Foraging Theory Apply to Program Maintenance?" Proc. IEEE Symp. Visual Languages and Human-Centric Computing, pp. 15-22, 2007.
[26] J. Lawrance, R. Bellamy, M. Burnett, and K. Rector, "Can Information Foraging Pick the Fix? A Field Study," Proc. IEEE Symp. Visual Languages and Human-Centric Computing, pp. 57-64, 2008.
[27] J. Lawrance, R. Bellamy, M. Burnett, and K. Rector, "Using Information Scent to Model the Dynamic Foraging Behavior of Programmers in Maintenance Tasks," Proc. ACM Conf. Human Factors in Computing Systems, pp. 1323-1332, 2008.
[28] A. Leontjev, Activity, Consciousness, and Personality. Prentice-Hall, 1978.
[29] S. Letovsky, "Cognitive Processes in Program Comprehension," Empirical Studies of Programmers, E. Soloway and S. Iyengar, eds., pp. 58-79, Ablex Publishing Corporation, 1986.
[30] F. Long, X. Wang, and Y. Cai, "API Hyperlinking via Structural Overlap," Proc. ESEC/ACM SIGSOFT Symp. Foundations of Software Eng., pp. 203-212, 2009.
[31] M. Nanja and C. Cook, "An Analysis of the On-Line Debugging Process," Proc. Second Workshop Empirical Studies of Programmers, E. Soloway and S. Sheppard, eds., pp. 172-184, 1987.
[32] J. Nielsen, "Information Foraging: Why Google Makes People Leave Your Site Faster," , June 2003.
[33] P. Pirolli, "Computational Models of Information Scent-Following in a Very Large Browsable Text Collection," Proc. ACM Conf. Human Factors in Computing Systems, pp. 3-10, 1997.
[34] P. Pirolli and S. Card, "Information Foraging," Psychology Rev., vol. 106, no. 4, pp. 643-675, 1999.
[35] M. Robillard, W. Coelho, and G. Murphy, "How Effective Developers Investigate Source Code: An Exploratory Study," IEEE Trans. Software Eng., vol. 30, no. 12, pp. 889-903, Dec. 2004.
[36] M. Robillard, "Automatic Generation of Suggestions for Program Investigation." Proc. ESEC/ACM SIGSOFT Symp. Foundations of Software Eng., pp. 11-20, 2005.
[37] M. Robillard, "Topology Analysis of Software Dependencies," ACM Trans. Software Eng. and Methodology, vol. 17, no. 4,article 18, 2008.
[38] P. Romero, B. du Boulay, R. Cox, R. Lutz, and S. Bryant, "Debugging Strategies and Tactics in a Multi-Representation Software Environment," Int'l J. Human-Computer Studies, vol. 65, no. 12, pp. 992-1009, Dec. 2007.
[39] Z.M. Saul, V. Filkov, P. Devanbu, and C. Bird, "Recommending Random Walks," Proc. ESEC/ACM SIGSOFT Symp. Foundations of Software Eng., pp. 15-24, 2007.
[40] C. Scaffidi, S. Fleming, D. Piorkowski, M. Burnett, R. Bellamy, and J. Lawrance, "Unifying Software Engineering Methods and Tools: Principles and Patterns from Information Foraging," in review.
[41] K. Schneider, C. Gutwin, R. Penner, and D. Paquette, "Mining a Software Developer's Local Interaction History," Proc. Int'l Workshop Mining Software Repositories, 2004.
[42] T. Schummer, "Lost and Found in Software Space," Proc. Hawaii Int'l Conf. System Sciences, 2001.
[43] J. Shirabad, T. Lethbridge, and S. Matwin, "Mining the Maintenance History of a Legacy System," Proc. Int'l Conf. Software Maintenance, 2003.
[44] J. Shrager and D. Klahr, "Instructionless Learning about a Complex Device," Int'l J. Man-Machine Studies, vol. 25, pp. 153-189, 1986.
[45] S.E. Sim and R.C. Holt, "The Ramp-Up Problem in Software Projects: A Case Study of How Software Immigrants Naturalize," Proc. Int'l Conf. Software Eng., pp. 361-370, Apr. 1998.
[46] J. Singer, R. Elves, and M.A. Storey, "NavTracks: Supporting navigation in Software Maintenance," Proc. Int'l Conf. Software Maintenance, pp. 325-334, 2005.
[47] D.I.K. Sjøberg, T. Dybå, B.C.D. Anda, and J.E. Hannay, "Building Theories in Software Engineering," Guide to Advanced Empirical Software Eng., pp. 312-336, Springer, 2008.
[48] J. Spool, C. Profetti, and D. Britain, "Designing for the Scent of Information," User Interface Eng., 2004.
[49] L. Suchman, Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge Univ. Press, 1987.
[50] A. Vans and A. von Mayrhauser, "Program Understanding Behavior during Corrective Maintenance of Large-Scale Software," Int'l J. Human-Computer Studies, vol. 51, pp. 31-70, 1999.
[51] C. Wohlin, P. Runeson, M. Host, B. Regnell, and A. Wesslen, Experimentation in Software Engineering. Kluwer Academic Publishers, 2000.
[52] A. Ying, G. Murphy, R. Ng, and M. Chu-Carroll, "Predicting Source Code Changes by Mining Change History," IEEE Trans. Software Eng., vol. 30, no. 9, pp. 574-586, Sept. 2004.
[53] A. Zeller, "Yesterday, My Program Worked. Today, It Does Not. Why?" Proc. ESEC/ACM SIGSOFT Symp. Foundations of Software Eng., pp. 253-267, 1999.
[54] T. Zimmermann, P. Weissgerber, S. Diehl, and A. Zeller, "Mining Version Histories to Guide Software Changes," Proc. IEEE Int'l Conf. Software Eng., 2004.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool