The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2013 vol.39)
pp: 930-953
Enrico di Bella , University of Genova, Genova
Ilenia Fronza , Free University of Bolzano - Bozen, Bolzano
Nattakarn Phaphoom , Free University of Bolzano - Bozen, Bolzano
Alberto Sillitti , Free University of Bolzano - Bozen, Bolzano
Giancarlo Succi , Free University of Bozen - Bolzano, Bolzano
Jelena Vlasenko , Free University of Bolzano - Bozen, Bolzano
ABSTRACT
In the last decade, there has been increasing interest in pair programming (PP). However, despite the existing work, there is still a lack of substantial evidence of the effects of PP in industrial environments. To address this issue, we have analyzed the work of a team of 17 industrial developers for 14 months. The team is part of the IT department of a large Italian manufacturing company; it adopts a customized version of extreme programming (XP). We have investigated the effects of PP on software quality in five different scenarios. The results show that PP appears to provide a perceivable but small effect on the reduction of defects in these settings.
INDEX TERMS
Programming, Software, case study, Pair programming, software defects
CITATION
Enrico di Bella, Ilenia Fronza, Nattakarn Phaphoom, Alberto Sillitti, Giancarlo Succi, Jelena Vlasenko, "Pair Programming and Software Defects--A Large, Industrial Case Study", IEEE Transactions on Software Engineering, vol.39, no. 7, pp. 930-953, July 2013, doi:10.1109/TSE.2012.68
REFERENCES
[1] H. Al-Kilidar, P. Parkin, A. Aurum, and R. Jeffery, "Evaluation of Effects of Pair Work on Quality of Designs," Proc. Australian Conf. Software Eng., pp. 78-87, 2005.
[2] E. Arisholm, H. Gallis, T. Dyba, and D.I.K. Sjoberg, "Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise," IEEE Trans. Software Eng., vol. 33, no. 2, pp. 65-86, Feb. 2007.
[3] P. Baheti, E.F. Gehringer, and P.D. Stotts, "Exploring the Efficacy of Distributed Pair Programming," Proc. Second XP Universe and First Agile Universe Conf. Extreme Programming and Agile Methods— XP/Agile Universe, pp. 208-220, 2002.
[4] V. Basili, "Applying the Goal Question Metric Paradigm in the Experience Factory," Proc. 10th Ann. Conf. Software Metrics and Quality Assurance in Industry, 1993.
[5] K. Beck, Extreme Programming Explained: Embrace Change. Addison-Wesley Professional, Oct. 1999.
[6] A. Begel and N. Nagappan, "Pair Programming: What's in It for Me?" Proc. ACM-IEEE Second Int'l Symp. Empirical Software Eng. and Measurement, pp. 120-128, 2008.
[7] E. Bellini, G. Canfora, A. Cimitile, F. Garcia, M. Piattini, and C. Visaggio, "The Impact of Educational Background on Design Knowledge Sharing during Pair Programming: An Empirical Study," Professional Knowledge Management, K.-D. Althoff, A. Dengel, R. Bergmann, M. Nick, and T. Roth-Berghofer, eds., vol. 3782, pp. 455-465, Springer, 2005.
[8] T. Bipp, A. Lepper, and D. Schmedding, "Pair Programming in Software Development Teams—An Empirical Study of Its Benefits," Information Software Technology, vol. 50, no. 3, pp. 231-240, 2008.
[9] G. Braught, J. MacCormick, and T. Wahls, "The Benefits of Pairing by Ability," Proc. 41st ACM Technical Symp. Computer Science Education, 2010.
[10] G. Canfora, A. Cimitile, C.A. Visaggio, F. Garcia, and M. Piattini, "Performances of Pair Designing on Software Evolution: A Controlled Experiment," Proc. European Conf. Software Maintenance and Reeng., pp. 197-205, 2006.
[11] G. Canfora, A. Cimitile, F. Garcia, M. Piattini, and C.A. Visaggio, "Evaluating Performances of Pair Designing in Industry," J. Systems Software, vol. 80, no. 8, pp. 1317-1327, 2007.
[12] J. Chong and T. Hurlbutt, "The Social Dynamics of Pair Programming," Proc. 29th Int'l Conf. Software Eng., pp 354-363, 2007.
[13] A. Cockburn and L. Williams, The Costs and Benefits of Pair Programming, pp. 223-243. Addison-Wesley Longman Publishing Co., Inc., 2001.
[14] I.D. Coman, A. Sillitti, and G. Succi, "Investigating the Usefulness of Pair-Programming in a Mature Agile Team," Proc. Ninth Int'l Conf. eXtreme Programming and Agile Processes in Software Eng., pp. 10-14, 2008.
[15] I.D. Coman, A. Sillitti, and G. Succi, "A Case-Study on Using an Automated In-Process Software Engineering Measurement and Analysis System in an Industrial Environment," Proc. 31st Int'l Conf. Software Eng., pp. 89-99, 2009.
[16] I. Coman, A. Sillitti, and G. Succi, "Ensuring Continuous Data Accuracy in AISEMA Systems," Proc. 23rd Int'l Conf. Software Eng. and Knowledge Eng., July 2011.
[17] E. Di Bella, A. Sillitti, and G. Succi, "A Multivariate Classification of Open Source Developers," Information Sciences, vol. 221, no. 1, pp. 72-83, Feb. 2013.
[18] M.A. Domino, R.W. Collins, and A.R. Hevner, "Controlled Experimentation on Adaptations of Pair Programming," Information Technology and Management, vol. 8, no. 4, pp. 297-312, 2007.
[19] R. Duque and C. Bravo, "Analyzing Work Productivity and Program Quality in Collaborative Programming," Proc. Third Int'l Conf. Software Eng. Advances, pp. 270-276, 2008.
[20] T. Dyba, E. Arisholm, D.I.K. Sjøberg, J.E. Hannay, and F. Shull, "Are Two Heads Better Than One? On the Effectiveness of Pair Programming," IEEE Software, vol. 24, no. 6, 12-15, Nov./Dec. 2007.
[21] R.L. Edwards, J.K. Stewart, and M. Ferati, "Assessing the Effectiveness of Distributed Pair Programming for an Online Informatics Curriculum," ACM Inroads, vol. 1, pp. 48-54, 2010.
[22] J. Favela, H. Natsu, C. Prez, O. Robles, A.L. Morn, R. Romero, A.M. Martnez-Enrquez, and D. Decouchant, "Empirical Evaluation of Collaborative Support for Distributed Pair Programming," Groupware: Design, Implementation and Use, G.-J. de Vreede, L.A. Guerrero, and G. Marn Ravents, eds., pp. 215-222, Springer, 2004.
[23] I. Fronza, A. Sillitti, and G. Succi, "An Interpretation of the Results of the Analysis of Pair Programming during Novices Integration in a Team," Proc. Third Int'l Symp. Empirical Software Eng. and Measurement, pp. 225-235, 2009.
[24] I. Fronza, A. Sillitti, G. Succi, and J. Vlasenko, "Analysing the Usage of Tools in Pair Programming Sessions," Proc. Int'l Conf. Agile Processes and eXtreme Programming in Software Eng., pp. 1-11, 2011.
[25] I. Fronza, A. Sillitti, G. Succi, and J. Vlasenko, "Understanding How Novices Are Integrated in a Team Analysing Their Tool Usage," Proc. Int'l Conf. Software and Systems Processes, May 2011.
[26] I. Fronza, A. Sillitti, G. Succi, M. Terho, and J. Vlasenko, "Failure Prediction Based on Log Files Using Random Indexing and Support Vector Machines," J. Systems and Software, vol. 86, no. 1, pp. 2-11, Jan. 2013.
[27] H. Gallis, E. Arisholm, and T. Dyba, "An Initial Framework for Research on Pair Programming," Proc. Int'l Symp. Empirical Software Eng., p. 132, 2003.
[28] B. Hanks, "Student Performance in CS1 with Distributed Pair Programming," Proc. 10th Ann. SIGCSE Conf. Innovation and Technology in Computer Science Education, pp. 316-320, 2005.
[29] J. Hannay and M. Jorgensen, "The Role of Deliberate Artificial Design Elements in Software Engineering Experiments," IEEE Trans. Software Eng., vol. 34, no. 2, pp. 242-259, Mar./Apr. 2008.
[30] J.E. Hannay, E. Arisholm, H. Engvik, and D.I.K. Sjoberg, "Effects of Personality on Pair Programming," IEEE Trans. Software Eng., vol. 36, no. 1, pp. 61-80, Jan./Feb. 2010.
[31] W. Härdle, Smoothing Techniques. Springer-Verlag, 1991.
[32] S. Heiberg, U. Puus, P. Salumaa, and A. Seeba, "Pair-Programming Effect on Developers Productivity," Proc. Int'l Conf. Agile Processes and eXtreme Programming in Software Eng., 2003.
[33] H. Hulkko and P. Abrahamsson, "A Multiple Case Study on the Impact of Pair Programming on Product Quality," Proc. 27th Int'l Conf. Software Eng., pp. 495-504, 2005.
[34] A. Janes, B. Russo, P. Zuliani, and G. Succi, "An Empirical Analysis of the Discontinuous Use of Pair Programming," Proc. Fourth Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 205-214, 2003.
[35] J.R. Katzenbach and D.K. Smith, The Wisdom of Teams: Creating the High-Performance Organization. Harper Business, 1994.
[36] M. Keeling, "Put It to the Test: Using Lightweight Experiments to Improve Team Processes," Proc. 11th Int'l Conf. Agile Processes in Software Eng. and Extreme Programming, vol. 48, pp. 287-296, 2010.
[37] B. Kitchenham, "Procedures for Performing Systematic Reviews," technical report, Keele Univ. and NICTA, 2004.
[38] A.G. Koru, D. Zhang, K. El Emam, and H. Liu, "An Investigation into the Functional Form of the Size-Defect Relationship for Software Modules," IEEE Trans. Software Eng., vol. 35, no. 2, pp. 293-304, Mar./Apr. 2009.
[39] K.M. Lui, K.C.C. Chan, and J. Nosek, "The Effect of Pairs in Program Design Tasks," IEEE Trans. Software Eng., vol. 34, no. 2, pp. 197-211, Mar./Apr. 2008.
[40] K.M. Lui, K.A. Barnes, and K.C.C. Chan, "Pair Programming: Issues and Challenges," Proc. Int'l Conf. Agile Software Development: Current Research and Future Directions. pp. 143-163, 2010.
[41] K.M. Lui and K.C.C. Chan, "When Does a Pair Outperform Two Individuals?" Proc. Fourth Int'l Conf. Extreme programming and agile Processes in Software Eng., pp. 225-233, 2003.
[42] Q. Li and J.S. Racine, Nonparametric Econometrics: Theory and Practice. Princeton Univ. Press, Nov. 2006.
[43] L. Madeyski, "The Impact of Pair Programming and Testdriven Development on Package Dependencies in Objectoriented Design —An Experiment," Proc. Seventh Int'l Product-Focused Software Process Improvement, vol. 4034, pp. 278-289, 2006.
[44] L. Madeyski, On the Effects of Pair Programming on Thoroughness and Fault-Finding Effectiveness of Unit Tests, J. Münch, P. Abrahamsson, eds., pp. 207-221. Springer, 2007.
[45] E. Mendes, L. Al-Fakhri, and A. Luxton-Reilly, "A Replicated Experiment of Pair-Programming in a Second-Year Software Development and Design Computer Science Course," Proc. 11th Ann. SIGCSE Conf. Innovation and Technology in Computer Science Education, pp. 108-112, 2006.
[46] R. Moser, M. Scotto, A. Sillitti, P. Abrahamsson, and G. Succi, "Does XP Deliver Quality and Maintainable Code?" Proc. Eighth Int'l Conf. eXtreme Programming and Agile Processes in Software Eng., pp. 18-22, 2007.
[47] C. McDowell, L. Werner, H.E. Bullock, and J. Fernald, "The Impact of Pair Programming on Student Performance, Perception and Persistence," Proc. 25th Int'l Conf. Software Eng., pp. 602-607, 2003.
[48] L. Murphy, S. Fitzgerald, B. Hanks, R. McCauley, "Pair Debugging: A Transactive Discourse Analysis," Proc. Sixth Int'l Workshop Computing Education Research, 2010.
[49] M.M. Müller, "Are Reviews an Alternative to Pair Programming?" Empirical Software Eng., vol. 9, no. 4, pp. 335-351, 2004.
[50] M.M. Müller, "Two Controlled Experiments Concerning the Comparison of Pair Programming to Peer Review," J. Systems and Software, vol. 78, no. 2, pp. 166-179, 2005.
[51] M.M. Müller, "A Preliminary Study on the Impact of a Pair Design Phase on Pair Programming and Solo Programming," Information and Software Technology, vol. 48, 335-344, May 2006.
[52] M.M. Müller, "Do Programmer Pairs Make Different Mistakes than Solo Programmers?" J. Systems and Software, vol. 80, no. 9, pp. 1460-1471, 2007.
[53] E.A. Nadaraya, "On Estimating Regression," Theory of Probability and Its Applications, vol. 9, no. 1, 141-142, 1964.
[54] N. Nagappan, L. Williams, M. Ferzli, E. Wiebe, K. Yang, C. Miller, and S. Balik, "Improving the CS1 Experience with Pair Programming," Proc. 34th SIGCSE Technical Symp. Computer Science Education, pp. 359-362, 2003.
[55] J. Nawrocki and A. Wojciechowski, "Experimental Evaluation of Pair Programming," Proc. European Software Control and Metrics Conf., 2001.
[56] A. Nickel and T. Barnes, "Games for CS Education: Computer-Supported Collaborative Learning and Multiplayer Games," Proc. Fifth Int'l Conf. Foundations of Digital Games, pp. 274-276, 2010.
[57] J.T. Nosek, "The Case for Collaborative Programming," Comm. ACM, vol. 41, no. 3, 105-108, 1998.
[58] W. Pedrycz, G. Succi, P. Musilek, and X. Bai, "Using Self-Organizing Maps to Analyze Object-Oriented Software Measures," J. Systems and Software, vol. 59, no. 1, 65-82, 2001.
[59] N. Phaphoom, A. Sillitti, and G. Succi, "Pair Programming and Software Defects—An Industrial Case Study," Proc. Int'l Conf. Agile Processes in Software Eng. and Extreme Programming, vol. 77, pp. 208-222, 2011.
[60] M. Phongpaibul and B. Boehm, "An Empirical Comparison between Pair Development and Software Inspection in Thailand," Proc. ACM/IEEE Int'l Symp. Empirical Software Eng., pp. 85-94, 2006.
[61] M. Phongpaibul and B. Boehm, "A Replicate Empirical Comparison between Pair Development and Software Development with Inspection," Proc. First Int'l Symp. Empirical Software Eng. and Measurement, pp. 265-274, 2007.
[62] A.D. Radermacher and G.S. Walia, "Investigating the Effective Implementation of Pair Programming: An Empirical Investigation," Proc. 42nd ACM Technical Symp. Computer Science Education, pp. 655-660, 2011.
[63] N. Salleh, E. Mendes, J. Grundy, and G.St.J. Burch, "The Effects of Neuroticism on Pair Programming: An Empirical Study in the Higher Education Context," Proc. ACM-IEEE Int'l Symp. Empirical Software Eng. and Measurement. 2010.
[64] N. Salleh, E. Mendes, J. Grundy, and G.S.J Burch, "An Empirical Study of the Effects of Conscientiousness in Pair Programming Using the Five-Factor Personality Model," Proc. ACM/IEEE 32nd Int'l Conf. Software Eng., vol. 1, pp. 577-586, 2010.
[65] A. Sillitti, A. Janes, G. Succi, and T. Vernazza, "Collecting, Integrating and Analyzing Software Metrics and Personal Software Process Data," Proc. 29th Conf. EUROMICRO, p. 336, 2003.
[66] J.S. Simonoff, Smoothing Methods in Statistics. Springer, May 1998.
[67] R. Sison, "Investigating Pair Programming in a Software Engineering Course in an Asian Setting," Proc. 15th Asia-Pacific Software Eng. Conf., pp. 325-331, 2008.
[68] R. Sison, "Investigating the Effect of Pair Programming and Software Size on Software Quality and Programmer Productivity," Proc. 16th Asia-Pacific Software Eng. Conf., pp. 187-193, 2009.
[69] D. Stotts, L. Williams, L. Williams, P. Baheti, P. Baheti, D. Jen, D. Jen, A. Jackson, and A. Jackson, "Virtual Teaming: Experiments and Experiences with Distributed Pair Programming," technical report, 2003.
[70] A. Sillitti, A. Janes, G. Succi, and T. Vernazza, "Measures for Mobile Users: An Architecture," J. System Architecture, vol. 50, no. 7, pp. 393-405, 2004.
[71] A. Sillitti, G. Succi, and J. Vlasenko, "Toward a Better Understanding of Tool Usage (NIER Track)," Proc. 33rd Int'l Conf. Software Eng., 2011.
[72] A. Sillitti, G. Succi, and J. Vlasenko, "Understanding the Impact of Pair Programming on Developers Attention: A Case Study on a Large Industrial Experimentation," Proc. 34th Int'l Conf. Software Eng., 2012.
[73] G. Succi, W. Pedrycz, M. Marchesi, and L.A. Williams, "Preliminary Analysis of the Effects of Pair Programming on Job Satisfaction," Proc. Third Int'l Conf. Extreme Programming, pp. 212-215, 2002.
[74] J.W. Tukey, Exploratory Data Analysis. Addison-Wesley, 1977.
[75] T. VanDeGrift, "Coupling Pair Programming and Writing: Learning about Students' Perceptions and Processes," Proc. 35th SIGCSE Technical Symp. Computer Science Education, pp. 2-6, 2004.
[76] J. Vanhanen and C. Lassenius, "Perceived Effects of Pair Programming in an Industrial Context," Proc. 33rd EUROMICRO Conf. Software Eng. and Advanced Applications, pp. 211-218, 2007.
[77] J. Vanhanen and H. Korpi, "Experiences of Using Pair Programming in an Agile Project," Proc. 40th Ann. Hawaii Int'l Conf. System Sciences, p. 274b, 2007.
[78] J. Vanhanen and C. Lassenius, "Effects of Pair Programming at the Development Team Level: An Experiment," Proc. Int'l Symp. Empirical Software Eng., pp. 336-345, 2005.
[79] L.G. Votta, "By the Way, Has Anyone Studied Any Real Programmers, Yet?" Proc. Ninth Int'l Software Process Workshop, pp. 93-95, 1994.
[80] G. Watson, "Smooth Regression Analysis," Sankhya, vol. 26, pp. 359-372, 1969.
[81] L.A. Williams, R.R. Kessler, W. Cunningham, and R. Jeffries, "Strengthening the Case for Pair Programming," IEEE Software, vol. 17, no. 4, pp. 19-25, July 2000.
[82] L.A. Williams, A. Shukla, and A.I. Anton, "An Initial Exploration of the Relationship between Pair Programming and Brooks' Law," Proc. Agile Development Conf., pp. 11-20, 2004.
[83] L.A. Williams and R.R. Kessler, "All I Really Need to Know about Pair Programming I Learned in Kindergarten," Comm. ACM, vol. 43, no. 5, pp. 108-114, 2000.
[84] L.A. Williams and R.R. Kessler, Pair Programming Illuminated. Addison-Wesley Longman Publishing Co., Inc., 2002.
[85] C. Wohlin, P. Runeson, M. Host, M.C. Ohlsson, B. Regnell, and A. Wesslen, Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, 2000.
[86] S. Xu and X. Chen, "Pair Programming in Software Evolution," Proc. Canadian Conf. Electrical and Computer Eng., pp. 1846 -1849, May 2005.
[87] S. Xu and V. Rajlich, "Empirical Validation of Test-Driven Pair Programming in Game Development," Proc. Fifth IEEE/ACIS Int'l Conf. Computer and Information Science and First IEEE/ACIS Int'l Workshop Component-Based Software Eng., Software Architecture and Reuse, pp. 500-505, 2006.
[88] N.Z. Zacharis, "Measuring the Effects of Virtual Pair Programming in an Introductory Programming Java Course," IEEE Trans. Education, vol. 54, no. 1, pp. 168-170, Feb. 2011.
[89] A.W. Bowman and A. Azzalini, Applied Smoothing Techniques for Data Analysis. Oxford Univ. Press, 1997.
[90] P. Berander, "Using Students as Subjects in Requirements Prioritization," Proc. 2004 Int'l Symp. Empirical Software Eng., pp. 167-176, Aug. 2004.
[91] P.I. Good, Resampling Methods: A Practical Guide to Data Analysis. Springer, 2001.
[92] D.J. Sheskin, Parametric and Nonparametric Statistical Procedures. CRC, 2000.
[93] W. Harrison, "N = 1: An Alternative for Software Engineering Research," Proc. 22nd Int'l Conf. Software Eng., Beg, Borrow, or Steal: Using Multidisciplinary Approaches in Empirical Software Eng. Research Workshop, 2000.
[94] M. Höst, B. Regnell, and C. Wohlin, "Using Student as Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment," Empirical Software Eng., vol. 5, no. 3, pp. 201-214, 2000.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool