The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - July/August (2011 vol.37)
pp: 509-525
Emilia Mendes , University of Auckland, Auckland
Norsaremah Salleh , International Islamic University of Malaysia, Kuala Lumpur and University of Auckland, Auckland
ABSTRACT
The objective of this paper is to present the current evidence relative to the effectiveness of pair programming (PP) as a pedagogical tool in higher education CS/SE courses. We performed a systematic literature review (SLR) of empirical studies that investigated factors affecting the effectiveness of PP for CS/SE students and studies that measured the effectiveness of PP for CS/SE students. Seventy-four papers were used in our synthesis of evidence, and 14 compatibility factors that can potentially affect PP's effectiveness as a pedagogical tool were identified. Results showed that students' skill level was the factor that affected PP's effectiveness the most. The most common measure used to gauge PP's effectiveness was time spent on programming. In addition, students' satisfaction when using PP was overall higher than when working solo. Our meta-analyses showed that PP was effective in improving students' grades on assignments. Finally, in the studies that used quality as a measure of effectiveness, the number of test cases succeeded, academic performance, and expert opinion were the quality measures mostly applied. The results of this SLR show two clear gaps in this research field: 1) a lack of studies focusing on pair compatibility factors aimed at making PP an effective pedagogical tool and 2) a lack of studies investigating PP for software design/modeling tasks in conjunction with programming tasks.
INDEX TERMS
Empirical studies, pair programming, systematic review.
CITATION
Emilia Mendes, Norsaremah Salleh, "Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review", IEEE Transactions on Software Engineering, vol.37, no. 4, pp. 509-525, July/August 2011, doi:10.1109/TSE.2010.59
REFERENCES
[1] H. Al-Kilidar, P. Parkin, A. Aurum, and R. Jeffery, "Evaluation of Effects of Pair Work on Quality of Designs," Proc. 2005 Australian Software Eng. Conf., pp. 78-87, 2005.
[2] V. Balijepally, "Task Complexity and Effectiveness of Pair Programming: An Experimental Study," PhD dissertation, The Univ. of Texas at Arlington, 2006.
[3] E. Bellini, G. Canfora, F. Garcia, M. Piattini, and C.A. Visaggio, "Pair Designing as Practice for Enforcing and Diffusing Design Knowledge," J. Software Maintenance and Evolution-Research and Practice, vol. 17, pp. 401-423, 2005.
[4] S.B. Berenson, K.M. Slaten, L. Williams, and C.-W. Ho, "Voices of Women in a Software Engineering Course: Reflections on Collaboration," J. Educational Resources in Computing, vol. 4, no. 1, 2004.
[5] T. Bipp, A. Lepper, and D. Schmedding, "Pair Programming in Software Development Teams—An Empirical Study of Its Benefits," Information and Software Technology, vol. 50, pp. 231-240, 2008.
[6] G. Canfora, A. Cimitile, and C.A. Visaggio, "Working in Pairs as a Means for Design Knowledge Building: An Empirical Study," Proc. 12th IEEE Int'l Workshop Program Comprehension, pp. 62-68, 2004.
[7] G. Canfora, A. Cimitile, and C.A. Visaggio, "Empirical Study on the Productivity of the Pair Programming," Proc. Sixth Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 92-99, 2005.
[8] G. Canfora, A. Cimitile, F. Garcia, M. Piattini, and C.A. Visaggio, "Confirming the Influence of Educational Background in Pair-Design Knowledge through Experiments," Proc. ACM Symp. Applied Computing, pp. 1478-1484, 2005.
[9] G. Canfora, A. Cimitile, F. Garcia, M. Piattini, and C.A. Visaggio, "Performances of Pair Designing on Software Evolution: A Controlled Experiment," Proc. 10th European Conf. Software Maintenance and Reeng., pp. 195-202, 2006.
[10] H. Srikanth, L. Williams, E. Wiebe, C. Miller, and S. Balik, "On Pair Rotation in the Computer Science Course," Proc. 17th Conf. Software Eng. Education and Training, pp. 144-149, 2004.
[11] C. Lan and B. Ramesh, "An Exploratory Study on the Effects of Pair Programming," Proc. Eighth Int'l Conf. Empirical Assessment in Software Eng. Workshop—26th Int'l Conf. Software Eng., pp. 21-28, 2004.
[12] J.C. Carver, L. Henderson, L. He, J. Hodges, and D. Reese, "Increased Retention of Early Computer Science and Software Engineering Students Using Pair Programming," Proc. 20th Conf. Software Eng. Education and Training, pp. 115-122, 2007.
[13] J. Chao and G. Atli, "Critical Personality Traits in Successful Pair Programming," Proc. AGILE '06, pp. 89-93, 2006.
[14] E.A. Chaparro, A. Yuksel, P. Romero, and S. Bryant, "Factors Affecting the Perceived Effectiveness of Pair Programming in Higher Education," Proc. 17th Workshop Psychology of Programming Interest Group, pp. 5-18, 2005.
[15] D.C. Cliburn, "Experiences with Pair Programming at a Small College," J. Computing Sciences in Colleges, vol. 19, pp. 20-29, 2003.
[16] T.H. DeClue, "Pair Programming and Pair Trading: Effects on Learning and Motivation in a CS2 Courses," J. Computing Sciences in Colleges, vol. 18, no. 5, pp. 49-56, 2003.
[17] M.A. Domino, R.W. Collins, A.R. Hevner, and C.F. Cohen, "Conflict in Collaborative Software Development," Proc. 2003 Conf. Computer Personnel Research: Freedom in Philadelphia-Leveraging Differences and Diversity in the IT Workforce, pp. 44-51, 2003.
[18] M.A. Domino, R.W. Collins, and A.R. Hevner, "Controlled Experimentation on Adaptations of Pair Programming," Information Technology and Management, vol. 8, pp. 297-312, 2007.
[19] S.F. Freeman, B.K. Jaeger, and J.C. Brougham, "Pair Programming: More Learning and Less Anxiety in a First Programming Course," Proc. ASEE Ann. Conf., pp. 8885-8893, 2003.
[20] E.F. Gehringer, "A Pair-Programming Experiment in a Non-Programming Courses," Proc. 18th Ann. ACM SIGPLAN Conf. Object Oriented Programming Systems, Languages and Applications, pp. 187-190, 2003.
[21] B. Hanks, C. McDowell, D. Draper, and M. Krnjajic, "Program Quality with Pair Programming in CS1," ACM SIGCSE Bull., vol. 36, pp. 176-180, 2004.
[22] B. Hanks, "Student Attitudes toward Pair Programming," Proc. 11th Ann. Proc. SIGCSE Conf. Innovation and Technology in Computer Science Education, pp. 113-117, 2006.
[23] S. Heiberg, U. Puus, P. Salumaa, and A. Seeba, "Pair-Programming Effect on Developers Productivity," Proc. Fourth Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 215-224, 2003.
[24] C.-W. Ho, "Examining Impact of Pair Programming on Female Students," Technical Report TR-2004-20, Dept. of Computer Science, North Carolina State Univ., 2004.
[25] M. Ciolkowski and M. Schlemmer, "Experiencing with a Case Study on Pair Programming," Proc. First Int'l Workshop Empirical Studies in Software Eng., 2002.
[26] S.D. James and J.C. Hansen, "Student-Based Pair Programming: An Examination," Proc. Sixth World Multiconf. Systemics, vol. 8, pp. 485-489, 2002.
[27] A. Janes, B. Russo, P. Zuliani, and G. Succi, "An Empirical Analysis on the Discontinuous Use of Pair Programming," Proc. Fourth Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 205-214, 2003.
[28] N. Katira, L. Williams, E. Wiebe, C. Miller, S. Balik, and E. Gehringer, "On Understanding Compatibility of Student Pair Programmers," ACM SIGCSE Bull., vol. 36, pp. 7-11, 2004.
[29] N. Katira, L. Williams, and J. Osborne, "Towards Increasing the Compatibility of Student Pair Programmers," Proc. 27th Int'l Conf. Software Eng., pp. 625-626, 2005.
[30] S. Kuppuswami and K. Vivekanandan, "The Effects of Pair Programming on Learning Efficiency in Short Programming Assignments," Informatics in Education, vol. 3, pp. 251-266, 2004.
[31] L. Layman, L. Williams, J. Osborne, S. Berenson, K. Slaten, and M. Vouk, "How and Why Collaborative Software Development Impacts the Software Engineering Course," Proc. 35th Ann. Conf. Frontiers in Education, pp. T4C-9-T4C-14, 2005.
[32] L. Layman, "Changing Students' Perceptions: An Analysis of the Supplementary Benefits of Collaborative Software Development," Proc. 19th IEEE Conf. Software Eng. Education and Training, pp. 159-166, 2006.
[33] K.M. Lui and K.C.C. Chan, "Pair Programming Productivity: Novice-Novice versus Expert-Expert," Int'l J. Human-Computer Studies, vol. 64, pp. 915-925, 2006.
[34] L. Madeyski, "Preliminary Analysis of the Effects of Pair Programming and Test-Driven Development on the External Code Quality," Software Eng. Evolution and Emerging Technologies, K. Zieliński and T. Szmuc, eds., pp. 113-123, IOS Press, 2005.
[35] L. Madeyski, "The Impact of Pair Programming and Test-Driven Development on Package Dependencies in Object-oriented Design—An Experiment," Proc. Seventh Int'l Conf. Product-Focused Software Process Improvement, pp. 278-289, 2006.
[36] L. Madeyski, "On the Effects of Pair Programming on Thoroughness and Fault-Finding Effectiveness of Unit Tests," Proc. Int'l Conf. Product-Focused Software Process Improvement, pp. 207-221, 2007.
[37] C. McDowell, L. Werner, H. Bullock, and J. Fernald, "The Effects of Pair-Programming on Performance in an Introductory Programming Course," ACM SIGCSE Bull., vol. 34, pp. 38-42, 2002.
[38] C. McDowell, B. Hanks, and L. Werner, "Experimenting with Pair Programming in the Classroom," ACM SIGCSE Bull., vol. 35, pp. 60-64, 2003.
[39] C. McDowell, L. Werner, H.E. Bullock, and J. Fernald, "The Impact of Pair Programming on Student Performance, Perception and Persistence," Proc. 25th Int'l Conf. Software Eng., pp. 602-607, 2003.
[40] E. Mendes, L. Al-Fakhri, and A. Luxton-Reilly, "Investigating Pair-Programming in a 2nd-Year Software Development and Design Computer Science Course," ACM SIGCSE Bull., vol. 37, no. 3, pp. 296-300, 2005.
[41] E. Mendes, L. Al-Fakhri, and A. Luxton-Reilly, "A Replicated Experiment of Pair-Programming in a 2nd-Year Software Development and Design Computer Science Course," Proc. 11th Ann. SIGCSE Conf. Innovation and Technology in Computer Science Education, pp. 108-112, 2006.
[42] M.M. Muller and F. Padberg, "An Empirical Study about the Feelgood Factor in Pair Programming," Proc. 10th Int'l Symp. Software Metrics, pp. 151-158, 2004.
[43] M.M. Muller, "Are Reviews an Alternative to Pair Programming?" Empirical Software Eng., vol. 9, pp. 335-351, 2004.
[44] M.M. Muller, "Two Controlled Experiments Concerning the Comparison of Pair Programming to Peer Review," J. Systems and Software, vol. 78, pp. 166-179, 2005.
[45] M.M. Muller, "Do Programmer Pairs Make Different Mistakes than Solo Programmers?" J. Systems and Software, vol. 80, no. 9, pp. 1460-1471, 2006.
[46] M.M. Muller, "A Preliminary Study on the Impact of a Pair Design Phase on Pair Programming and Solo Programming," Information and Software Technology, vol. 48, no. 5, pp. 335-344, 2006.
[47] J. Nawrocki and A. Wojciechowski, "Experimental Evaluation of Pair Programming," Proc. European Software Control and Metrics Conf., pp. 269-276, 2001.
[48] T.C. Ahren, "Work in Progress—Effect of Instructional Design and Pair Programming on Student Performance in an Introductory Programming Course," Proc. 35th Ann. Conf. Frontiers in Education, pp. F3E-11-12, 2005.
[49] M. Phongpaibul and B. Boehm, "An Empirical Comparison between Pair Development and Software Inspection in Thailand," Proc. 2006 Fifth ACM-IEEE Int'l Symp. Empirical Software Eng., pp. 85-94, 2006.
[50] P. Sfetsos, I. Stamelos, L. Angelis, and I. Deligiannis, "Investigating the Impact of Personality Types on Communication and Collaboration-Viability in Pair Programming—An Empirical Study," Proc. Seventh Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 43-52, 2006.
[51] K.M. Slaten, M. Droujkova, S.B. Berenson, L. Williams, and L. Layman, "Undergraduate Student Perceptions of Pair Programming and Agile Software Methodologies: Verifying a Model of Social Interaction," Proc. Agile '05, pp. 323-330, 2005.
[52] X. Shaochun and V. Rajlich, "Pair Programming in Graduate Software Engineering Course Projects," Proc. 35th Ann. Conf. Frontiers in Education, pp. F1G-7-F1G-12, 2005.
[53] X. Shaochun and V. Rajlich, "Empirical Validation of Test-Driven Pair Programming in Game Development," Proc. Fifth IEEE/ACIS Int'l Conf. Computational and Information Science—in Conjunction with First IEEE/ACIS Workshop Component-Based Software Engineer Architecture and Reuse, pp. 500-505, 2006.
[54] L. Thomas, M. Ratcliffe, and A. Robertson, "Code Warriors and Code-a-Phobes: A Study in Attitude and Pair Programming," ACM SIGCSE Bull., vol. 35, pp. 363-367, 2003.
[55] J.E. Tomayko, "A Comparison of Pair Programming to Inspections for Software Defect Reduction," Computer Science Education, vol. 12, no. 3, pp. 213-222, 2002.
[56] T. VanDeGrift, "Coupling Pair Programming and Writing: Learning about Students' Perceptions and Processes," ACM SIGCSE Bull., pp. 2-6, 2004.
[57] J. Vanhanen and C. Lassenius, "Effects of Pair Programming at the Development Team Level: An Experiment," Proc. Int'l Symp. Software Eng., pp. 336-345, 2005.
[58] T. Van Toll Iii, R. Lee, and T. Ahlswede, "Evaluating the Usefulness of Pair Programming in a Classroom Setting," Proc. Sixth IEEE/ACIS Int'l Conf. Computational and Information Science, pp. 302-308, 2007.
[59] L.A. Williams and R.R. Kessler, "The Effects of 'Pair-Pressure' and 'Pair-Learning' on Software Engineering Education," Proc. 13th Conf. Software Eng. Education and Training , pp. 59-65, 2000.
[60] L. Williams, R.R. Kessler, W. Cunningham, and R. Jeffries, "Strengthening the Case for Pair Programming," IEEE Software, vol. 17, no. 4, pp. 19-25, July/Aug. 2000.
[61] L. Williams, E. Wiebe, K. Yang, M. Ferzli, and C. Miller, "In Support of Pair Programming in the Introductory Computer Science Course," Computer Science Education, vol. 12, pp. 197-212, 2002.
[62] L. Williams, C. McDowell, N. Nagappan, J. Fernald, and L. Werner, "Building Pair Programming Knowledge through a Family of Experiments," Proc. 2003 Int'l Symp. Empirical Software Eng., pp. 143-152, 2003.
[63] L. Williams, L. Layman, J. Osborne, and N. Katira, "Examining the Compatibility of Student Pair Programmers," Proc. AGILE 2006 Conf., 2006.
[64] D. Winkler and S. Biffl, "An Empirical Study on Design Quality Improvement from Best-Practice Inspection and Pair Programming," Product-Focused Software Process Improvement, pp. 319-333, Springer, 2006.
[65] J.R. Nawrocki, M. Jasinski, L. Olek, and B. Lange, "Pair Programming versus Side-by-Side Programming," Proc. 12th European Conf. Software Process Improvement, pp. 28-38, 2005.
[66] E.V. Howard, "Attitudes on Using Pair-Programming," J. Educational Technology Systems, vol. 35, no. 1, pp. 89-103, 2006.
[67] M. Mujeeb-u-Rehman, Y. Xiaohu, D. Jinxiang, and M.A. Ghafoor, "Heterogeneous and Homogenous Pairs in Pair Programming: An Empirical Analysis," Proc. Canadian Conf. Electrical and Computer Eng., pp. 1116-1119, 2006.
[68] L. Madeyski, "Is External Code Quality Correlated with Programming Experience or Feelgood Factor?" Proc. Seventh Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 65-74, 2006.
[69] B. Simon and B. Hanks, "First Year Students' Impressions of Pair Programming in CS1," Proc. Third Int'l Computing Education Research Workshop, pp. 73-86, 2007.
[70] L. Williams, L. Layman, K.M. Slaten, S.B. Berenson, and C. Seaman, "On the Impact of a Collaborative Pedagogy on African American Millennial Students in Software Engineering," Proc. Int'l Conf. Software Eng., pp. 677-686, 2007.
[71] B. Hanks, "Problems Encountered by Novice Pair Programmers," Proc. Third Int'l Computing Education Research Workshop, pp. 159-164, 2007.
[72] A.T. Williams, "Pair Formation in CS1: Self-Selection vs Random Pairing," PhD dissertation, Pace Univ., 2007.
[73] K.S. Choi, "A Discovery and Analysis of Influencing Factors of Pair Programming," PhD dissertation, Dept. of Information Systems, New Jersey Inst. of Tech nology, 2004.
[74] H. Gevaert, "Pair Programming Unearthed," MSc thesis, Univ. of Manitoba, 2007.
25 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool