This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review
July/August 2011 (vol. 37 no. 4)
pp. 509-525
ASCII Text
x 
 
Norsaremah Salleh, Emilia Mendes, John C. Grundy, "Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review," IEEE Transactions on Software Engineering, vol. 37, no. 4, pp. 509-525, July/August, 2011.
 
 
BibTex
x
 
@article{ 10.1109/TSE.2010.59,
author = {Norsaremah Salleh and Emilia Mendes and John C. Grundy},
title = {Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review},
journal ={IEEE Transactions on Software Engineering},
volume = {37},
number = {4},
issn = {0098-5589},
year = {2011},
pages = {509-525},
doi = {http://doi.ieeecomputersociety.org/10.1109/TSE.2010.59},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Software Engineering
TI - Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review
IS - 4
SN - 0098-5589
SP509
EP525
EPD - 509-525
A1 - Norsaremah Salleh,
A1 - Emilia Mendes,
A1 - John C. Grundy,
PY - 2011
KW - Empirical studies
KW - pair programming
KW - systematic review.
VL - 37
JA - IEEE Transactions on Software Engineering
ER -
 
Norsaremah Salleh, International Islamic University of Malaysia, Kuala Lumpur and University of Auckland, Auckland
Emilia Mendes, University of Auckland, Auckland
John C. Grundy, Swinburne University of Technology, Hawthorn
The objective of this paper is to present the current evidence relative to the effectiveness of pair programming (PP) as a pedagogical tool in higher education CS/SE courses. We performed a systematic literature review (SLR) of empirical studies that investigated factors affecting the effectiveness of PP for CS/SE students and studies that measured the effectiveness of PP for CS/SE students. Seventy-four papers were used in our synthesis of evidence, and 14 compatibility factors that can potentially affect PP's effectiveness as a pedagogical tool were identified. Results showed that students' skill level was the factor that affected PP's effectiveness the most. The most common measure used to gauge PP's effectiveness was time spent on programming. In addition, students' satisfaction when using PP was overall higher than when working solo. Our meta-analyses showed that PP was effective in improving students' grades on assignments. Finally, in the studies that used quality as a measure of effectiveness, the number of test cases succeeded, academic performance, and expert opinion were the quality measures mostly applied. The results of this SLR show two clear gaps in this research field: 1) a lack of studies focusing on pair compatibility factors aimed at making PP an effective pedagogical tool and 2) a lack of studies investigating PP for software design/modeling tasks in conjunction with programming tasks.

[1] H. Al-Kilidar, P. Parkin, A. Aurum, and R. Jeffery, "Evaluation of Effects of Pair Work on Quality of Designs," Proc. 2005 Australian Software Eng. Conf., pp. 78-87, 2005.
[2] V. Balijepally, "Task Complexity and Effectiveness of Pair Programming: An Experimental Study," PhD dissertation, The Univ. of Texas at Arlington, 2006.
[3] E. Bellini, G. Canfora, F. Garcia, M. Piattini, and C.A. Visaggio, "Pair Designing as Practice for Enforcing and Diffusing Design Knowledge," J. Software Maintenance and Evolution-Research and Practice, vol. 17, pp. 401-423, 2005.
[4] S.B. Berenson, K.M. Slaten, L. Williams, and C.-W. Ho, "Voices of Women in a Software Engineering Course: Reflections on Collaboration," J. Educational Resources in Computing, vol. 4, no. 1, 2004.
[5] T. Bipp, A. Lepper, and D. Schmedding, "Pair Programming in Software Development Teams—An Empirical Study of Its Benefits," Information and Software Technology, vol. 50, pp. 231-240, 2008.
[6] G. Canfora, A. Cimitile, and C.A. Visaggio, "Working in Pairs as a Means for Design Knowledge Building: An Empirical Study," Proc. 12th IEEE Int'l Workshop Program Comprehension, pp. 62-68, 2004.
[7] G. Canfora, A. Cimitile, and C.A. Visaggio, "Empirical Study on the Productivity of the Pair Programming," Proc. Sixth Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 92-99, 2005.
[8] G. Canfora, A. Cimitile, F. Garcia, M. Piattini, and C.A. Visaggio, "Confirming the Influence of Educational Background in Pair-Design Knowledge through Experiments," Proc. ACM Symp. Applied Computing, pp. 1478-1484, 2005.
[9] G. Canfora, A. Cimitile, F. Garcia, M. Piattini, and C.A. Visaggio, "Performances of Pair Designing on Software Evolution: A Controlled Experiment," Proc. 10th European Conf. Software Maintenance and Reeng., pp. 195-202, 2006.
[10] H. Srikanth, L. Williams, E. Wiebe, C. Miller, and S. Balik, "On Pair Rotation in the Computer Science Course," Proc. 17th Conf. Software Eng. Education and Training, pp. 144-149, 2004.
[11] C. Lan and B. Ramesh, "An Exploratory Study on the Effects of Pair Programming," Proc. Eighth Int'l Conf. Empirical Assessment in Software Eng. Workshop—26th Int'l Conf. Software Eng., pp. 21-28, 2004.
[12] J.C. Carver, L. Henderson, L. He, J. Hodges, and D. Reese, "Increased Retention of Early Computer Science and Software Engineering Students Using Pair Programming," Proc. 20th Conf. Software Eng. Education and Training, pp. 115-122, 2007.
[13] J. Chao and G. Atli, "Critical Personality Traits in Successful Pair Programming," Proc. AGILE '06, pp. 89-93, 2006.
[14] E.A. Chaparro, A. Yuksel, P. Romero, and S. Bryant, "Factors Affecting the Perceived Effectiveness of Pair Programming in Higher Education," Proc. 17th Workshop Psychology of Programming Interest Group, pp. 5-18, 2005.
[15] D.C. Cliburn, "Experiences with Pair Programming at a Small College," J. Computing Sciences in Colleges, vol. 19, pp. 20-29, 2003.
[16] T.H. DeClue, "Pair Programming and Pair Trading: Effects on Learning and Motivation in a CS2 Courses," J. Computing Sciences in Colleges, vol. 18, no. 5, pp. 49-56, 2003.
[17] M.A. Domino, R.W. Collins, A.R. Hevner, and C.F. Cohen, "Conflict in Collaborative Software Development," Proc. 2003 Conf. Computer Personnel Research: Freedom in Philadelphia-Leveraging Differences and Diversity in the IT Workforce, pp. 44-51, 2003.
[18] M.A. Domino, R.W. Collins, and A.R. Hevner, "Controlled Experimentation on Adaptations of Pair Programming," Information Technology and Management, vol. 8, pp. 297-312, 2007.
[19] S.F. Freeman, B.K. Jaeger, and J.C. Brougham, "Pair Programming: More Learning and Less Anxiety in a First Programming Course," Proc. ASEE Ann. Conf., pp. 8885-8893, 2003.
[20] E.F. Gehringer, "A Pair-Programming Experiment in a Non-Programming Courses," Proc. 18th Ann. ACM SIGPLAN Conf. Object Oriented Programming Systems, Languages and Applications, pp. 187-190, 2003.
[21] B. Hanks, C. McDowell, D. Draper, and M. Krnjajic, "Program Quality with Pair Programming in CS1," ACM SIGCSE Bull., vol. 36, pp. 176-180, 2004.
[22] B. Hanks, "Student Attitudes toward Pair Programming," Proc. 11th Ann. Proc. SIGCSE Conf. Innovation and Technology in Computer Science Education, pp. 113-117, 2006.
[23] S. Heiberg, U. Puus, P. Salumaa, and A. Seeba, "Pair-Programming Effect on Developers Productivity," Proc. Fourth Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 215-224, 2003.
[24] C.-W. Ho, "Examining Impact of Pair Programming on Female Students," Technical Report TR-2004-20, Dept. of Computer Science, North Carolina State Univ., 2004.
[25] M. Ciolkowski and M. Schlemmer, "Experiencing with a Case Study on Pair Programming," Proc. First Int'l Workshop Empirical Studies in Software Eng., 2002.
[26] S.D. James and J.C. Hansen, "Student-Based Pair Programming: An Examination," Proc. Sixth World Multiconf. Systemics, vol. 8, pp. 485-489, 2002.
[27] A. Janes, B. Russo, P. Zuliani, and G. Succi, "An Empirical Analysis on the Discontinuous Use of Pair Programming," Proc. Fourth Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 205-214, 2003.
[28] N. Katira, L. Williams, E. Wiebe, C. Miller, S. Balik, and E. Gehringer, "On Understanding Compatibility of Student Pair Programmers," ACM SIGCSE Bull., vol. 36, pp. 7-11, 2004.
[29] N. Katira, L. Williams, and J. Osborne, "Towards Increasing the Compatibility of Student Pair Programmers," Proc. 27th Int'l Conf. Software Eng., pp. 625-626, 2005.
[30] S. Kuppuswami and K. Vivekanandan, "The Effects of Pair Programming on Learning Efficiency in Short Programming Assignments," Informatics in Education, vol. 3, pp. 251-266, 2004.
[31] L. Layman, L. Williams, J. Osborne, S. Berenson, K. Slaten, and M. Vouk, "How and Why Collaborative Software Development Impacts the Software Engineering Course," Proc. 35th Ann. Conf. Frontiers in Education, pp. T4C-9-T4C-14, 2005.
[32] L. Layman, "Changing Students' Perceptions: An Analysis of the Supplementary Benefits of Collaborative Software Development," Proc. 19th IEEE Conf. Software Eng. Education and Training, pp. 159-166, 2006.
[33] K.M. Lui and K.C.C. Chan, "Pair Programming Productivity: Novice-Novice versus Expert-Expert," Int'l J. Human-Computer Studies, vol. 64, pp. 915-925, 2006.
[34] L. Madeyski, "Preliminary Analysis of the Effects of Pair Programming and Test-Driven Development on the External Code Quality," Software Eng. Evolution and Emerging Technologies, K. Zieliński and T. Szmuc, eds., pp. 113-123, IOS Press, 2005.
[35] L. Madeyski, "The Impact of Pair Programming and Test-Driven Development on Package Dependencies in Object-oriented Design—An Experiment," Proc. Seventh Int'l Conf. Product-Focused Software Process Improvement, pp. 278-289, 2006.
[36] L. Madeyski, "On the Effects of Pair Programming on Thoroughness and Fault-Finding Effectiveness of Unit Tests," Proc. Int'l Conf. Product-Focused Software Process Improvement, pp. 207-221, 2007.
[37] C. McDowell, L. Werner, H. Bullock, and J. Fernald, "The Effects of Pair-Programming on Performance in an Introductory Programming Course," ACM SIGCSE Bull., vol. 34, pp. 38-42, 2002.
[38] C. McDowell, B. Hanks, and L. Werner, "Experimenting with Pair Programming in the Classroom," ACM SIGCSE Bull., vol. 35, pp. 60-64, 2003.
[39] C. McDowell, L. Werner, H.E. Bullock, and J. Fernald, "The Impact of Pair Programming on Student Performance, Perception and Persistence," Proc. 25th Int'l Conf. Software Eng., pp. 602-607, 2003.
[40] E. Mendes, L. Al-Fakhri, and A. Luxton-Reilly, "Investigating Pair-Programming in a 2nd-Year Software Development and Design Computer Science Course," ACM SIGCSE Bull., vol. 37, no. 3, pp. 296-300, 2005.
[41] E. Mendes, L. Al-Fakhri, and A. Luxton-Reilly, "A Replicated Experiment of Pair-Programming in a 2nd-Year Software Development and Design Computer Science Course," Proc. 11th Ann. SIGCSE Conf. Innovation and Technology in Computer Science Education, pp. 108-112, 2006.
[42] M.M. Muller and F. Padberg, "An Empirical Study about the Feelgood Factor in Pair Programming," Proc. 10th Int'l Symp. Software Metrics, pp. 151-158, 2004.
[43] M.M. Muller, "Are Reviews an Alternative to Pair Programming?" Empirical Software Eng., vol. 9, pp. 335-351, 2004.
[44] M.M. Muller, "Two Controlled Experiments Concerning the Comparison of Pair Programming to Peer Review," J. Systems and Software, vol. 78, pp. 166-179, 2005.
[45] M.M. Muller, "Do Programmer Pairs Make Different Mistakes than Solo Programmers?" J. Systems and Software, vol. 80, no. 9, pp. 1460-1471, 2006.
[46] M.M. Muller, "A Preliminary Study on the Impact of a Pair Design Phase on Pair Programming and Solo Programming," Information and Software Technology, vol. 48, no. 5, pp. 335-344, 2006.
[47] J. Nawrocki and A. Wojciechowski, "Experimental Evaluation of Pair Programming," Proc. European Software Control and Metrics Conf., pp. 269-276, 2001.
[48] T.C. Ahren, "Work in Progress—Effect of Instructional Design and Pair Programming on Student Performance in an Introductory Programming Course," Proc. 35th Ann. Conf. Frontiers in Education, pp. F3E-11-12, 2005.
[49] M. Phongpaibul and B. Boehm, "An Empirical Comparison between Pair Development and Software Inspection in Thailand," Proc. 2006 Fifth ACM-IEEE Int'l Symp. Empirical Software Eng., pp. 85-94, 2006.
[50] P. Sfetsos, I. Stamelos, L. Angelis, and I. Deligiannis, "Investigating the Impact of Personality Types on Communication and Collaboration-Viability in Pair Programming—An Empirical Study," Proc. Seventh Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 43-52, 2006.
[51] K.M. Slaten, M. Droujkova, S.B. Berenson, L. Williams, and L. Layman, "Undergraduate Student Perceptions of Pair Programming and Agile Software Methodologies: Verifying a Model of Social Interaction," Proc. Agile '05, pp. 323-330, 2005.
[52] X. Shaochun and V. Rajlich, "Pair Programming in Graduate Software Engineering Course Projects," Proc. 35th Ann. Conf. Frontiers in Education, pp. F1G-7-F1G-12, 2005.
[53] X. Shaochun and V. Rajlich, "Empirical Validation of Test-Driven Pair Programming in Game Development," Proc. Fifth IEEE/ACIS Int'l Conf. Computational and Information Science—in Conjunction with First IEEE/ACIS Workshop Component-Based Software Engineer Architecture and Reuse, pp. 500-505, 2006.
[54] L. Thomas, M. Ratcliffe, and A. Robertson, "Code Warriors and Code-a-Phobes: A Study in Attitude and Pair Programming," ACM SIGCSE Bull., vol. 35, pp. 363-367, 2003.
[55] J.E. Tomayko, "A Comparison of Pair Programming to Inspections for Software Defect Reduction," Computer Science Education, vol. 12, no. 3, pp. 213-222, 2002.
[56] T. VanDeGrift, "Coupling Pair Programming and Writing: Learning about Students' Perceptions and Processes," ACM SIGCSE Bull., pp. 2-6, 2004.
[57] J. Vanhanen and C. Lassenius, "Effects of Pair Programming at the Development Team Level: An Experiment," Proc. Int'l Symp. Software Eng., pp. 336-345, 2005.
[58] T. Van Toll Iii, R. Lee, and T. Ahlswede, "Evaluating the Usefulness of Pair Programming in a Classroom Setting," Proc. Sixth IEEE/ACIS Int'l Conf. Computational and Information Science, pp. 302-308, 2007.
[59] L.A. Williams and R.R. Kessler, "The Effects of 'Pair-Pressure' and 'Pair-Learning' on Software Engineering Education," Proc. 13th Conf. Software Eng. Education and Training , pp. 59-65, 2000.
[60] L. Williams, R.R. Kessler, W. Cunningham, and R. Jeffries, "Strengthening the Case for Pair Programming," IEEE Software, vol. 17, no. 4, pp. 19-25, July/Aug. 2000.
[61] L. Williams, E. Wiebe, K. Yang, M. Ferzli, and C. Miller, "In Support of Pair Programming in the Introductory Computer Science Course," Computer Science Education, vol. 12, pp. 197-212, 2002.
[62] L. Williams, C. McDowell, N. Nagappan, J. Fernald, and L. Werner, "Building Pair Programming Knowledge through a Family of Experiments," Proc. 2003 Int'l Symp. Empirical Software Eng., pp. 143-152, 2003.
[63] L. Williams, L. Layman, J. Osborne, and N. Katira, "Examining the Compatibility of Student Pair Programmers," Proc. AGILE 2006 Conf., 2006.
[64] D. Winkler and S. Biffl, "An Empirical Study on Design Quality Improvement from Best-Practice Inspection and Pair Programming," Product-Focused Software Process Improvement, pp. 319-333, Springer, 2006.
[65] J.R. Nawrocki, M. Jasinski, L. Olek, and B. Lange, "Pair Programming versus Side-by-Side Programming," Proc. 12th European Conf. Software Process Improvement, pp. 28-38, 2005.
[66] E.V. Howard, "Attitudes on Using Pair-Programming," J. Educational Technology Systems, vol. 35, no. 1, pp. 89-103, 2006.
[67] M. Mujeeb-u-Rehman, Y. Xiaohu, D. Jinxiang, and M.A. Ghafoor, "Heterogeneous and Homogenous Pairs in Pair Programming: An Empirical Analysis," Proc. Canadian Conf. Electrical and Computer Eng., pp. 1116-1119, 2006.
[68] L. Madeyski, "Is External Code Quality Correlated with Programming Experience or Feelgood Factor?" Proc. Seventh Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 65-74, 2006.
[69] B. Simon and B. Hanks, "First Year Students' Impressions of Pair Programming in CS1," Proc. Third Int'l Computing Education Research Workshop, pp. 73-86, 2007.
[70] L. Williams, L. Layman, K.M. Slaten, S.B. Berenson, and C. Seaman, "On the Impact of a Collaborative Pedagogy on African American Millennial Students in Software Engineering," Proc. Int'l Conf. Software Eng., pp. 677-686, 2007.
[71] B. Hanks, "Problems Encountered by Novice Pair Programmers," Proc. Third Int'l Computing Education Research Workshop, pp. 159-164, 2007.
[72] A.T. Williams, "Pair Formation in CS1: Self-Selection vs Random Pairing," PhD dissertation, Pace Univ., 2007.
[73] K.S. Choi, "A Discovery and Analysis of Influencing Factors of Pair Programming," PhD dissertation, Dept. of Information Systems, New Jersey Inst. of Tech nology, 2004.
[74] H. Gevaert, "Pair Programming Unearthed," MSc thesis, Univ. of Manitoba, 2007.
[1] M. Ally, F. Darroch, and M. Toleman, "A Framework for Understanding the Factors Influencing Pair Programming Success," Extreme Programming and Agile Processes in Software Engineering, pp. 82-91, Springer, 2005.
[2] L. Bax, L.-M. Yu, N. Ikeda, H. Tsuruta, and K.G. Moons, "Development and Validation of MIX: Comprehensive Free Software for Meta-Analysis of Causal Research Data," BMC Medical Research Methodology, vol. 6, 2006.
[3] L. Bax, L.-M. Yu, N. Ikeda, H. Tsuruta, and K.G. Moons, "Comprehensive Free Software for Meta-Analysis of Causal Research Data.Version 1.7," BMC Medical Research Methodology, vol. 6, no. 50,http://www.biomedcentral.com/1471-2288/6 50, Oct. 2006.
[4] K. Beck, Extreme Programming Explained: Embrace Change. Addison-Wesley, 2000.
[5] S.B. Berenson, K.M. Slaten, L. Williams, and C.-W. Ho, "Voices of Women in a Software Engineering Course: Reflections on collaboration," J. Educational Resources in Computing, vol. 4, no.1, 2004.
[6] C.A. Bowers, J.A. Pharmer, and E. Salas, "When Member Homogeneity Is Needed in Work Teams: A Meta-Analysis," Small Group Research, vol. 31, pp. 305-327, 2000.
[7] G. Burch and N. Anderson, "Personality as a Predictor of Work-Related Behavior and Performance: Recent Advances and Directions for Future Research," Int'l Rev. Industrial and Organizational Psychology, G.P. Hodgkinson and J.K. Ford, eds., pp. 261-305, John Wiley & Sons, Ltd., 2008.
[8] V.V. Busato, F.J. Prins, J.J. Elshout, and C. Hamaker, "Intellectual Ability, Learning Style, Personality, Achievement Motivation and Academic Success of Psychology Students in Higher Education," Personality and Individual Differences, vol. 29, no. 6, pp. 1057-1068, 2000.
[9] E.A. Chaparro, A. Yuksel, P. Romero, and S. Bryant, "Factors Affecting the Perceived Effectiveness of Pair Programming in Higher Education," Proc. 17th Workshop Psychology of Programming Interest Group, pp. 5-18, 2005.
[10] S.R. Chidamber and C.F. Kemerer, "A Metrics Suite for Object-Oriented Design," IEEE Trans. Software Eng., vol. 20, no. 6, pp. 476-493, June 1994.
[11] D.C. Cliburn, "Experiences with Pair Programming at a Small College," J. Computing Sciences in Colleges, vol. 19, no. 1, pp. 20-29, 2003.
[12] A. Cockburn and L. Williams, "The Costs and Benefits of Pair Programming," Proc. Second Int'l Conf. Extreme Programming and Flexible Processes in Software Eng., 2001.
[13] A.L. Comrey and C.K. Staats, "Group Performance in a Cognitive Task," J. Applied Psychology, vol. 39, pp. 354-356, 1955.
[14] M.A. Conard, "Aptitude Is Not Enough: How Personality and Behavior Predict Academic Performance," J. Research in Personality, vol. 40, pp. 449-346, 2006.
[15] P.T. Costa and R.R. McCrae, The NEO Personality Inventory Manual, 1985.
[16] J.W. Creswell, Research Design Qualitative, Quantitative and Mixed Method Approaches. Sage Publications, 2003.
[17] I.K. Crombie, The Pocket Guide to Appraisal. BMJ Books, 1996.
[18] T.H. DeClue, "Pair Programming and Pair Trading: Effects on Learning and Motivation in a CS2 Courses," J. Computing Sciences in Colleges, vol. 18, no. 5, pp. 49-56, 2003.
[19] T. Dyba, E. Arisholm, D.I.L. Sjoberg, J.E. Hannay, and F. Shull, "Are Two Heads Better Than One? On the Effectiveness of Pair Programming," IEEE Software, vol. 24, no. 6, pp. 12-15, Nov./Dec. 2007.
[20] T. Dyba and T. Dingsoyr, "Empirical Studies of Agile Software Development: A Systematic Review," Information and Software Technology, vol. 50, pp. 833-859, 2008.
[21] B.V. Elsevier, "Scopus Overview: What Is It," http://www.info. scopus.comabout/, 2008.
[22] T. Farsides and R. Woodfield, "Individual Differences and Undergraduate Academic Success: The Roles of Personality, Intelligence, and Application," Personality and Individual Differences, vol. 34, pp. 1225-1243, 2003.
[23] R.M. Felder and L.K. Silverman, "Learning and Teaching Styles in Engineering Education," Eng. Education, vol. 78, pp. 674-681, 1988.
[24] A. Fink, Conducting Research Literature Review: From Paper to the Internet. Sage Publication, Inc., 2005.
[25] D.R. Forsyth, Group Dynamics, fourth ed. Thomson Wadsworth, 2006.
[26] A. Furnham, "The Big Five vs the Big Four: The Relationship between Myers-Briggs Type Indicator (MBTI) and NEO-PI Five Factor Model of Personality," Personality and Individual Differences, vol. 21, pp. 303-307, 1996.
[27] T. Greenhalgh, How to Read a Paper: The Basics of Evidence-Based Medicine. BMJ Books, 2000.
[28] B. Hanks, C. McDowell, D. Draper, and M. Krnjajic, "Program Quality with Pair Programming in CS1," ACM SIGCSE Bull., vol. 36, pp. 176-180, 2004.
[29] E.V. Howard, "Attitudes on Using Pair-Programming," J. Educational Technology Systems, vol. 35, pp. 89-103, 2006.
[30] A. Jedlitschka and D. Pfahl, "Reporting Guidelines for Controlled Experiments in Software Engineering," Proc. Int'l Symp. Empirical Software Eng., pp. 95-104, 2005.
[31] V.B. Kampenes, T. Dyba, J.E. Hannay, and D.I.K. Sjoberg, "A Systematic Review of Effect Size in Software Engineering," Information and Software Technology, vol. 49, pp. 1073-1086, 2007.
[32] N. Katira, L. Williams, E. Wiebe, C. Miller, S. Balik, and E. Gehringer, "On Understanding Compatibility of Student Pair Programmers," ACM SIGCSE Bull., vol. 36, pp. 7-11, 2004.
[33] D. Keirsey and M. Bates, Please Understand Me. Prometheus Book Company, 1984.
[34] K.S. Khan, R. Kunz, J. Kleijnen, and G. Antes, Systematic Review to Support Evidence-Based Medicine. The Royal Soc. of Medicine Press, Ltd., 2003.
[35] B.A. Kitchenham and S. Charters, "Procedures for Performing Systematic Literature Review in Software Engineering," EBSE Technical Report version 2.3, EBSE-2007-01, Software Eng. Group, Keele Univ., Univ. of Durham, 2007.
[36] B.A. Kitchenham, E. Mendes, and G.H. Travassos, "Cross versus Within-Company Cost Estimation Studies: A Systematic Review," IEEE Trans. Software Eng., vol. 33, no. 5, pp. 316-329, May 2007.
[37] B.A. Kitchenham, S.L. Pfleeger, L.M. Pickard, P.W. Jones, D.C. Hoaglin, K. El Emam, and J. Rosenberg, "Preliminary Guidelines for Empirical Research in Software Engineering," IEEE Trans. Software Eng., vol. 28, no. 8, pp. 721-734, Aug. 2002.
[38] L. Layman, "Changing Students' Perceptions: An Analysis of the Supplementary Benefits of Collaborative Software Development," Proc. 19th Conf. Software Eng. Education and Training, pp. 159-166, 2006.
[39] P.D. Leedy and J.E. Ormrod, Practical Research Planning and Design, eighth ed. Pearson/Merril/Prentice Hall, 2005.
[40] W.W. Liddel and J.J.W. Slocum, "The Effects of Individual-Role Compatibility upon Group Performance: An Extension of Schutz's FIRO Theory," The Academy of Management J., vol. 19, no. 3, pp. 413-426, 1976.
[41] J.A. Livermore, "What Elements of XP Are Being Adopted by Industry Practitioners?" Proc. IEEE Southeast Conf., pp. 149-152, May 2006.
[42] Y. Lou, P.C. Abrami, and S. d'Apollonia, "Small Group and Individual Learning with Technology: A Meta-Analysis," Rev. Educational Research, vol. 71, no. 3, pp. 449-521, 2001.
[43] R.C. Martin, "OO Design Quality Metrics: An Analysis of Dependencies," Proc. Workshop Pragmatic and Theoretical Directions in Object-Oriented Software Metrics, 1994.
[44] S. McConnel, Code Complete: A Practical Handbook of Software Construction. Microsoft Press, 1993.
[45] R.R. McCrae and O.P. John, "An Introduction to the Five-Factor Model and Its Application," J. Personality, vol. 60, no. 2, pp. 175-215, 1992.
[46] C. McDowell, L. Werner, H.E. Bullock, and J. Fernald, "The Impact of Pair Programming on Student Performance, Perception and Persistence," Proc. 25th Int'l Conf. Software Eng., pp. 602-607, 2003.
[47] E. Mendes, L. Al-Fakhri, and A. Luxton-Reilly, "Investigating Pair-Programming in a 2nd-Year Software Development and Design Computer Science Course," ACM SIGCSE Bull., vol. 37, pp. 296-300, 2005.
[48] E. Mendes, "A Systematic Review of Web Engineering Research," Proc. Int'l Symp. Empirical Software Eng., pp. 498-507, 2005.
[49] E. Mendes, L. Al-Fakhri, and A. Luxton-Reilly, "A Replicated Experiment of Pair-Programming in a 2nd-Year Software Development and Design Computer Science Course," Proc. 11th Ann. SIGCSE Conf. Innovation and Technology in Computer Science Education, pp. 108-112, 2006.
[50] S. Mohammed and L.C. Angell, "Personality Heterogeneity in Teams: Which Differences Make a Difference for Team Performance?" Small Group Research, vol. 34, pp. 651-677, 2003.
[51] M.M. Muller and F. Padberg, "An Empirical Study about the Feelgood Factor in Pair Programming," Proc. 10th Int'l Symp. Software Metrics, pp. 151-158, 2004.
[52] I. Myers-Briggs, M.H. McCaulley, N.L. Quenk, and A. Hammer, MBTI Manual (A Guide to the Development and use of the Myers Briggs Type Indicator), third ed. Consulting Psychologists Press, 1998.
[53] N. Nagappan, L. Williams, M. Ferzli, E. Wiebe, K. Yang, C. Miller, and S. Balik, "Improving the CS1 Experience with Pair Programming," ACM SIGCSE Bull., vol. 35, pp. 359-362, 2003.
[54] A.I. Nevin, K.A. Smith, and A. Udvari-Solner, "Cooperative Group Learning and Higher Education," Creativity and Collaborative Learning: A Practical Guide to Empowering Students and Teachers, J.S. Thousand, R.A. Villa, and A.I. Nevin, eds., Paul H. Brookes Publishing Co., Inc., 1994.
[55] J.T. Nosek, "The Case for Collaborative Programming," Comm. ACM, vol. 41, pp. 105-108, 1998.
[56] M. Petticrew and H. Roberts, Systematic Review in the Social Sciences: A Practical Guide. Blackwell Publishing, 2006.
[57] L.M. Pickard, B.A. Kitchenham, and P.W. Jones, "Combining Empirical Results in Software Engineering," Information and Software Technology, vol. 40, no. 14, pp. 811-821, 1998.
[58] D.J. Pittenger, "Measuring the MBTI…and Coming Up Short," J. Career Planning and Employment, vol. 54, pp. 48-52, 1993.
[59] K.M. Slaten, M. Droujkova, S.B. Berenson, L. Williams, and L. Layman, "Undergraduate Student Perceptions of Pair Programming and Agile Software Methodologies: Verifying a Model of Social Interaction," Proc. Agile Conf. '05, pp. 323-330, 2005.
[60] R.E. Slavin, "Cooperative Learning," Rev. of Educational Research, vol. 50, no. 2, pp. 315-342, 1980.
[61] R.E. Slavin, Cooperative Learning: Theory, Research and Practice. Prentice Hall, 1990.
[62] L. Spencer, J. Ritchie, J. Lewis, and L. Dillon, "Quality in Qualitative Evaluation: A Framework for Assessing Research Evidence," Govt. Chief Social Researcher's Office, 2003.
[63] L.D. Steiner, Group Process and Productivity. Academic Press, 1972.
[64] L. Thomas, M. Ratcliffe, and A. Robertson, "Code Warriors and Code-a-Phobes: A Study in Attitude and Pair Programming," ACM SIGCSE Bull., vol. 35, pp. 363-367, 2003.
[65] J. Vanhanen and C. Lassenius, "Effects of Pair Programming at the Development Team Level: An Experiment," Proc. Int'l Symp. Software Eng., pp. 336-345, 2005.
[66] L.S. Vygotsky, Mind in Society: The Development of Higher Psychological Processes. MIT Press, 1978.
[67] L.L. Werner, B. Hanks, and C. McDowell, "Pair-Programming Helps Female Computer Science Students," J. Educational Resources in Computing, vol. 4, 2004.
[68] L.A. Williams, "The Collaborative Software Process," PhD dissertation, Univ. of Utah, 2000.
[69] L.A. Williams and R.R. Kessler, "The Effects of 'Pair-Pressure' and 'Pair-Learning' on Software Engineering Education," Proc. 13th Conf. Software Eng. Education and Training, pp. 59-65, 2000.
[70] L. Williams and R.R. Kessler, Pair Programming Iluminated. Addision-Wesley Longman Publishing Co., Inc., 2002.
[71] L. Williams, R.R. Kessler, W. Cunningham, and R. Jeffries, "Strengthening the Case for Pair Programming," IEEE Software, vol. 17, no. 4, pp. 19-25, July/Aug. 2000.
[72] L. Williams, L. Layman, J. Osborne, and N. Katira, "Examining the Compatibility of Student Pair Programmers," Proc. AGILE 2006 Conf., 2006.
[73] L. Williams, C. McDowell, N. Nagappan, J. Fernald, and L. Werner, "Building Pair Programming Knowledge through a Family of Experiments," Proc. 2003 Int'l Symp. Empirical Software Eng., pp. 143-152, 2003.
[74] C. Wohlin, P. Runeson, M. Host, M.C. Ohlsson, B. Regnell, and A. Wesslen, Experimentation in Software Eng.: An Introduction. Kluwer Academic Publisher, 2000.
[75] R. Zemke, "Second Thoughts about the MBTI," Training, vol. 29, no. 4, pp. 43-47, 1992.

Index Terms:
Empirical studies, pair programming, systematic review.
Citation:
Norsaremah Salleh, Emilia Mendes, John C. Grundy, "Empirical Studies of Pair Programming for CS/SE Teaching in Higher Education: A Systematic Literature Review," IEEE Transactions on Software Engineering, vol. 37, no. 4, pp. 509-525, July-Aug. 2011, doi:10.1109/TSE.2010.59
Usage of this product signifies your acceptance of the Terms of Use.