The Community for Technology Leaders
RSS Icon
Issue No.01 - January/February (2010 vol.36)
pp: 61-80
Jo E. Hannay , Simula Research Laboratory, Lysaker and University of Oslo, Oslo
Erik Arisholm , Simula Research Laboratory, Lysaker and University of Oslo, Oslo
Harald Engvik , University of Oslo, Oslo
Dag I.K. Sjøberg , University of Oslo, Oslo
Personality tests in various guises are commonly used in recruitment and career counseling industries. Such tests have also been considered as instruments for predicting the job performance of software professionals both individually and in teams. However, research suggests that other human-related factors such as motivation, general mental ability, expertise, and task complexity also affect the performance in general. This paper reports on a study of the impact of the Big Five personality traits on the performance of pair programmers together with the impact of expertise and task complexity. The study involved 196 software professionals in three countries forming 98 pairs. The analysis consisted of a confirmatory part and an exploratory part. The results show that: 1) Our data do not confirm a meta-analysis-based model of the impact of certain personality traits on performance and 2) personality traits, in general, have modest predictive value on pair programming performance compared with expertise, task complexity, and country. We conclude that more effort should be spent on investigating other performance-related predictors such as expertise, and task complexity, as well as other promising predictors, such as programming skill and learning. We also conclude that effort should be spent on elaborating on the effects of personality on various measures of collaboration, which, in turn, may be used to predict and influence performance. Insights into such malleable, rather than static, factors may then be used to improve pair programming performance.
Pair programming, personality, Big Five, expertise, task complexity, performance.
Jo E. Hannay, Erik Arisholm, Harald Engvik, Dag I.K. Sjøberg, "Effects of Personality on Pair Programming", IEEE Transactions on Software Engineering, vol.36, no. 1, pp. 61-80, January/February 2010, doi:10.1109/TSE.2009.41
[1] M. Ally, F. Darroch, and M. Toleman, "A Framework for Understanding the Factors Influencing Pair Programming Success," Proc. Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 82-91, 2005.
[2] E. Arisholm, H. Gallis, T. Dybå, and D.I.K. Sjøberg, "Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise," IEEE Trans. Software Eng., vol. 33, no. 2, pp. 65-86, Feb. 2007.
[3] E. Arisholm and D.I.K. Sjøberg, "Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software," IEEE Trans. Software Eng., vol. 30, no. 8, pp. 521-534, Aug. 2004.
[4] E. Arisholm, D.I.K. Sjøberg, G.J. Carelius, and Y. Lindsjørn, "A Web-Based Support Environment for Software Engineering Experiments," Nordic J. Computing, vol. 9, no. 4, pp. 231-247, 2002.
[5] S.B. Bacharach, "Organizational Theories: Some Criteria for Evaluation," Academy of Management Rev., vol. 14, no. 4, pp. 496-515, 1989.
[6] M.B. Barrick, M.K. Mount, and T.A. Judge, "Personality and Performance at the Beginning of the New Millennium: What Do We Know and Where Do We Go Next?" Int'l J. Selection and Assessment, vol. 9, nos. 1/2, pp. 9-30, 2001.
[7] M.R. Barrick, G.L. Stewart, M.J. Neubert, and M.K. Mount, "Relating Member Ability and Personality to Work-Team Processes and Team Effectiveness," J. Applied Psychology, vol. 83, no. 3, pp. 377-391, 1998.
[8] A. Basilevsky, Statistical Factor Analysis and Related Methods: Theory and Applications. John Wiley and Sons, Inc., 1994.
[9] K. Beck and C. Andres, Extreme Programming Explained: Embrace Change, second ed. Addison-Wesley, 2003.
[10] S.T. Bell, "Deep-Level Composition Variables as Predictors of Team Performance: A Meta-Analysis," J. Applied Psychology, vol. 92, no. 3, pp. 595-615, 2007.
[11] K. Bollen and R. Lennox, "Conventional Wisdom on Measurement: A Structural Equation Perspective," Psychological Bull., vol. 110, no. 2, pp. 305-314, 1991.
[12] K.A. Bollen and R.A. Stine, "Bootstrapping Goodness-of-Fit Measures in Structural Equation Models," Sociological Methods and Research, vol. 21, pp. 205-229, 1992.
[13] L.C. Briand and J. Wust, "Modeling Development Effort in Object-Oriented Systems Using Design Properties," IEEE Trans. Software Eng., vol. 27, no. 11, pp. 963-986, Nov. 2001.
[14] M.W. Browne and R. Cudeck, "Alternative Ways of Assessing Model Fit," Testing Structural Equation Models, K.A. Bollen and J.S. Long, eds., pp. 136-162, Sage Publications, 1993.
[15] S. Bryant, B. du Boulay, and P. Romero, "XP and Pair Programming Practices," J. Computer Soc. of India, extended version at pdf, newsletter of the Psychology of Programming Interest Group (PPIG), vol. 30, no. 5, pp. 17-20, 2007.
[16] C.B. Jarvis, S.B. Mackenzie, and P.M. Podsakoff, "A Critical Review of Construct Indicators and Measurement Model Misspecification in Marketing and Consumer Research," J. Consumer Research, vol. 30, pp. 199-218, Sept. 2003.
[17] J.M. Burkhardt, F. Détienne, and S. Wiedenbeck, "Object-Oriented Program Comprehension: Effect of Expertise, Task and Phase," Empirical Software Eng., vol. 7, no. 2, pp. 115-156, June 2002.
[18] J.N. Butcher, W. Dahlstrom, J.R. Graham, A. Tellegen, and B. Kaemmer, Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for Administration and Scoring. Univ. of Minnesota Press, 1989.
[19] L. Cao and P. Xu, "Activity Patterns of Pair Programming," Proc. 38th Ann. Hawaii Int'l Conf. System Sciences, pp. 1-10, 2005.
[20] R.B. Cattell, The Description and Measurement of Personality. Harcourt, Brace & World, 1946.
[21] R.B. Cattell, Personality and Motivation Structure and Measurement. World Book, 1957.
[22] C.G. Cegielski and D.J. Hall, "What Makes a Good Programmer?" Comm. ACM, vol. 49, no. 10, pp. 73-75, Oct. 2006.
[23] J. Chao and G. Atli, "Critical Personality Traits in Successful Pair Programming," Proc. Agile '06 Conf., 2006.
[24] G.W. Cheung, "Introducing the Latent Congruence Model for Improving the Assessment of Similarity, Agreement, and Fit in Organizational Research," Organizational Research Methods, vol. 12, pp. 6-33, 2009.
[25] G.W. Cheung, "A Multiple-Perspective Approach to Data Analysis in Congruence Research," Organizational Research Methods, vol. 12, pp. 63-68, 2009.
[26] K.S. Choi, "A Discovery and Analysis of Influencing Factors of Pair Programming," PhD dissertation, Dept. of Information Systems, Faculty of New Jersey Inst. of Tech nology, 2004.
[27] A. Cockburn, Agile Software Development. Addison-Wesley, 2002.
[28] B.P. Cohen, Developing Sociological Knowledge: Theory and Method, second ed. Nelson-Hall Publishers, 1989.
[29] L.L. Constantine, Constantine on Peopleware. Prentice Hall, 1995.
[30] J.O. Coplien, "A Generative Development-Process Pattern Language," Pattern Languages of Program Design, J.O. Coplien and D.C. Schmidt, eds., pp. 183-237, Addison-Wesley, 1995.
[31] P.T. Costa and R.R. McCrae, The NEO Personality Inventory Manual, 1985.
[32] P. Cramer, The Development of Defense Mechanisms: Theory, Research, and Assessment. Springer-Verlag, 1991.
[33] A.D. Da Cunha and D. Greathead, "Does Personality Matter? An Analysis of Code Review Ability," Comm. ACM, vol. 50, no. 5, pp. 109-112, 2007.
[34] A. Diamantopoulos and J.A. Siguaw, "Formative versus Reflective Indicators in Organizational Measure Development: A Comparison and Empirical Illustration," British J. Management, vol. 17, pp. 263-282, 2006.
[35] A.J. Dick and B. Zarnett, "Paired Programming & Personality Traits," Proc. Third Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 82-85, 2002.
[36] D.H. Dickson and I.W. Kelly, "The 'Barnum Effect' in Personality Assessment: A Review of the Literature," Psychological Reports, vol. 57, pp. 367-382, 1985.
[37] R.M. Felder and L.K. Silverman, "Learning and Teaching Styles in Engineering Education," Eng. Education, vol. 78, no. 7, pp. 674-681, 1988.
[38] N.V. Flor and E.L. Hutchins, "Analyzing Distributed Cognition in Software Teams: A Case Study of Team Programming during Perfective Software Maintenance," Proc. Fourth Workshop Empirical Studies of Programmers, pp. 36-64, 1991.
[39] B.R. Forer, "The Fallacy of Personal Validation: A Classroom Demonstration of Gullibility," J. Abnormal and Social Psychology, vol. 44, pp. 118-123, 1949.
[40] D.R. Forsyth, Group Dynamics, fourth ed. Thomson Wadsworth, 2006.
[41] R.J. Freund, W.J. Wilson, and P. Sa, Regression Analysis: Statistical Modeling of a Response Variable, second ed. Academic Press, 2006.
[42] A. Furnham, "The Big Five versus the Big Four: The Relationship between the Myers-Briggs Type Indicator (MBTI) and NEO-PI Five Factor Model of Personality," Personality and Individual Differences, vol. 21, no. 2, pp. 303-307, 1996.
[43] H. Gallis, E. Arisholm, and T. Dybå, "An Initial Framework for Research on Pair Programming," Proc. 2003 Int'l Symp. Empirical Software Eng., pp. 132-142, 2003.
[44] L.R. Goldberg, "An Alternative Description of Personality: The Big-Five Factor Structure," J. Personality and Social Psychology, vol. 59, pp. 1216-1229, 1990.
[45] L.R. Goldberg, "The Development of Markers for the Big-Five Factor Structure," Psychological Assessment, vol. 4, no. 1, pp. 26-42, 1992.
[46] L.R. Goldberg, "The Structure of Phenotypic Personality Traits," Am. Psychologist, vol. 48, pp. 26-34, 1993.
[47] L.R. Goldberg, "A Broad-Bandwidth, Public Domain, Personality Inventory Measuring the Lower-Level Facets of Several Five-Factor Models," Personality Psychology in Europe, I. Mervielde, I. Deary, F.D. Fruyt, and F. Ostendorf, eds., vol. 7, pp. 7-28, Tilburg Univ. Press, 1999.
[48] L.R. Goldberg, J.A. Johnson, H.W. Eber, R. Hogan, M.C. Ashton, C.R. Cloninger, and H.C. Gough, "The International Personality Item Pool and the Future of Public-Domain Personality Measures," J. Research in Personality, vol. 40, pp. 84-96, 2006.
[49] D. Goleman, "What Makes a Leader?" Harvard Business Rev., vol. 76, no. 6, pp. 92-105, 1998.
[50] R.L. Gorsuch, Factor Analysis, second ed. Lawrence Erlbaum Assoc., 1983.
[51] S. Gregor, "The Nature of Theory in Information Systems," MIS Quarterly, vol. 30, no. 3, pp. 611-642, Sept. 2006.
[52] J. Grenning, "Launching Extreme Programming at a Process-Intensive Company," IEEE Software, vol. 18, no. 6, pp. 27-33, Nov./Dec. 2001.
[53] T. Hærem, "Task Complexity and Expertise as Determinants of Task Perceptions and Performance: Why Technology-Structure Research Has Been Unreliable and Inconclusive," PhD dissertation, Norwegian School of Management BI, 2002.
[54] T. Hærem and D. Rau, "The Influence of Degree of Expertise and Objective Task Complexity on Perceived Task Complexity and Performance," J. Applied Psychology, vol. 92, no. 5, pp. 1320-1331, 2007.
[55] B. Hanks, "Student Attitudes toward Pair Programming," Proc. 11th Ann. Conf. Innovation and Technology in Computer Science Education, pp. 113-117, 2006.
[56] J.E. Hannay and M. Jørgensen, "The Role of Deliberate Artificial Design Elements in Software Engineering Experiments," IEEE Trans. Software Eng., vol. 34, no. 2, pp. 242-259, Mar./Apr. 2008.
[57] J.E. Hannay, D.I.K. Sjøberg, and T. Dybå, "A Systematic Review of Theory Use in Software Engineering Experiments," IEEE Trans. Software Eng., vol. 33, no. 2, pp. 87-107, Feb. 2007.
[58] W.E. Hanson and C.D. Claiborn, "Effects of Test Interpretation Style and Favorability in the Counseling Process," J. Counseling and Development, vol. 84, no. 3, pp. 349-358, 2006.
[59] W.K. Hastings, "Monte Carlo Sampling Methods Using Markov Chains and Their Applications," Biometrika, vol. 57, no. 1, pp. 97-109, 1970.
[60] S.R. Hathaway and J.C. McKinley, "A Multiphasic Personality Schedule (Minnesota): I. Construction of the Schedule," J. Psychology, vol. 10, pp. 249-254, 1940.
[61] M. Höst, B. Regnell, and C. Wohlin, "Using Students as Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment," Empirical Software Eng., vol. 5, no. 3, pp. 201-214, Nov. 2000.
[62] D.C. Howell, "The Treatment of Missing Data," The Sage Handbook of Social Science Methodology, W. Outhwaite and S.P. Turner, eds., Sage Publications, Ltd., 2007.
[63] L. Hu and P.M. Bentler, "Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives," Structural Equation Modeling, vol. 6, pp. 1-55, 1999.
[64] "International Personality Item Pool. A Scientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences," http:/, 2007.
[65] N. Katira, L. Williams, E. Wiebe, C. Miller, S. Balik, and E. Gehringer, "On Understanding Compatibility of Student Pair Programmers," Proc. 35th Technical Symp. Computer Science Education, pp. 7-11, 2004.
[66] D. Keirsey, Please Understand Me II. Prometheus Nemesis Book Company, 1988.
[67] D. Keirsey and M. Bates, Please Understand Me. Prometheus Book Company, 1984.
[68] S.L. Kichuk and W.H. Wiesner, "Work Teams: Selecting Members for Optimal Performance," Canadian Psychology, vol. 39, pp. 23-32, 1998.
[69] B.A. Kitchenham and S. Charters, "Guidelines for Performing Systematic Literature Reviews in Software Engineering," EBSE Technical Report EBSE-2007-01, Keele Univ., 2007.
[70] R.B. Kline, Principles and Practice of Structural Equation Modeling, second ed. The Guilford Press, 2005.
[71] L. Layman, "Changing Students' Perceptions: An Analysis of the Supplementary Benefits of Collaborative Software Development," Proc. 19th Conf. Software Eng. Education and Training, 2006.
[72] S.Y. Lee, Structural Equation Modelling. A Bayesian Approach. Wiley, 2007.
[73] C.E. Lindblom, "Alternatives to Validity. Some Thoughts Suggested by Campbell's Guidelines," Knowledge Creation, Diffusion, Utilization, vol. 8, pp. 509-520, 1987.
[74] R.J.A. Little and D.B. Rubin, Statistical Analysis with Missing Data, second ed. John Wiley & Sons, Inc., 2002.
[75] J.C. Loehlin, Latent Variable Models: An Introduction to Factor, Path, and Structural Equation Analysis, fourth ed. Lawrence Erlbaum Assoc., 2003.
[76] B. Markovsky, "The Structure of Theories," Group Processes, M. Foschi and E.J. Lawler, eds., pp. 3-24, Nelson-Hall Publishers, 1994.
[77] W.M. Marston, Emotions of Normal People. Harcourt, Brace & Co., 1928.
[78] P.E. Meehl, "Wanted—A Good Cookbook," Am. Psychologist, vol. 11, no. 3, pp. 263-272, 1956.
[79] N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller, and E. Teller, "Equations of State Calculations by Fast Computing Machines," J. Chemical Physics, vol. 21, no. 6, pp. 1087-1092, 1953.
[80] S. Mohammed and L.C. Angell, "Personality Heterogeneity in Teams: Which Differences Make a Difference for Team Performance?" Small Group Research, vol. 34, pp. 651-677, 2003.
[81] J.E. Moore, "Personality Characteristics of Information Systems Professionals," Proc. 1991 SIGCPR, pp. 140-155, 1991.
[82] I.B. Myers and M.H. McCaulley, A Guide to the Development and Use of the Myers-Briggs Type Indicator, Consulting Psychologists Press, 1985.
[83] I.B. Myers and P. Myers, Gifts Differing: Understanding Personality Type. Davies-Black Publishing, 1995.
[84] G.A. Neuman, S.H. Wagner, and N.D. Christiansen, "The Relationship between Work-Team Personality Composition and the Job Performance of Teams," Group and Organization Management, vol. 24, pp. 28-45, 1999.
[85] A. Newell, J.C. Shaw, and H.A. Simon, "Elements of a Theory of Human Problem Solving," Psychological Rev., vol. 65, pp. 151-166, 1958.
[86] J.C. Nunnally and I. Bernstein, Psychometric Theory, third ed. McGraw-Hill, Inc., 1994.
[87] A.M. Paul, The Cult of Personality Testing: How Personality Tests Are Leading Us to Miseducate Our Children, Mismanage Our Companies, and Misunderstand Ourselves. Free Press, 2005.
[88] M.A.G. Peeters, H.F.J.M. van Tuijl, C.G. Rutte, and I.M.M.J. Reymen, "Personality and Team Performance: A Meta-Analysis," European J. Personality, vol. 20, pp. 377-396, 2006.
[89] L.A. Pervin and O.P. John, Personality: Theory and Research, seventh ed. John Wiley & Sons, Inc., 1997.
[90] S. Ramanujan, R.W. Scamell, and J.R. Shah, "An Experimental Investigation of the Impact of Individual, Program, and Organizational Characteristics on Software Maintenance Effort," J. Systems and Software, vol. 54, no. 2, pp. 137-157, Oct. 2000.
[91] A. Saggio, C. Cooper, and P. Kline, "A Confirmatory Factor Analysis of the Myers-Briggs Type Indicator," Personality and Individual Differences, vol. 30, pp. 3-9, 2001.
[92] A. Saggio and P. Kline, "The Location of the Myers-Briggs Type Indicator in Personality Factor Space," Personality and Individual Differences, vol. 21, no. 4, pp. 591-597, 1996.
[93] J. Sall, "Monte Carlo Calibration of Distributions of Partition Statistics," technical report, SAS Inst., , 2002.
[94] P. Sfetsos, I. Stamelos, L. Angelis, and I. Deligiannis, "Investigating the Impact of Personality Types on Communication and Collaboration-Viability in Pair Programming—An Empirical Study," Proc. Seventh Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 43-52, 2006.
[95] W.R. Shadish, T.D. Cook, and D.T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, 2002.
[96] T.M. Shaft and I. Vessey, "The Relevance of Application Domain Knowledge," J. Management Information Systems, vol. 15, no. 1, pp. 51-78, 1998.
[97] B. Shneiderman, Software Psychology: Human Factors in Computer and Information Systems. Winthrop Publishers, 1980.
[98] D.I.K. Sjøberg, B. Anda, E. Arisholm, T. Dybå, M. Jørgensen, A. Karahasanović, and M. Vokáč, "Challenges and Recommendations When Increasing the Realism of Controlled Software Engineering Experiments," Empirical Methods and Studies in Software Engineering: Experiences from ESERNET, R. Conradi and A.I. Wang, eds., pp. 24-38, Springer-Verlag, 2003.
[99] D.I.K. Sjøberg, J.E. Hannay, O. Hansen, V.B. Kampenes, A. Karahasanović, N.K. Liborg, and A.C. Rekdal, "A Survey of Controlled Experiments in Software Engineering," IEEE Trans. Software Eng., vol. 31, no. 9, pp. 733-753, Sept. 2005.
[100] D.C. Smith, "The Personality of the Systems Analyst: An Investigation," SIGCPR Computer Personnel, vol. 12, no. 2, pp. 12-14, 1989.
[101] E. Soloway, J. Pinto, S. Letovsky, D. Littman, and R. Lampert, "Designing Documentation to Compensate for Delocalized Plans," Comm. ACM, vol. 31, no. 11, pp. 1259-1267, Nov. 1988.
[102] S. Sonnentag, "Expertise in Professional Software Design," J. Applied Psychology, vol. 83, no. 5, pp. 703-715, 1998.
[103] I.D. Steiner, Group Process and Productivity. Academic Press, 1972.
[104] R.J. Sternberg, "Cognitive Conceptions of Expertise," Int'l J. Expert Systems, vol. 7, no. 1, pp. 1-12, 1994.
[105] L. Thomas, M. Ratcliffe, and A. Robertson, "Code Warriors and Code-a-Phobes: A Study in Attitude and Pair Programming," Proc. 34th Technical Symp. Computer Science Education, 2003.
[106] A.S. Tippetts and P.R. Marques, "Compensating for Deficiencies in Perinatal Data Sets: Parametric Perspectives," Treatment for Drug-Exposed Women and Children: Advances in Research Methodology. NIH Publication No. 96-3632, E.R. Rahdert, ed., Nat'l Inst. on Drug Abuse, pp. 272-291, 1996.
[107] B. Van Fraassen, The Scientific Image. Oxford Univ. Press, 1980.
[108] A.E.M. Van Vianen and C.K.W. De Dreu, "Personality in Teams: Its Relations to Social Cohesion, Task Cohesion, and Team Performance," European J. Work and Organizational Psychology, vol. 10, pp. 97-120, 2001.
[109] J. Vanhanen and C. Lassenius, "Effects of Pair Programming at the Development Team Level: An Experiment," Proc. 33rd EUROMICRO Conf. Software Eng. and Advanced Applications, pp. 211-218, 2007.
[110] K. Visram, "Extreme Programming: Pair-Programmers, Team Players or Future Leaders?" Proc. Eighth IASTED Int'l Conf. Software Eng. and Applications, pp. 659-664, 2004.
[111] G.M. Weinberg, The Psychology of Computer Programming. Van Nostrand Reinhold, 1971.
[112] G.M. Weinberg, The Psychology of Computer Programming, silver anniversary ed. Dorset House Publishing, 1998.
[113] D. Westen, "Clinical Assessment of Object Relations Using the TAT," J. Personality Assessment, vol. 56, no. 1, pp. 56-74, 1991.
[114] D.A. Whetten, "What Constitutes a Theoretical Contribution," Academy of Management Rev., vol. 14, no. 4, pp. 490-495, 1989.
[115] L. Williams and R.R. Kessler, Pair Programming Illuminated. Addison-Wesley, 2002.
[116] L. Williams, R.R. Kessler, W. Cunningham, and R. Jeffries, "Strengthening the Case for Pair Programming," IEEE Software, vol. 17, no. 4, pp. 19-25, Jul./Aug. 2000.
[117] L. Williams, L. Layman, J. Osborne, and N. Katira, "Examining the Compatibility of Student Pair Programmers," Proc. Agile '06 Conf., 2006.
[118] L. Williams, A. Shukla, and A.I. Antón, "An Initial Exploration of the Relationship between Pair Programming and Brooks' Law," Proc. Agile Development Conf., pp. 11-20, 2004.
[119] R.J. Wirfs-Brock, "Characterizing Your Application's Control Style," Report on Object Analysis and Design, vol. 1, no. 3, 1994.
[120] R.J. Wirfs-Brock and B. Wilkerson, "Object-Oriented Design: A Responsibility-Driven Approach," SIGPLAN Notices, vol. 24, no. 10, pp. 71-75, 1989.
[121] C.K. Woodruff, "Personality Profiles of Male and Female Data Processing Personnel," Proc. 17th Ann. Southeast Regional Conf., pp. 124-128, 1979.
47 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool