This Article 
 Bibliographic References 
 Add to: 
Qualitative Methods in Empirical Studies of Software Engineering
July/August 1999 (vol. 25 no. 4)
pp. 557-572

Abstract—While empirical studies in software engineering are beginning to gain recognition in the research community, this subarea is also entering a new level of maturity by beginning to address the human aspects of software development. This added focus has added a new layer of complexity to an already challenging area of research. Along with new research questions, new research methods are needed to study nontechnical aspects of software engineering. In many other disciplines, qualitative research methods have been developed and are commonly used to handle the complexity of issues involving human behavior. This paper presents several qualitative methods for data collection and analysis and describes them in terms of how they might be incorporated into empirical studies of software engineering, in particular how they might be combined with quantitative methods. To illustrate this use of qualitative methods, examples from real software engineering studies are used throughout.

[1] T. Athey, "Leadership Challenges for the Future," IEEE Software, Vol. 15, No. 3, May/June 1998, pp. 72-77.
[2] S.R. Barley, “The Alignment of Technology and Structure through Roles and Networks,” Administrative Science Quarterly, vol. 35, pp. 61–103, 1990.
[3] V.R. Basili, R.W. Selby, and D.H. Hutchens, "Experimentation in Software Engineering," IEEE Trans. Software Eng., vol. 12, pp. 733-743, 1986.
[4] F.P. Brooks, Jr., The Mythical Man-Month: Essays on Software Engineering, Addison Wesley Longman, Reading, Mass., 1975.
[5] N. Brown, "Industrial-Strength Management Strategies," IEEE Software, July 1996, pp. 94-103.
[6] B. Curtis, “By the Way, Did Anyone Study any Real Programmers?” Empirical Studies of Programmers: First Workshop, pp. 256–262, 1986.
[7] K.M. Eisenhardt, “Building Theories from Case Study Research,” Academy of Management Review, vol. 14, pp. 532–550, 1989.
[8] J.F. Gilgun, “Definitions, Methodologies, and Methods in Qualitative Family Research,” Qualitative Methods in Family Research. Thousand Oaks: Sage, 1992.
[9] B.G. Glaser and A.L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine Publishing, 1967.
[10] J.T. Hackos and J.D. Redish, User and Task Analysis for Interface Design. pp. 258–259, New York: John Wiley&Sons, ch. 9, 1998.
[11] C.M. Judd, E.R. Smith, and L.H. Kidder, Research Methods in Social Relations, sixth ed., ch. 13, pp. 298–320, Ft. Worth: Harcourt Brace Jova novich, 1991.
[12] Y.S. Lincoln and E.G. Guba, Naturalistic Inquiry. Thousand Oaks Calif.: Sage, 1985.
[13] S. McConnell, Rapid Development, Microsoft Press, Redmond, Wash., 1996.
[14] M.B. Miles and A.M. Huberman, Qualitative Data Analysis: An Expanded Sourcebook, second ed. Thousand Oaks, Calif.: Sage, 1994.
[15] A. Parra, C. Seaman, V.R. Basili, S. Kraft, S. Condon, S. Burke, and D. Yakimovich, “The Package-Based Development Process in the Flight Dynamics Division,” Proc. 22nd Software Eng. Workshop, pp. 21–56, NASA/Goddard Space Flight Center Software Eng. Laboratory (SEL), Dec. 1997.
[16] D.E. Perry, N.A. Staudenmayer, and L.G. Votta, "People, Organizations, and Process Improvement." IEEE Software, vol. 11, no. 4, pp. 36-45, 1994.
[17] C.B. Seaman and V.R. Basili, “Communication and Organization: An Empirical Study of Discussion in Inspection Meetings,” IEEE Trans. Software Eng., vol. 24, no. 7, pp. 559–572, July 1998.
[18] B. Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction, 3rd Edition, Addison Wesley, Reading, Mass., 1998, to appear.
[19] S.J. Taylor and R. Bogdan, Introduction to Qualitative Research Methods. New York: John Wiley&Sons, 1984.
[20] A. von Mayrhauser and A.M. Mans, “Identification of Dynamic Comprehension Processes During Large Scale Maintenance,” IEEE Trans. Software Eng., vol. 22, no. 6, pp. 424–437, 1996.
[21] W.F. Whyte, Learning from the Field: A Guide from Experience. Beverly Hills, Calif.: Sage, 1984.

Index Terms:
Qualitative methods, data collection, data analysis, experimental design, empirical software engineering, participant observation, interviewing.
Carolyn B. Seaman, "Qualitative Methods in Empirical Studies of Software Engineering," IEEE Transactions on Software Engineering, vol. 25, no. 4, pp. 557-572, July-Aug. 1999, doi:10.1109/32.799955
Usage of this product signifies your acceptance of the Terms of Use.