This Article 
 Bibliographic References 
 Add to: 
Combining Perceptions and Prescriptions in Requirements Engineering Process Assessment: An Industrial Case Study
September/October 2009 (vol. 35 no. 5)
pp. 593-606
Nannette P. Napier, Georgia Gwinnett College, Lawrenceville
Lars Mathiassen, Georgia State University, Atlanta
Roy D. Johnson, University of Pretoria, Pretoria
Requirements engineering (RE) is a key discipline in software development and several methods are available to help assess and improve RE processes. However, these methods rely on prescriptive models of RE; they do not, like other disciplines within software engineering, draw directly on stakeholder perceptions and subjective judgments. Given this backdrop, we present an empirical study in RE process assessment. Our aim was to investigate how stakeholder perceptions and process prescriptions can be combined during assessments to effectively inform RE process improvement. We first describe existing methods for RE process assessment and the role played by stakeholder perceptions and subjective judgments in the software engineering and management literature. We then present a method that combines perceptions and prescriptions in RE assessments together with an industrial case study in which the method was applied and evaluated over a three-year period at TelSoft. The data suggest that the combined method led to a comprehensive and rich assessment and it helped TelSoft consider RE as an important and integral part of the broader engineering context. This, in turn, led to improvements that combined plan-driven and adaptive principles for RE. Overall, the combined method helped TelSoft move from Level 1 to Level 2 in RE maturity, and the employees perceived the resulting engineering practices to be improved. Based on these results, we suggest that software managers and researchers combine stakeholder perceptions and process prescriptions as one way to effectively balance the specificity, comparability, and accuracy of software process assessments.

[1] G. Kotonya and I. Sommerville, Requirements Engineering Processes and Techniques. John Wiley & Sons, 1998.
[2] S. Beecham, T. Hall, C. Britton, M. Cottee, and A. Rainer, “Using an Expert Panel to Validate a Requirement Process Improvement Model,” J. Systems and Software, vol. 76, no. 3, pp. 251-275, 2005.
[3] R. Schmidt, K. Lyytinen, M. Keil, and P. Cule, “Identifying Software Project Risks: An International Delphi Study,” J.Management Information Systems, vol. 17, no. 4, pp. 5-36, 2001.
[4] L. Mathiassen, O.K. Ngwenyama, and I. Aaen, “Managing Change in Software Process Improvement,” IEEE Software, vol. 22, no. 6, pp. 84-91, Nov./Dec. 2005.
[5] P.A. Nielsen and J. Pries-Heje, “A Framework for Selecting an Assessment Strategy,” Improving Software Organizations: From Principles to Practice, L. Mathiassen, J. Pries-Heje, and O.Ngwenyama, eds., Addison-Wesley, 2002.
[6] M. Sanders and I. Richardson, “Research into Long-Term Improvements in Small- to Medium-Sized Organisations Using SPICE as a Framework for Standards,” Software Process Improvement and Practice, vol. 12, pp. 351-359, 2007.
[7] F. Pettersson, M. Ivarsson, T. Gorschek, and P. Ohman, “A Practitioner's Guide to Light Weight Software Process Assessment and Improvement Planning,” J. Systems and Software, vol. 81, no. 6, pp. 972-995, 2007.
[8] CMMI Product Team “CMMI for Systems Engineering/Software Engineering/Integrated Product and Process Development/Supplier Sourcing,” CMU/SEI-2002-TR-011, Software Eng. Inst., Carnegie Mellon Univ., 2002.
[9] ISO/IEC, ISO/IEC 15288:2002 Systems Engineering—Systems Life Cycle Processes, 2002.
[10] H.D. Frederiksen and L. Mathiassen, “Information-Centric Assessment of Software Metrics Practices,” IEEE Trans. Eng. Management, vol. 52, no. 3, pp. 350-362, Aug. 2005.
[11] X. Liu, Y. Sun, C.S. Veera, Y. Kyoya, and K. Noguchi, “Priority Assessment of Software Process Requirements from Multiple Perspectives,” J. Systems and Software, vol. 79, pp. 1649-1660, 2006.
[12] C. Wohlin and A.A. Andrews, “Prioritizing and Assessing Software Project Success Factors and Project Characteristics Using Subjective Data,” Empirical Software Eng., vol. 8, pp. 285-308, 2003.
[13] M. Host and C. Wohlin, “A Subjective Effort Estimation Experiment,” Int'l J. Information and Software Technology, vol. 39, no. 11, pp. 755-762, 1997.
[14] R.T. Hughes, “Expert Judgment as an Estimating Method,” Information and Software Technology, vol. 38, pp. 67-75, 1996.
[15] M. Lewis, B. Young, L. Mathiassen, A. Rai, and R. Welke, “Business Process Innovation Based on Stakeholder Perceptions,” Information, Knowledge, Systems Management, vol. 6, nos. 1/2, pp. 7-27, 2007.
[16] J. Ropponen and K. Lyytinen, “Components of Software Development Risk: How to Address Them? A Project Manager Survey,” IEEE Trans. Software Eng., vol. 26, no. 2, pp. 98-112, Feb. 2000.
[17] P.A. Nielsen and J. Nørbjerg, “Assessing Software Processes: Low Maturity or Sensible Practice,” Scandinavian J. Information Systems, vol. 13, pp. 23-36, 2001.
[18] B. Curtis and M. Paulk, “Creating a Software Process Improvement Program,” Information and Software Technology, vol. 35, nos.6/7, pp. 381-386, 1993.
[19] W.S. Humphrey, Managing the Software Process. Addison-Wesley, 1989.
[20] T.P. Rout and A. Tuffley, “Harmonizing ISO/IEC 15504 and CMMI,” Software Process Improvement and Practice, vol. 12, pp. 361-371, 2007.
[21] F.G. Wilkie, F. McCaffery, D. McFall, N. Lester, and E. Wilkinson, “A Low-Overhead Method for Software Process Appraisal,” Software Process Improvement and Practice, vol. 12, pp. 339-349, 2007.
[22] C.G. Von Wangenheim, T. Varkoi, and C.F. Salviano, “Standard Based Software Process Assessments in Small Companies,” Software Process: Improvement and Practice, vol. 11, no. 3, pp. 329-335, 2006.
[23] S. Beecham, T. Hall, and A. Rainer, “Defining a Requirements Process Improvement Model,” Software Quality J., vol. 13, no. 3, pp. 247-279, 2005.
[24] D. Damian, D. Zowghi, L. Vaidyanathasamy, and Y. Pal, “An Industrial Case Study of Immediate Benefits of Requirements Engineering Process Improvement at the Australian Center for Unisys Software,” Empirical Software Eng., vol. 9, nos. 1/2, pp. 45-75, 2004.
[25] K. El Emam and N.H. Madhavji, “Measuring the Success of Requirements Engineering Processes,” Proc. IEEE Int'l Symp. Requirements Eng., pp. 204-211, 1995.
[26] I. Sommerville and P. Sawyer, Requirements Engineering: A Good Practice Guide. John Wiley & Sons, 1997.
[27] L. Mathiassen, “Collaborative Practice Research,” Information Technology & People, vol. 15, no. 4, pp. 321-345, 2002.
[28] R.K. Yin, Case Study Research: Design and Methods, third ed., vol. 5. Sage, 2003.
[29] K. El Emam and A. Birk, “Validating the ISO/IEC 15504 Measure of Software Requirements Analysis Process Capability,” IEEE Trans. Software Eng., vol. 26, no. 6, pp. 541-566, June 2000.
[30] J.-M. Simon, “SPICE: Overview for Software Process Improvement,” J. Systems Architecture, vol. 42, no. 8, p. 633, 1996.
[31] V. Basili and H. Rombach, “The Tame Project: Towards Improvement-Oriented Software Environments,” IEEE Trans. Software Eng., vol. 14, no. 6, pp. 758-773, June 1988.
[32] M. Daneva, “Lessons Learnt from Five Years of Experience in ERP Requirements Engineering,” Proc. 11th IEEE Int'l Conf. Requirements Eng., pp. 45-54, 2003.
[33] M. Daneva, “Using Maturity Assessments to Understand the ERP Requirements Engineering Process,” Proc. IEEE Joint Int'l Conf. Requirements Eng., pp. 255-262, 2002.
[34] M. Niazi, “An Instrument for Measuring the Maturity of Requirements Engineering Process,” Proc. Sixth Int'l Conf. Product Focused Software Process Improvement, 2005.
[35] M. Kauppinen, T. Aaltio, and S. Kujala, “Lessons Learned from Applying the Requirements Engineering Good Practice Guide for Process Improvement,” Proc. Seventh Int'l Conf. Software Quality, 2002.
[36] M. Kauppinen, M. Vartiainen, J. Kontio, S. Kujala, and R. Sulonen, “Implementing Requirements Engineering Processes throughout Organizations: Success Factors and Challenges,” Information and Software Technology, vol. 46, no. 14, pp. 937-953, 2004.
[37] I. Sommerville and J. Ransom, “An Empirical Study of Industrial Requirements Engineering Process Assessment and Improvement,” ACM Trans. Software Eng. and Methodology, vol. 14, no. 1, pp. 85-117, 2005.
[38] K. Lyytinen, L. Mathiassen, and J. Ropponen, “Attention Shaping and Software Risk—A Categorical Analysis of Four Classical Risk Management Approaches,” Information Systems Research, vol. 9, no. 3, pp. 233-255, 1998.
[39] V. Kasi, M. Keil, L. Mathiassen, and K. Pedersen, “The Post Mortem Paradox: A Delphi Study of IT Specialist Perceptions,” European J. Information Systems, vol. 17, pp. 62-78, 2008.
[40] M. Tiedeman, “Post-Mortems—Methodology and Experiences,” IEEE J. Selected Areas in Comm., vol. 8, no. 2, pp. 176-180, Feb. 1990.
[41] B. McFeeley, “IDEAL: A User's Guide for Software Process Improvement,” CMU/SEI-96-HB-001, Software Eng. Inst., Carnegie Mellon Univ., 1996.
[42] G. Susman and R. Evered, “An Assessment of the Scientific Merits of Action Research,” Administrative Science Quarterly, vol. 23, no. 4, pp. 582-603, 1978.
[43] B. Kitchenham, S. Pfleeger, D. Hoaglin, K.E. Emam, and J. Rosenberg, “Preliminary Guidelines for Empirical Research in Software Engineering,” IEEE Trans. Software Eng., vol. 28, no. 8, pp. 721-734, Aug. 2002.
[44] R. Rapoport, “Three Dilemmas in Action Research,” Human Relations, vol. 23, no. 6, pp. 499-513, 1970.
[45] R. Baskerville and T. Wood-Harper, “Diversity in Information Systems Action Research Methods,” European J. Information Systems, vol. 7, no. 2, pp. 90-107, 1998.
[46] A.M. Pettigrew, “Longitudinal Field Research on Change: Theory and Practice,” Organization Science, vol. 1, no. 3, pp. 267-292, 1990.
[47] M.B. Miles and A.M. Huberman, Qualitative Data Analysis: An Expanded Sourcebook. Sage Publications, 1994.
[48] E.A. Weitzman and M.B. Miles, Computer Programs for Qualitative Data Analysis. Sage Publications, 1995.
[49] Y. Lincoln and E. Guba, Naturalistic Inquiry. Sage, 1985.
[50] J. Mason, Qualitative Researching, second ed. Sage, 2002.
[51] C.B. Seaman, “Qualitative Methods in Empirical Studies of Software Engineering,” IEEE Trans. Software Eng., vol. 25, no. 4, pp. 557-572, July/Aug. 1999.
[52] N.P. Napier, “Improving Practices in a Small Software Firm: An Ambidextrous Perspective,” doctoral dissertation, Computer Information Systems, Georgia State Univ., 2007.
[53] M. Kauppinen, S. Kujala, T. Aaltio, and L. Lehtola, “Introducing Requirements Engineering: How to Make a Cultural Change Happen in Practice,” Proc. Joint Int'l Conf. Requirements Eng., pp.43-51, 2002.
[54] B.W. Boehm, “Get Ready for Agile Methods, with Care,” Computer, vol. 35, no. 1, pp. 64-69, Jan. 2002.
[55] B.W. Boehm and R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed. Addison-Wesley, 2004.
[56] P. Checkland and J. Scholes, Soft Systems Methodology: A 30-Year Retrospective. John Wiley, 1999.

Index Terms:
Process implementation and change, qualitative process analysis, requirements engineering process, software management, software process.
Nannette P. Napier, Lars Mathiassen, Roy D. Johnson, "Combining Perceptions and Prescriptions in Requirements Engineering Process Assessment: An Industrial Case Study," IEEE Transactions on Software Engineering, vol. 35, no. 5, pp. 593-606, Sept.-Oct. 2009, doi:10.1109/TSE.2009.33
Usage of this product signifies your acceptance of the Terms of Use.