Abstract—Chris Morris writes a letter to the editor in response to the Voice of Evidence column “Does Involving Users in Software Development Really Influence System Success?” from the Nov./Dec. 2013 issue of IEEE Software in which he discusses the correlation between user and involvement and project success.
Keywords—letter to the editor; Chris Morris; user involvement; voice of evidence; software; engineering; development; IEEE
WITH A LIMITED project budget, how much should be allocated to user participation and involvement? Ulrike Abelein, Helen Sharp, and Barbara Paech show a positive correlation between user participation and involvement and project success (Voice of Evidence, IEEE Software, Nov./Dec. 2013, pp. 17–23), suggesting that the right answer is more than is typical today. However, they stretch the evidence when they conclude, “We encourage all practitioners to increase user participation and involvement in all phases of their software development projects as much as possible.”
Importantly, the authors show that increased levels of user involvement are a significant predictor of success. However, they're guilty of the post hoc fallacy when they conclude that “the positive effects will … improve the resulting system from a quality perspective.”
This is probably often the case, but the evidence they present doesn't show it, and another factor springs to mind. If users decline to invest in a project, then the project may be misconceived or stakeholder management may have failed at an early stage. Most projects with low initial buy-in should be terminated for one of these reasons. Obtaining increased user input would rescue only some of them. However, some projects should continue even with a low level of user involvement, including some that are based on a vision that's constructive yet is also very disruptive to current methods of work. In that situation, successful user participation is hard to achieve.
Intriguingly, a minority of the studies they reviewed showed a negative correlation between user participation and involvement and project success. Maybe these are outliers. Alternatively, many readers will have experienced projects with unmanaged stakeholder conflict, in which case increased user input can bring the development effort to its knees.
There's another possible explanation of this negative correlation. Certainly, some projects fail because design isn't informed by users, but others because of a lack of professional input. Many readers have participated in conversations like this:
We must listen to users to understand the context of use, as well as to the customers who pay for the project to understand the business goals. But neither group is qualified to design software features—that job requires specific training and experience.
I'd like to thank the authors for a thought-provoking article. But demonstrating a correlation is the start, not the end, of the discussion.
Member, IEEE Computer Society Science & Technology Facilities Council, UK