This Article 
 Bibliographic References 
 Add to: 
An Empirical Study of Open-Source and Closed-Source Software Products
April 2004 (vol. 30 no. 4)
pp. 246-256
Giancarlo Succi, IEEE Computer Society
Armin Eberlein, IEEE Computer Society
This paper describes an empirical study of open-source and closed-source software projects. The motivation for this research is to quantitatively investigate common perceptions about open-source projects, and to validate these perceptions through an empirical study. This paper investigates the hypothesis that open-source software grows more quickly, but does not find evidence to support this. The project growth is similar for all the projects in the analysis, indicating that other factors may limit growth. The hypothesis that creativity is more prevalent in open-source software is also examined, and evidence to support this hypothesis is found using the metric of functions added over time. The concept of open-source projects succeeding because of their simplicity is not supported by the analysis, nor is the hypothesis of open-source projects being more modular. However, the belief that defects are found and fixed more rapidly in open-source projects is supported by an analysis of the functions modified. The paper finds support for two of the five common beliefs and concludes that, when implementing or switching to the open-source development model, practitioners should ensure that an appropriate metrics collection strategy is in place to verify the perceived benefits.

[1] C. Cook and A. Roesch, Real-Time Software Metrics J. Systems and Software, vol. 24, pp. 223-237, 1994.
[2] J. Dalle and N. Jullien, Windows vs. Linux: Some Explorations into the Economics of Free Software Proc. Acts of SSII Conf., 2000.
[3] G. Gall, M. Jazayeri, R.R. Klosch, and G. Trausmuth, Software Evolution Observations Based on Product Release History Proc. Int'l Conf. Software Maintenance, 1997.
[4] M.W. Godfrey and Q. Tu, Evolution in Open Source Software: A Case Study Proc. Int'l Conf. Software Metrics, 2000.
[5] C.F. Kemerer and S. Slaughter, An Empirical Approach to Studying Software Evolution IEEE Trans. Software Eng., vol. 25, no. 4, pp. 493-509, July/Aug. 1999.
[6] M.M. Lehman, J.F. Ramil, and P.D. Wernick, “Metrics and Laws of Software Evolution—The Nineties View,” Proc. Fourth Int'l Software Metrics Symp. (Metrics '97), pp. 20-32, 1997.
[7] A. Mockus, R.T. Fielding, and J. Herbsleb, "A Case Study of Open Source Software Development: The Apache Server," Proc. 22nd Int'l Conf. Software Eng. (ICSE 2000), ACM Press, 2000, pp. 263-272.
[8] Mozilla, Our Mission http://www.mozilla.orgmission.html, Aug. 2003.
[9] T. O'Reilly, Lessons From Open-Source Software Development Comm. ACM, vol. 42, no. 4, pp. 32-37, 1999.
[10] J.W. Paulson, An Empirical Study on the Growth of Open Source and Commercial Software Products master's thesis, Dept. of Electrical and Computer Eng., Univ. of Calgary, Alberta, 2001.
[11] K. Pearson, Mathematical Contributions to the Theory of Evolution. III. Regression, Heredity and Panmixia Philosophical Trans. Royal Soc. of London, 1896.
[12] E.S. Raymond, The Cathedral and the Bazaar http://www. cathedral-bazaar/, Aug. 2003.
[13] S. Wheeler, Open Source Software opensource.pdf, Aug. 2003.
[14] D. Wheeler, Why Open Source Software/Free Software (OSS/FS)? Look at the Numbers! http://www.dwheeler.comoss_fs_why.html, Aug. 2003.

Index Terms:
Open source, software metrics, empirical studies, software growth models.
James W. Paulson, Giancarlo Succi, Armin Eberlein, "An Empirical Study of Open-Source and Closed-Source Software Products," IEEE Transactions on Software Engineering, vol. 30, no. 4, pp. 246-256, April 2004, doi:10.1109/TSE.2004.1274044
Usage of this product signifies your acceptance of the Terms of Use.