The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - January/February (2011 vol.37)
pp: 95-108
Lutz Prechelt , Freie Universität Berlin, Berlin
ABSTRACT
Background: For developing Web-based applications, there exist several competing and widely used technological platforms (consisting of a programming language, framework(s), components, and tools), each with an accompanying development culture and style. Research question: Do Web development projects exhibit emergent process or product properties that are characteristic and consistent within a platform, but show relevant substantial differences across platforms or do team-to-team individual differences outweigh such differences, if any? Such a property could be positive (i.e., a platform advantage), negative, or neutral, and it might be unobvious which is which. Method: In a nonrandomized, controlled experiment, framed as a public contest called “Plat_Forms,” top-class teams of three professional programmers competed to implement the same requirements for a Web-based application within 30 hours. Three different platforms (Java EE, PHP, or Perl) were used by three teams each. We compare the resulting nine products and process records along many dimensions, both external (usability, functionality, reliability, security, etc.) and internal (size, structure, modifiability, etc.). Results: The various results obtained cover a wide spectrum: First, there are results that many people would have called “obvious” or “well known,” say, that Perl solutions tend to be more compact than Java solutions. Second, there are results that contradict conventional wisdom, say, that our PHP solutions appear in some (but not all) respects to be actually at least as secure as the others. Finally, one result makes a statement we have not seen discussed previously: Along several dimensions, the amount of within-platform variation between the teams tends to be smaller for PHP than for the other platforms. Conclusion: The results suggest that substantial characteristic platform differences do indeed exist in some dimensions, but possibly not in others.
INDEX TERMS
Emergent properties, usability, functionality, reliability, security, product size, design structure, modifiability, Java, PHP, Perl.
CITATION
Lutz Prechelt, "Plat_Forms: A Web Development Platform Comparison by an Exploratory Experiment Searching for Emergent Platform Properties", IEEE Transactions on Software Engineering, vol.37, no. 1, pp. 95-108, January/February 2011, doi:10.1109/TSE.2010.22
REFERENCES
[1] B.C.D. Anda, D.I.K. Sjøberg, and A. Mockus, "Variability and Reproducibility in Software Engineering: A Study of Four Companies that Developed the Same System," IEEE Trans. Software Eng., vol. 35, no. 3, pp. 407-429, May/June 2009.
[2] V.R. Basili, R.W. Selby, and D.H. Hutchens, "Experimentation in Software Engineering," IEEE Trans. Software Eng., vol. 12, no. 7, pp. 733-743, July 1986.
[3] V.R. Basili, F. Shull, and F. Lanubile, "Building Knowledge through Families of Experiments," IEEE Trans. Software Eng., vol. 25, no. 4, pp. 456-473, July/Aug. 1999.
[4] L. Christensen, Experimental Methodology, 10th ed. Allyn and Bacon, 2006.
[5] A. Cockburn, Writing Effective Use Cases. Addison-Wesley, 2001.
[6] B. Curtis, "By the Way, Did Anyone Study Any Real Programmers?" Proc. First Workshop Empirical Studies of Programmers, pp. 256-262, 1986.
[7] A. Green and B. Askins, "A Rails/Django Comparison," http://docs.google.comView?docid=dcn8282p_1hg4sr9 , Sept. 2006.
[8] J. Highsmith and A. Cockburn, "Agile Software Development: The Business of Innovation," IEEE Software, vol. 18, no. 5, pp. 120-122, Sept./Oct. 2001.
[9] V.B. Kampenes, T. Dybå, J.E. Hannay, and D.I.K. Sjøberg, "A Systematic Review of Quasi-Experiments in Software Engineering," Information and Software Technology, vol. 51, no. 1, pp. 71-82, Jan. 2009.
[10] S. Kelly, "Better Web App Development," Quicktime Video, http://oodt.jpl.nasa.govbetter-web-app.mov , Feb. 2006.
[11] M. Kunze and H. Schulz, "Gute Nachbarschaft: C't Lädt zum Datenbank-Contest Ein," C't, 20/2005:156, 2005. see also http://www.heise.de/ct/05/20156/, english translation on http:// firebird.sourceforge.net/connect ct-dbContest.html, overview on http://www.heise.de/ctdbcontest/, results in issue 13, 2006.
[12] D. Müller-Solger, "Wettbewerb der Portale," IT Management, vol. 9, pp. 47-48, 2006.
[13] Usability Inspection Methods, J. Nielsen and R.L. Mack, eds. John Wiley and Sons, 1994.
[14] J.M. Perpich, D.E. Perry, A.A. Porter, L.G. Votta, and M.W. Wade, "Anywhere, Anytime Code Inspections: Using the Web to Remove Inspection Bottlenecks in Large-Scale Software Development," Proc. 19th Int'l Conf. Software Eng., pp. 14-21, 1997.
[15] K. Popper, The Logic of Scientific Discovery. Routledge, 1959/2002.
[16] A.A. Porter, H.P. Siy, C.A. Toman, and L.G. Votta, "An Experiment to Assess the Cost-Benefits of Code Inspections in Large Scale Software Development," IEEE Trans. Software Eng., vol. 23, no. 6, pp. 329-346, June 1997.
[17] L. Prechelt, "The 28:1 Grant/Sackman Legend Is Misleading, or: How Large Is Interpersonal Variation Really?" Technical Report 1999-18, Fakultät für Informatik, Univ. Karlsruhe, ftp.ira.uka.de, Dec. 1999.
[18] L. Prechelt, "An Empirical Comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a Search/String-Processing Program," Technical Report 2000-5, Fakultät für Informatik, Univ. Karlsruhe, ftp.ira.uka.de, Mar. 2000.
[19] L. Prechelt, "An Empirical Comparison of Seven Programming Languages," Computer, vol. 33, no. 10, pp. 23-29, Oct. 2000.
[20] L. Prechelt, Kontrollierte Experimente in der Softwaretechnik—Potenzial und Methodik. Springer-Verlag, 2001.
[21] L. Prechelt, "Plat_Forms—A Contest: The Web Development Platform Comparison," Technical Report TR-B-06-11, Freie Univ. Berlin, Institut für Informatik, Germany, Oct. 2006.
[22] L. Prechelt, "Plat_Forms 2007 Task: PbT," Technical Report TR-B-07-03, Freie Universität Berlin, Institut für Informatik, Jan. 2007.
[23] L. Prechelt, "Plat_Forms 2007: The Web Development Platform Comparison—Evaluation and Results," Technical Report TR-B-07-10, Freie Univ. Berlin, Institut für Informatik, www.plat-forms. org, Apr. 2007.
[24] R Development Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, 2006.
[25] D.I.K. Sjøberg, J.E. Hannay, O. Hansen, V.B. Kampenes, A. Karahasanovic, N.-K. Liborg, and A.C. Rekdal, "A Survey of Controlled Experiments in Software Engineering," IEEE Trans. Software Eng., vol. 31, no. 9, pp. 733-753, Sept. 2005.
[26] R.K. Yin, Case Study Research: Design and Methods. Sage, 2003.
6 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool