This Article 
 Bibliographic References 
 Add to: 
Testing Homogeneous Spreadsheet Grids with the "What You See Is What You Test" Methodology
June 2002 (vol. 28 no. 6)
pp. 576-594

Abstract—Although there has been recent research into ways to design environments that enable end users to create their own programs, little attention has been given to helping these end users systematically test their programs. To help address this need in spreadsheet systems—the most widely used type of end-user programming language—we previously introduced a visual approach to systematically testing individual cells in spreadsheet systems. However, the previous approach did not scale well in the presence of largely homogeneous grids, which introduce problems somewhat analogous to the array-testing problems of imperative programs. In this paper, we present two approaches to spreadsheet testing that explicitly support such grids. We present the algorithms, time complexities, and performance data comparing the two approaches. This is part of our continuing work to bring to end users at least some of the benefits of formalized notions of testing without requiring knowledge of testing beyond a naive level.

[1] A.V. Aho, R. Sethi, and J.D. Ullman, Compilers, Principles, Techniques and Tools.New York: Addison-Wesley, 1985.
[2] P. Brown and J. Gould, “An Experimental Study of People Creating Spreadsheets,” ACM Trans. Office Automation, vol. 5, no. 3, pp. 258-272, July 1987.
[3] M. Burnett and H. Gottfried, “Graphical Definitions: Expanding Spreadsheet Languages through Direct Manipulation and Gestures,” ACM Trans. Computer-Human Interaction, Vol. 5, No. 1, pp. 1–33, Mar. 1998.
[4] M. Burnett, A. Agrawal, and P. Zee, “Exception Handling in the Spreadsheet Paradigm,” IEEE Trans. Software Eng., vol. 26, no. 10, pp. 923-942, Oct. 2000.
[5] M. Burnett, J. Atwood, R. Djang, H. Gottfried, J. Reichwein, and S. Yang, “Forms/3: A First-Order Visual Language to Explore the Boundaries of the Spreadsheet Paradigm,” J. Functional Programming, pp. 155-206, Mar. 2001.
[6] M. Burnett, A. Sheretov, and G. Rothermel, “Scaling Up a `What You See is What You Test' Methodology to Spreadsheet Grids,” Proc. IEEE Symp. Visual Languages, pp. 30–37, Sept. 1999.
[7] E. Chi, J. Riedl, P. Barry, and J. Konstan, “Principles for Information Visualization Spreadsheets,” IEEE Computer Graphics and Applications, pp. 30–38, July/Aug. 1998.
[8] C. Cook, K. Rothermel, M. Burnett, T. Adams, G. Rothermel, A. Sheretov, F. Cort, and J. Reichwein, “Does a Visual `Testedness' Methodology Aid Debugging?” Technical Report TR #99-60-07, Oregon State Univ., Mar. 2001, .
[9] Coopers&Lybrand UK, “Spreadsheet Modelling in Financial Services Institutions,” http://www.planningobjects.comjungle1.htm , June 1997.
[10] R. Djang and M. Burnett, “Similarity Inheritance: A New Model of Inheritance for Spreadsheet VPLs,” Proc. 1998 IEEE Symp. Visual Languages, pp. 134-141, Sept. 1998.
[11] E. Duesterwald, R. Gupta, and M.L. Soffa, “Rigorous Data Flow Testing Through Output Influences,” Proc. Second Irvine Software Symp., pp. 131-145, Mar. 1992.
[12] P. Frankl and E.J. Weyuker, “An Applicable Family of Data Flow Testing Criteria,” IEEE Trans. Software Eng., vol. 14, no. 10, pp. 1,483–1,498, Oct. 1988.
[13] D. Galletta, D. Abraham, M. El Louadi, W. Lekse, Y. Pollalis, and J. Sampler, “An Empirical Study of Spreadsheet Error-Finding Performance,” Accounting, Management, and Information Technology, vol. 3, no. 2, pp. 79-95, 1993.
[14] D. Hamlet, B. Gifford, and B. Nikolik, “Exploring Dataflow Testing of Arrays,” Proc. Int'l Conf. Software Eng., pp. 118-129, May 1993.
[15] J.R. Horgan and S.A. London,“Dataflow coverage and the C language,” Proc. Fourth Symp. on Software Testing, Analysis and Verification,Victoria, British Columbia, Canada, Oct. 1991, pp. 87-97.
[16] J. Laski and B. Korel, “A Data Flow Oriented Program Testing Strategy,” IEEE Trans. Software Eng., vol. 9, no. 5, pp. 347-354, May 1993.
[17] J. Leopold and A. Ambler, “Keyboardless Visual Programming Using Voice, Handwriting, and Gesture,” Proc. IEEE Symp. Visual Languages, pp. 28–35, Sept. 1997.
[18] Microsoft Corporation, Microsoft Excel 4.0 User's Guide 1 and Microsoft Excel 4.0 Function Reference. 1992.
[19] B. Myers, “Graphical Techniques in a Spreadsheet for Specifying User Interfaces,” Proc. ACM Conf. Human Factors in Computing Systems, pp. 243–249, Apr./May 1991.
[20] B.A. Myers et al., "Garnet: Comprehensive Support for Graphical, Highly-Interactive User Interfaces," Computer, vol. 23, no. 11, pp. 71-85, 1990.
[21] R. Panko, “What We Know About Spreadsheet Errors,” J. End User Computing, pp. 15–21, 1998.
[22] S. Rapps and E.J. Weyuker, “Selecting Software Test Data Using Data Flow Information,” IEEE Trans. Software Eng., vol. 11, no. 4, pp. 367–375, Apr. 1985.
[23] J. Reichwein, G. Rothermel, and M. Burnett, “Slicing Spreadsheets: An Integrated Methodology for Spreadsheet Testing and Debugging,” Proc. Conf. Domain-Specific Languages, pp. 25–38, Oct. 1999.
[24] B. Ronen, R. Palley, and H. Lucas, “Spreadsheet Analysis and Design,” Comm. ACM, vol. 32, no. 1, pp. 84-93, Jan. 1989.
[25] G. Rothermel, M. Burnett, L. Li, C. DuPuis, and A. Sheretov, “A Methodology for Testing Spreadsheets,” ACM Trans. Software Eng. and Methodology, pp. 110-147, Jan. 2001.
[26] G. Rothermel, L. Li, and M. Burnett, “Testing Strategies for Form-Based Visual Programs,” Proc. Eighth Int'l Symp. Software Reliability Eng., pp. 96-107, Nov. 1997.
[27] G. Rothermel, L. Li, C. DuPuis, and M. Burnett, “What You See is What You Test: A Methodology for Testing Form-Based Visual Programs,” Proc. Int'l Conf. Software Eng., pp. 198–207, Apr. 1998.
[28] K. Rothermel, C. Cook, M. Burnett, J. Schonfeld, T. Green, and G. Rothermel, “An Empirical Evaluation of a Methodology for Testing Spreadsheets,” Proc. Int'l Conf. Software Eng., pp. 230-239, June 2000.
[29] T. Smedley, P. Cox, and S. Byrne, “Expanding the Utility of Spreadsheets through the Integration of Visual Programming and User Interface Objects,” Proc. ACM Workshop Advanced Visual Interfaces, pp. 148–155, May 1996.
[30] T. Teo and M. Tan, “Quantitative and Qualitative Errors in Spreadsheet Development,” Proc. 13th Hawaii Int'l Conf. System Sciences, Part 3, vol. 3, pp. 149-155, Jan. 1997.
[31] G. Wang and A. Ambler, “Solving Display-Based Problems,” Proc. IEEE Symp. Visual Languages, pp. 122–129, Sept. 1996.
[32] N. Wilde and C. Lewis, “Spreadsheet-Based Interactive Graphics: From Prototype to Tool,” Proc. ACM Conf. Human Factors in Computing Systems, pp. 153–159, Apr. 1990.

Index Terms:
Software testing, spreadsheets, visual programming.
Margaret Burnett, Andrei Sheretov, Bing Ren, Gregg Rothermel, "Testing Homogeneous Spreadsheet Grids with the "What You See Is What You Test" Methodology," IEEE Transactions on Software Engineering, vol. 28, no. 6, pp. 576-594, June 2002, doi:10.1109/TSE.2002.1010060
Usage of this product signifies your acceptance of the Terms of Use.