The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - September/October (2011 vol.37)
pp: 616-634
Xin Feng , University of Limerick, Limerick and United International College, Zhuhai, Guangdong
David Lorge Parnas , University of Limerick, Limerick
T.H. Tse , The University of Hong Kong, Hong Kong
Tony O'Callaghan , University of Limerick, Limerick
ABSTRACT
Tabular expressions have been proposed as a notation to document mathematically precise but readable software specifications. One of the many roles of such documentation is to guide testers. This paper 1) explores the application of four testing strategies (the partition strategy, decision table-based testing, the basic meaningful impact strategy, and fault-based testing) to tabular expression-based specifications, and 2) compares the strategies on a mathematical basis through formal and precise definitions of the subsumption relationship. We also compare these strategies through experimental studies. These results will help researchers improve current methods and will enable testers to select appropriate testing strategies for tabular expression-based specifications.
INDEX TERMS
Tabular expression, test case constraint, subsume, unconditionally subsume, conditionally subsume.
CITATION
Xin Feng, David Lorge Parnas, T.H. Tse, Tony O'Callaghan, "A Comparison of Tabular Expression-Based Testing Strategies", IEEE Transactions on Software Engineering, vol.37, no. 5, pp. 616-634, September/October 2011, doi:10.1109/TSE.2011.78
REFERENCES
[1] H. Agrawal, R.A. De Millo, B. Hathaway, W.M. Hsu, W. Hsu, E.W. Krawser, R.J. Martin, A.P. Mathur, and E. Spafford, "Design of Mutant Operators for the C Programming Language," Technical Report SERC-TR-41-P, Software Eng. Research Center, Dept. of Computer Sciences, Purdue Univ., 1989.
[2] T.A. Alspaugh, S.R. Faulk, K.H. Britton, R.A. Parker, and D.L. Parnas, "Software Requirements for the A-7E Aircraft," Technical Report NRL/FR/5530-92-9194, Naval Research Lab, 1992.
[3] R.L. Baber, D.L. Parnas, S.A. Vilkomir, P. Harrison, and T. O'Connor, "Disciplined Methods of Software Specification: A Case Study," Proc. Int'l Conf. Information Technology: Coding and Computing, vol. 2, pp. 428-437, 2005.
[4] V.R. Basili and R.W. Selby, "Comparing the Effectiveness of Software Testing Strategies," IEEE Trans. Software Eng., vol. 13, no. 12, pp. 1278-1296, Dec. 1987.
[5] B. Beizer, Software Testing Techniques. Van Nostrand Reinhold, 1990.
[6] T.Y. Chen, F.-C. Kuo, and R.G. Merkel, "On the Statistical Properties of Testing Effectiveness Measures," J. Systems and Software, vol. 79, pp. 591-601, 2006.
[7] T.Y. Chen and M.F. Lau, "Test Case Selection Strategies Based on Boolean Specifications," Software Testing, Verification and Reliability, vol. 11, no. 3, pp. 165-180, 2001.
[8] T.Y. Chen, M.F. Lau, and Y.T. Yu, "MUMCUT: A Fault-Based Strategy for Testing Boolean Specifications," Proc. Sixth Asia-Pacific Software Eng. Conf., pp. 606-613, 1999.
[9] T.Y. Chen and Y.T. Yu, "On the Expected Number of Failures Detected by Subdomain Testing and Random Testing," IEEE Trans. Software Eng., vol. 22, no. 2, pp. 109-119, Feb. 1996.
[10] J.J. Chilenski and S.P. Miller, "Applicability of Modified Condition/Decision Coverage to Software Testing," Software Eng. J., vol. 9, no. 5, pp. 193-200, 1994.
[11] M. Clermont and D.L. Parnas, "Using Information about Functions in Selecting Test Cases," ACM SIGSOFT Software Eng. Notes, vol. 30, no. 4, pp. 1-7, 2005.
[12] R.A. DeMillo and A.J. Offutt, "Experimental Results from an Automatic Test Case Generator," J. ACM Trans. Software Eng. and Methodology, vol. 2, no. 2, pp. 109-127, 1993.
[13] X. Feng, "MIST: Towards a Minimum Set of Test Cases," PhD thesis, The Univ. of Hong Kong, 2002.
[14] X. Feng, S. Marr, and T. O'Callaghan, "ESTP: An Experimental Software Testing Platform," Proc. Testing: Academic and Industrial Conf.: Practice And Research Techniques, pp. 59-63, 2008.
[15] P.G. Frankl and E.J. Weyuker, "A Formal Analysis of the Fault-Detecting Ability of Testing Methods," IEEE Trans. Software Eng., vol. 19, no. 3, pp. 202-213, Mar. 1993.
[16] M. Grochtmann and K. Grimm, "Classification Trees for Partition Testing," Software Testing, Verification and Reliability, vol. 3, no. 2, pp. 63-82, 1993.
[17] P.R. Halmos, Naive Set Theory. Van Nostrand, 1960, reprinted by Springer, 1974.
[18] K.L. Heninger, "Specifying Software Requirements for Complex Systems: New Techniques and Their Application," IEEE Trans. Software Eng., vol. 6, no. 1, pp. 2-13, Jan. 1980.
[19] W.E. Howden, "Weak Mutation Testing and Completeness of Test Sets," IEEE Trans. Software Eng., vol. 8, no. 4, pp. 371-379, July 1982.
[20] R. Janicki and R. Khedri, "On a Formal Semantics of Tabular Expressions," Science of Computer Programming, vol. 39, nos. 2/3, pp. 189-213, 2001.
[21] R. Janicki, D.L. Parnas, and J.I. Zucker, "Tabular Representations in Relational Documents," Relational Methods in Computer Science, C. Brink, W. Kahl, and G. Schmidt, eds., pp. 184-196, Springer, 1997.
[22] R. Janicki and A. Wassyng, "Tabular Expressions and Their Relational Semantics," Fundamenta Informaticae, vol. 67, no. 4, pp. 343-370, 2005.
[23] P.C. Jorgensen, Software Testing: A Craftsman's Approach. Auerbach Publications, 2008.
[24] G. Kaminski, G. Williams, and P. Ammann, "Reconciling Perspectives of Software Logic Testing," Software Testing, Verification and Reliability, vol. 18, no. 3, pp. 149-188, 2008.
[25] D.R. Kuhn, "Fault Classes and Error Detection Capability of Specification-Based Testing," ACM Trans. Software Eng. and Methodology, vol. 8, no. 4, pp. 411-424, 1999.
[26] M.F. Lau and Y.T. Yu, "On the Relationships of Faults for Boolean Specification Based Testing," Proc. Australian Software Eng. Conf., pp. 21-28, 2001.
[27] M.F. Lau and Y.T. Yu, "An Extended Fault Class Hierarchy for Specification-Based Testing," ACM Trans. Software Eng. and Methodology, vol. 14, no. 3, pp. 247-276, 2005.
[28] S. Liu, "Generating Test Cases from Software Documentation," MEng thesis, Dept. of Electrical and Computer Eng., McMaster Univ., 2001.
[29] L.J. Morell, "A Theory of Fault-Based Testing," IEEE Trans. Software Eng., vol. 16, no. 8, pp. 844-857, Aug. 1990.
[30] G.J. Myers, The Art of Software Testing. Wiley, 1979.
[31] V. Okun, P.E. Black, and Y. Yesha, "Comparison of Fault Classes in Specification-Based Testing," Information and Software Technology, vol. 46, no. 8, pp. 525-533, 2004.
[32] T.J. Ostrand and M.J. Balcer, "The Category-Partition Method for Specifying and Generating Functional Tests," Comm. ACM, vol. 31, no. 6, pp. 676-686, 1988.
[33] D.L. Parnas, "Tabular Representation of Relations," Technical Report 260, McMaster Univ., 1992.
[34] D.L. Parnas, "Inspection of Safety-Critical Software Using Program-Function Tables," Proc. Int'l Federation Information Processing Congress, vol. 3, pp. 270-277, 1994.
[35] D.L. Parnas, G.J.K. Asmis, and J. Madey, "Assessment of Safety-Critical Software in Nuclear Power Plants," Nuclear Safety, vol. 32, no. 2, pp. 189-198, 1991.
[36] D.L. Parnas and J. Madey, "Functional Documents for Computer Systems," Science of Computer Programming, vol. 25, no. 1, pp. 41-61, 1995.
[37] D.L. Parnas, J. Madey, and M. Iglewski, "Precise Documentation of Well-Structured Programs," IEEE Trans. Software Eng., vol. 20, no. 12, pp. 948-976, Dec. 1994.
[38] D.K. Peters and D.L. Parnas, "Using Test Oracles Generated from Program Documentation," IEEE Trans. Software Eng., vol. 24, no. 3, pp. 161-173, Mar. 1998.
[39] C. Quinn, S. Vilkomir, D.L. Parnas, and S. Kostic, "Specification of Software Component Requirements Using the Trace Function Method," Proc. Int'l Conf. Software Eng. Advances, 2006.
[40] E. Sekerinski, "Exploring Tabular Verification and Refinement," Formal Aspects of Computing, vol. 15, nos. 2/3, pp. 215-236, 2003.
[41] H. Shen, "Implementation of Table Inversion Algorithms," MEng thesis, Dept. of Electrical and Computer Eng., McMaster Univ., 1995.
[42] C.-A. Sun, Y. Dong, R. Lai, K.Y. Sim, and T.Y. Chen, "Analyzing and Extending MUMCUT for Fault-Based Testing of General Boolean Expressions," Proc. Sixth IEEE Int'l Conf. Computer and Information Technology, pp. 184-189, 2006.
[43] K.-C. Tai, "Theory of Fault-Based Predicate Testing for Computer Programs," IEEE Trans. Software Eng., vol. 22, no. 8, pp. 552-562, Aug. 1996.
[44] A.J. van Schouwen, D.L. Parnas, and J. Madey, "Documentation of Requirements for Computer Systems," Proc. First IEEE Int'l Symp. Requirements Eng., pp. 198-207, 1993.
[45] E.J. Weyuker, "More Experience with Data Flow Testing," IEEE Trans. Software Eng., vol. 19, no. 9, pp. 912-919, Sept. 1993.
[46] E.J. Weyuker, T. Goradia, and A. Singh, "Automatically Generating Test Data from a Boolean Specification," IEEE Trans. Software Eng., vol. 20, no. 5, pp. 353-363, May 1994.
[47] E.J. Weyuker and B. Jeng, "Analyzing Partition Testing Strategies," IEEE Trans. Software Eng., vol. 17, no. 7, pp. 703-711, July 1991.
[48] Y.T. Yu and M.F. Lau, "A Comparison of MC/DC, MUMCUT and Several Other Coverage Criteria for Logical Decisions," J. Systems and Software, vol. 79, no. 5, pp. 577-590, 2006.
[49] J.I. Zucker, "Transformations of Normal and Inverted Function Tables," Formal Aspects of Computing, vol. 8, no. 6, pp. 679-705, 1996.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool