This Article 
 Bibliographic References 
 Add to: 
Comparing Partition and Random Testing via Majorization and Schur Functions
January 2003 (vol. 29 no. 1)
pp. 88-94

Abstract—The comparison of partition and random sampling methods for software testing has received considerable attention in the literature. A standard criterion for comparisons between random and partition testing based on their expected efficacy in program debugging is the probability of detecting at least one failure causing input in the program's domain. We investigate the relative effectiveness of partition testing versus random testing through the powerful mathematical technique of majorization, which was introduced by Hardy et al. The tools of majorization and the concepts of Schur (convex and concave) functions enable us to derive general conditions under which partition testing is superior to random testing and, consequently, to give further insights into the value of partition testing strategies.

[1] P.G. Bishop and F.D. Pullen, “PODS Revisited—A Study of Software Failure Behaviour,” Proc. 18th Int'l Symp. Fault-Tolerant Computing, 1988.
[2] P.G. Bishop, “The Variation of Software Survival Time for Different Operational Input Profiles,” Proc. IEEE Int'l Symp. Fault-Tolerant Computing FTCS-23, pp. 98-107, June 1993.
[3] P.J. Boland, E. El-Neweihi, and F. Proschan, “Applications of the Hazard Rate Ordering in Reliability and Order Statistics,” J. Applied Probability, vol. 31, no. 1, pp. 180-192, 1994.
[4] T. Chen and Y. Yu, “On the Relationship Between Partition and Random Testing,” IEEE Trans. Software Eng., vol. 20, no. 12, pp. 977-980, Dec. 1994.
[5] T. Chen and Y. Yu, “On the Expected Number of Failures Detected by Subdomain Testing and Random Testing,” IEEE Trans. Software Eng., vol. 22, no. 2, pp. 109-119, Feb. 1996.
[6] J. Duran and S. Ntafos, “An Evaluation of Random Testing,” IEEE Trans. Software Eng., vol. 10, pp. 438-444, 1984.
[7] N. Fenton and N. Ohlsson, Quantitative Analysis of Faults and Failures in a Complex Software System IEEE Trans. Software Eng., vol. 26, no. 8, pp. 797-814, Aug. 2000.
[8] G.B. Finelli, “Results of Software Error-Data Experiments,” Proc. AIAA/AHS/ASEE Aircraft Design, Systems and Operations Conf., Sept. 1988.
[9] P. Frankl et al., "Evaluating Testing Methods by Delivered Reliability," IEEE Trans. Software Eng., Aug. 1998, pp. 586-602.
[10] P.G. Frankl and E.J. Weyuker,“A formal analysis of the fault-detecting ability of testing methods,” IEEE Transactions on Software Engineering, vol. 19, no. 3, Mar. 1993, pp. 202-213,
[11] W. Gutjhar, "Partition vs. Random Testing: The Influence of Uncertainty," IEEE Trans. Software Eng., Vol. 25, No. 5, 1999, pp. 661-674.
[12] G.H. Hardy, J.E. Littlewood, and G. Polya, Inequalities. London: Cambridge Univ. Press, 1952.
[13] D. Hamlet and R. Taylor, "Partition Testing Does Not Inspire Confidence," IEEE Trans. Software Eng., vol. 16, pp. 1,402-1,412, Dec. 1990.
[14] R.M. Hierons and M.P. Wiper, “Estimation of Failure Rate using Random and Partition Testing,” Software Testing, Verification, and Reliability, vol. 7, pp. 153-164, 1997.
[15] B. Jeng and E.J. Weyuker, “Some Observations on Partition Testing,” Proc. ACM SIGSOFT Third Symp. Software Testing, Analysis and Verification (TAV3), pp. 38-47, 1989.
[16] A.W. Marshall and I. Olkin, Inequalities: Theory of Majorization and Its Applications. New York: Academic Press, 1979.
[17] H. Hecht and P. Crane, “Rare Conditions and Their Effect on Software Failures,” Proc. Ann. Reliability and Maintainability Symp., pp. 334-337, Jan. 1994.
[18] H. Hecht, M. Hecht, and D. Wallace, “Toward More Effective Testing for High Assurance Systems,” Proc. Second IEEE High Assurance Systems Eng. Workshop, Aug. 1997.
[19] D.M. Nicol, R. Simha, and D. Towsley, “Static Assignment of Stochastic Tasks Using Majorization,” IEEE Trans. Computers, vol. 45, no. 6, pp. 730-740, 1996.
[20] V.R. Prasad and M. Raghavachari, “Optimal Allocation of Interchangeable Components in a Series-Parallel System,” IEEE Trans. Reliability, vol. 47, no. 3, pp. 255-260, 1998.
[21] F. Proschan, “Applications of Majorization and Schur Functions in Reliability and Life Testing,” Reliability and Fault Tree Analysis, pp. 237-258, 1975.
[22] S. Rapps and E.J. Weyuker, “Selecting Software Test Data Using Data Flow Information,” IEEE Trans. Software Eng., vol. 11, no. 4, pp. 367–375, Apr. 1985.
[23] I. Schur, “Uber eine Klasse von Mittelbildungen mit Anwendungen die Determinanten,” Theorie Sitzungsber, Berlin, Math. Gesellschaft, vol. 22, pp. 9-20, 1923.
[24] M. Shaked and J.G. Shanthikumar, “Optimal Allocation of Resources to Nodes of Parallel and Series Systems,” Advances in Applied Probability, vol. 24, no. 4, pp. 894-914, 1992.
[25] E.J. Weyuker and B. Jeng,“Analyzing partition testing strategies,” IEEE Trans. Software Engineering, vol. 17, pp. 703-711, 1991.
[26] E.J. Weyuker and T.J. Ostrand, “Theories of Program Testing and the Application of Revealing Subdomains,” IEEE Trans. Software Eng., vol. 6, pp. 236-245, May 1980.

Index Terms:
Partition testing, random testing, software debugging, majorization, Schur functions.
Philip J. Boland, Harshinder Singh, Bojan Cukic, "Comparing Partition and Random Testing via Majorization and Schur Functions," IEEE Transactions on Software Engineering, vol. 29, no. 1, pp. 88-94, Jan. 2003, doi:10.1109/TSE.2003.1166591
Usage of this product signifies your acceptance of the Terms of Use.