This Article 
 Bibliographic References 
 Add to: 
Marginal Maximum Entropy Partitioning Yields Asymptotically Consistent Probability Density Functions
April 2001 (vol. 23 no. 4)
pp. 414-417

Abstract—The marginal maximum entropy criterion has been used to guide recursive partitioning of a continuous sample space. Although the criterion has been successfully applied in pattern discovery applications, its theoretical justification has not been clearly addressed. In this paper, it is shown that the basic marginal maximum entropy partitioning algorithm yields asymptotically consistent density estimates. This result supports the use of the marginal maximum entropy criterion in pattern discovery and implies that an optimal classifier can be constructed.

[1] A.P.A. daSilva, V.H. Quintana, and G.K.H. Pang, “Solving Data Acquisition and Processing Problems in Power Systems Using a Pattern Analysis Approach,” IEE Proc. C, vol. 138, no. 4, 1991.
[2] D.K.Y. Chiu, B. Cheung, and A.K.C. Wong, “Information Synthesis Based on Hierarchical Maximum Entropy Discretization,” J. Experimental and Theoretical Artificial Intelligence, vol. 2, pp. 117-129, 1990.
[3] A.K.C. Wong, D.K.Y. Chiu, and B. Cheung, “Information Discovery through Hierarchical Maximum Entropy Discretization,” Knowledge Discovery in Databases, G. Piatetsky-Shapiro and W.J. Frawley, eds., pp. 125-140, AAAI/MIT Press, 1987.
[4] T. Chau and A.K.C. Wong, “Pattern Discovery by Residual Analysis and Recursive Partitioning,” IEEE Trans. Data and Knowledge Eng., vol. 11, no. 6, pp. 833-852, Nov./Dec. 1999.
[5] E.G. Henrichon and K.S. Fu, “A Nonparametric Partitioning Procedure for Pattern Classification,” IEEE Trans. Computers, vol. 18, no. 7, pp. 614-624, July 1969.
[6] L. Gordon and R.A. Olshen, “Asymptotically Efficient Solutions to the Classification Problem,” The Annals of Statistics, vol. 6, no. 3, pp. 515-533, 1978.
[7] W.S. Meisel and D.A. Michalopoulos, “A Partitioning Algorithm with Application in Pattern Classification and the Optimization of Decision Trees,” IEEE Trans. Computers, vol. 22, no. 1, pp. 93-103, Jan. 1973.
[8] J.H. Friedman, “A Recursive Partitioning Decision Rule for Nonparametric Classification,” IEEE Trans. Computers, pp. 404-408, 1977.
[9] T.W. Anderson, “Some Nonparametric Multivariate Procedures Based on Statistically Equivalent Blocks,” Multivariate Analysis, P.R. Krishnaiah, ed., pp. 5-28, 1966.
[10] A. Papoulis, Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 1991.
[11] D. W. Scott, Multivariate Density Estimation. John Wiley and Sons, 1992.
[12] R.O. Duda and P.H. Hart, Pattern Classification and Scene Analysis. John Wiley and Sons, 1973.
[13] R.E. Haskell and A Noui-Mehidi, “Design of Hierarchical Classifiers,” Computing in the 90's: The First Great Lakes Computer Science Conf. Proc., pp. 118-124, 1991.
[14] J.A. Songquist, E.L. Baker, and J.N. Morgan, Searching for Structure. Survey Research Center, 1973.
[15] S.K. Murthy, “Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey,” Data Mining and Knowledge Discovery, pp. 345-389, 1998.
[16] M. Lascurain, “On Maximum Entropy Discretization and its Applications in Pattern Recognition,” Ph.D. thesis, Systems Design Eng., Univ. of Waterloo, Waterloo, Ontario, Canada, 1983.
[17] C.E. Shannon, “Mathematical Theory of Communication,” Bell Systems Technical J., vol. 27, no. 3, pp. 379-423, 1948.
[18] E.T. Jaynes, “Information Theory and Statistical Mechanics I,” Physical Rev., vol. 106, no. 1, pp. 620-630, 1957.
[19] H.O. Lancaster, The Chi-Squared Distribution. John Wiley and Sons, 1969.
[20] R.A. Christensen, Log-Linear Models. Springer-Verlag, 1990.
[21] A. Agresti, Categorical Data Analysis. New York: Wiley, 1990.

Index Terms:
Marginal maximum entropy, recursive partitioning, pattern discovery, asymptotic optimal classification.
Tom Chau, "Marginal Maximum Entropy Partitioning Yields Asymptotically Consistent Probability Density Functions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 4, pp. 414-417, April 2001, doi:10.1109/34.917576
Usage of this product signifies your acceptance of the Terms of Use.