This Article 
 Bibliographic References 
 Add to: 
High-Order Pattern Discovery from Discrete-Valued Data
November-December 1997 (vol. 9 no. 6)
pp. 877-893

Abstract—To uncover qualitative and quantitative patterns in a data set is a challenging task for research in the area of machine learning and data analysis. Due to the complexity of real-world data, high-order (polythetic) patterns or event associations, in addition to first-order class-dependent relationships, have to be acquired. Once the patterns of different orders are found, they should be represented in a form appropriate for further analysis and interpretation. In this paper, we propose a novel method to discover qualitative and quantitative patterns (or event associations) inherent in a data set. It uses the adjusted residual analysis in statistics to test the significance of the occurrence of a pattern candidate against its expectation. To avoid exhaustive search of all possible combinations of primary events, techniques of eliminating the impossible pattern candidates are developed. The detected patterns of different orders are then represented in an attributed hypergraph which is lucid for pattern interpretation and analysis. Test results on artificial and real-world data are discussed toward the end of the paper.

[1] R. Agrawal, S. Ghosh, T. Imielinski, B. Iyer, and A. Swami, “An Interval Classifier for Database Mining Applications,” Proc. 18th Conf. Very Large Databases, pp. 560–573, 1992.
[2] C. Berge, Hypergraph: Combinatorics of Finite Sets. NorthHolland, 1989.
[3] K.C.C. Chan, "Induction Learning in the Presence of Uncertainty," PhD dissertation, Dept. of Systems Design Engineering, Univ. of Waterloo, Ontario, Canada, 1989.
[4] K.C.C. Chan and A.K.C. Wong,“APACS: a system for automated pattern analysis and classification,” Computational Intelligence, vol. 6, 1990.
[5] J.Y. Ching, A.K.C. Wong, and K.C.C. Chan, Class-Dependent Discretization for Inductive Learning from Continuous and Mixed Mode Data IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 7, pp. 641-651, July 1995.
[6] D.K.Y. Chiu, "Pattern Analysis Using Event-Covering," PhD dissertation, Dept. of Systems Design Engineering, Univ. of Waterloo, Ontario, Canada, 1986.
[7] D.K.Y. Chiu and A.K.C. Wong, “Synthesizing Knowledge: A Cluster Analysis Approach Using Event Covering,” IEEE Trans. Systems, Man, and Cybernetics, vol. 16, no. 2, pp. 251-259, 1986.
[8] D.K.Y. Chiu, A.K.C. Wong, and B. Cheung, "Information Discovery through Hierarchical Maximum Entropy Discretization and Synthesis," Knowledge Discovery in Databases, G. Piatetsky-Shapiro and W.J. Frawley, eds. AAAI Press/The MIT Press, 1991.
[9] P. Clark and T. Niblett, "Induction in Noisy Domains," Progress in Machine Learning: Proc. Second European Working Session Learning, I. Bratko and N. Larvac, eds., 1987.
[10] P. Clark and T. Niblett, "The CN2 Induction Algorithm," Machine Learning, vol. 3, pp. 261-283, 1989.
[11] D.H. Fisher, "A Hierarchical Conceptual Clustering Algorithm," technical report, Dept. of Information and Computer Science, Univ. of California, Irvine, 1984.
[12] D.H. Fisher, “Knowledge Acquisition via Incremental Conceptual Clustering,” Machine Learning, no. 2, pp. 139-172, 1987.
[13] D.H. Fisher, "Concept Clustering, Learning from Examples, and Inference," Proc. Fourth Int'l Workshop Machine Learning, pp. 38-49, 1987.
[14] D.H. Fisher and K.B. McKusick, "An Empirical Comparison of ID3 and Back-Propagation," Proc. 11th Int'l Joint Conf. Artificial Intelligence, vol. 1, 1989.
[15] D.H. Fisher and P.K. Chan, "Statistical Guidance in Symbolic Learning," Annals Math. and Artificial Intelligence, no. 2, pp. 135-147, 1990.
[16] R.M. Fung and S.L. Crawford, T"Constructor: A System for the Induction of Probabilistic Models," Proc. Eighth Nat'l Conf. Artificial Intelligence, AAAI '90, 1990.
[17] R.M. Fung, S.L. Crawford, L. Appelbaum, and R. Tong, "An Architecture for Probabilistic Concept-Based Information Retrieval," Proce. 13th Int'l Conf. Research and Development in Information Retrieval, 1990.
[18] R.M. Goodman and P. Smyth, "Information-Theoretic Rule Induction," Proc. Eighth European Conf. Artificial Intelligence, pp. 357-362, 1988.
[19] S.J. Haberman, "The Analysis of Residuals in Cross-Classified Tables," Biometrics, vol. 29, pp. 205-220, 1973.
[20] S.J. Haberman, The Analysis of Frequency Data. Univ. of Chicago Press, 1974.
[21] M. Holsheimer and A. Siebes, "Data Mining: The Search for Knowledge in Databases," Technical Report CS-R9406, CWI, Amsterdam 1994.
[22] R.A. Kowalski,Logic for Problem Solving.New York, 1979.
[23] P. Langley and J.G. Carbonell, "Approaches to Machine Learning," J. Am. Soc. Information Science, vol. 35, no. 5, pp. 306-316, 1984.
[24] P. Langley and S. Sage, "Conceptual Clustering as Discrimination Learning," Proc. Fifth Biennial Conf. Canadian Soc. Computational Studies of Intelligence, 1984.
[25] R. Michalski and R. Chilauski, "Knowledge Acquisition by Encoding Expert Rules versus Computer Induction from Examples: A Case Study Involving Soybean Pathology," Int'l J. Man-Machine Studies, vol. 12, pp. 63-87, 1980.
[26] R. Michalski and P. Stepp, "Automated Construction of Classifications: Conceptual Clustering versus Numerical Taxonomy," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 4, pp. 396-409, 1983.
[27] S. Muggleton, Inductive Logic Programming. Academic Press, 1992.
[28] J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, Calif.: Morgan Kaufman, 1988.
[29] J.R. Quinlan,"Induction of decision trees," Machine Learning, vol. 1, pp. 81-106, 1986.
[30] P. Smyth, R.M. Goodman, and C. Higgins, "A Hybrid Rule-Based/Bayesian Classifier," Proc. Ninth European Conf. Artificial Intelligence, pp. 610-615,Stockholm, 1990.
[31] P. Smyth and R. Goodman, "An Information Theoretic Approach to Rule Induction from Databases," IEEE Trans Knowledge and Data Eng., vol. 4, no. 4, pp. 301-316, Aug. 1992.
[32] C.J. Thornton, Techniques in Computational Learning.London: Chapman&Hall, 1992.
[33] S.B. Thrun et al., "The MONK's Problems: A Performance Comparison of Different Learning Algorithms," Technical Report CS-CMU-91-197, Carnegie Mellon Univ., 1991.
[34] W.H. Wolberg and O.L. Mangasarian, "Multisurface Method of Pattern Separation for Medical Diagnosis Applied to Breast Cytology," Proc. Nat'l Academy Sciences,U.S.A., vol. 87, pp. 9,193-9,196, Dec. 1990.
[35] A.K.C. Wong and D.C.C. Wang, "DECA: A Discrete-Valued Data Clustering Algorithm," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 1, no. 4, pp. 342-349, 1979.
[36] A.K.C. Wong and D.K.Y. Chiu,“An event-covering method for effective probabilistic inference,” Pattern Recognition, vol. 20, no. 2, pp. 245-255, 1987.
[37] A.K.C. Wong and D.K.Y. Chiu,“Synthesizing statistical knowledge from incomplete mixed-mode data,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 6, pp. 796-805, 1987.
[38] A.K.C. Wong, S.W. Lu, and M. Roux, "Recognition and Shape Synthesis of 3D Object Based on Attributed Hypergraph," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 3, pp. 279-290, Mar. 1990.
[39] N. Wrigley, Categorical Data Analysis for Geographers and Environmental Scientists. Longman, 1985.

Index Terms:
Adjusted residual, attributed hypergraph, data analysis, database mining, machine learning, pattern discovery, pattern representation.
Andrew K.C. Wong, Yang Wang, "High-Order Pattern Discovery from Discrete-Valued Data," IEEE Transactions on Knowledge and Data Engineering, vol. 9, no. 6, pp. 877-893, Nov.-Dec. 1997, doi:10.1109/69.649314
Usage of this product signifies your acceptance of the Terms of Use.