This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction
Aug. 2012 (vol. 24 no. 8)
pp. 1491-1505
ASCII Text
x 
 
Xi-Zhao Wang, Ling-Cai Dong, Jian-Hui Yan, "Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction," IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 8, pp. 1491-1505, Aug., 2012.
 
 
BibTex
x
 
@article{ 10.1109/TKDE.2011.67,
author = {Xi-Zhao Wang and Ling-Cai Dong and Jian-Hui Yan},
title = {Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction},
journal ={IEEE Transactions on Knowledge and Data Engineering},
volume = {24},
number = {8},
issn = {1041-4347},
year = {2012},
pages = {1491-1505},
doi = {http://doi.ieeecomputersociety.org/10.1109/TKDE.2011.67},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Knowledge and Data Engineering
TI - Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction
IS - 8
SN - 1041-4347
SP1491
EP1505
EPD - 1491-1505
A1 - Xi-Zhao Wang,
A1 - Ling-Cai Dong,
A1 - Jian-Hui Yan,
PY - 2012
KW - Learning
KW - uncertainty
KW - sample selection
KW - fuzzy decision tree.
VL - 24
JA - IEEE Transactions on Knowledge and Data Engineering
ER -
 
Xi-Zhao Wang, Hebei University, Baoding
Ling-Cai Dong, Hebei University, Baoding
Jian-Hui Yan, Hebei University, Baoding
Sample selection is to select a number of representative samples from a large database such that a learning algorithm can have a reduced computational cost and an improved learning accuracy. This paper gives a new sample selection mechanism, i.e., the maximum ambiguity-based sample selection in fuzzy decision tree induction. Compared with the existing sample selection methods, this mechanism selects the samples based on the principle of maximal classification ambiguity. The major advantage of this mechanism is that the adjustment of the fuzzy decision tree is minimized when adding selected samples to the training set. This advantage is confirmed via the theoretical analysis of the leaf-nodes' frequency in the decision trees. The decision tree generated from the selected samples usually has a better performance than that from the original database. Furthermore, experimental results show that generalization ability of the tree based on our selection mechanism is far more superior to that based on random selection mechanism.

[1] N. Abe and H. Mamitsuka, "Query Learning Strategies Using Boosting and Bagging," Proc. 15th Int'l Conf. Machine Learning, pp. 1-10, 1998.
[2] D.W. Aha, D. Kibler, and M.K. Albert, "Instance-Based Learning Algorithms," Machine Learning, vol. 6, pp. 37-66, 1991.
[3] F. Angiulli, "Fast Condensed Nearest Neighbor Rule," Proc. 22nd Int'l Conf. Machine Learning: ACM Int'l Conf. Proceeding Series, vol. 119, pp. 25-32, 2005.
[4] D. Angluin, "Queries and Concept Learning," Machine Learning, vol. 2, no. 4, pp. 319-342, 1988.
[5] Y. Baram, R. El-Yaniv, and K. Luz, "Online Choice of Active Learning Algorithms," J. Machine Learning Research, vol. 5, pp. 255-291, 2003.
[6] H. Brighton and C. Mellish, "Advances in Instance Selection for Instance-Based Learning Algorithms," Data Mining and Knowledge Discovery, vol. 6, pp. 153-172, 2002.
[7] C. Campbell, N. Cristianini, and A. Smola, "A Query Learning with Large Margin Classifiers," Proc. 17th Int'l Conf. Machine Learning, pp. 111-118, 2000.
[8] C.L. Chang, "Finding Prototypes for Nearest Neighbor Classifiers," IEEE Trans. Computers, vol. C-23, no. 11, pp. 1179-1184, Nov. 1974.
[9] D. Cohn, Z. Ghahramani, and M.I. Jordan, "Active Learning with Statistical Models," Advances in Neural Information Processing Systems, vol. 7, pp. 705-712, 1995.
[10] D. Cohn and A.R. Ladner, "Improving Generalization with Active Learning," Machine Learning, vol. 5, no. 2, pp. 201-221, 1994.
[11] I. Dagon and S. Engelson, "Committee-Based Sampling for Training Probabilistic Classifiers," Proc. 12th Int'l Conf. Machine Learning, pp. 150-157, 1995.
[12] A. Deluca and S. Termini, "A Definition of Nonprobabilistic Entropy in the Setting of Fuzzy Sets Theory," Information and Control, vol. 20, pp. 301-312, 1972.
[13] D. Dubois and H. Prade, "Rough Fuzzy Sets and Fuzzy Rough Sets," Int'l J. General Systems, vol. 17, pp. 191-209, 1990.
[14] G.W. Gates, "The Reduced Nearest Neighbor Rule," IEEE Trans. Information Theory, vol. IT-18, no. 3, pp. 431-433, May 1972.
[15] P.E. Hart, "The Condensed Nearest Neighbor Rule," IEEE Trans. Information Theory, vol. IT-14, no. 3, pp. 515-516, May 1968.
[16] R.V.L. Hartley, "Transmission of Information," The Bell System Technical J., vol. 7, pp. 535-563, 1949.
[17] M. Higashi and G.J. Klir, "Measures of Uncertainty and Information Based on Possibility Distributions," Int'l J. General Systems, vol. 9, no. 1, pp. 43-58, 1983.
[18] V.S. Iyengar, C. Apte, and T. Zhang, "Active Learning Using Adaptive Resampling," KDD '00: Proc. Sixth ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 91-98, 2000.
[19] D. Lewis and J. Catlett, "Heterogenous Uncertainty Sampling for Supervised Learning," Proc. 17th Ann. ACM-SIGR Conf. Research and Development in Information Retrival, pp. 148-156, 1994.
[20] D. Lewis and W.A. Gail, "A Sequential Algorithm for Training Text Classifiers," Proc. 17th ACM Int'l Conf. Research and Development in Information Retrieval, pp. 3-12, 1994.
[21] H. Li, "Probability Representation of Fuzzy System," Science in China Series E: Technological Sciences, vol. 36, no. 4, pp. 373-397, 2006.
[22] B. Liu, Uncertainty Theory, pp. 57-138, third ed., Springer, http://orsc.edu.cn/liuut.pdf, 2008.
[23] P. Melville and R.J. Mooney, "Diverse Ensembles for Active Learning," Proc. 21th Int'l Conf. Machine Learning: ACM Int'l Conf. Proceeding Series, vol. 69, pp. 74-74, 2004.
[24] H.T. Nguyen and A. Smeulders, "Active Learning Using Pre-Clustering," ICML '04: Proc. 21st Int'l Conf. Machine Learning, pp. 79-86, 2004.
[25] Z. Pawlak, "Rough Sets," Int'l J. Information and Computer Sciences, vol. 11, pp. 341-356, 1982.
[26] G.L. Ritter, H.B. Woodruff, S.R. Lowry, and T.L. Isenhour, "An Algorithm for a Selective Nearest Neighbor Rule," IEEE Trans. Information Theory, vol. IT-21, no. 6, pp. 665-669, Nov. 1975.
[27] N. Roy and A. Mccallum, "Toward Optimal Active Learning through Sampling Estimation of Error Reduction," Proc. 18th Int'l Conf. Machine Learning, pp. 441-448, 2001.
[28] G. Schohn and D. Cohn, "Less Is More: Active Learning with Support Vector Machines," Proc. 17th Int'l Conf. Machine Learning, pp. 839-846, 2000.
[29] H.S. Seung, M. Opper, and H. Sompolinsky, "Query by Committee," Proc. Ann. Workshop Computational Learning Theory, pp. 287-294, 1992.
[30] C.E. Shannon, "A Mathematical Theory of Communication," The Bell System Technical J., vol. 27, pp. 379-423, 623-656, 1948.
[31] C. Tohompson, M.E. Califf, and R. Mooney, "Active Learning for Natural Language Parsing and Information Extraction," Proc. 16th Int'l Conf. Machine Learning, pp. 406-414, 1999.
[32] S. Tong and D. Koller, "Support Vector Machine Active Learning with Applications to Text Classification," Proc. 17th Int'l Conf. Machine Learning (ICML '00), pp. 999-1006, 2000.
[33] S. Tong and D. Koller, "Active Learning for Parameter Estimation in Bayesian Networks," Advances in Neural Information Processing Systems, pp. 647-653, 2000.
[34] E.C.C. Tsang and X.Z. Wang, "An Approach to Case-Based Maintenance: Selecting Representative Cases," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 19, pp. 79-89, 2005.
[35] V. Vapnik, The Nature of Statistical Learning Theory. Springer, 1995.
[36] X.Z. Wang, J.H. Yan, R. Wang, and C.R. Dong, "A Sample Selection Algorithm in Fuzzy Decision Tree Induction and Its Theoretical Analyses," ICSMC '07: Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 3621-3626, 2007.
[37] D.L. Wilson, "Asymptotic Properties of Nearest Neighbor Rules Using Edited Data," IEEE Trans. Systems, Man and Cybernetics, vol. 2, no. 3, pp. 408-421, July 1972.
[38] D.R. Wilson and T.R. Martinez, "Instance Pruning Techniques," Proc. 14th Int'l Conf. Machine Learning, pp. 403-411, 1997.
[39] D.R. Wilson and T.R. Martinez, "Reduction Techniques for Instance-Based Learning Algorithms," Machine Learning, vol. 38, no. 3, pp. 257-286, 2000.
[40] Y.F. Yuan and M.J. Shaw, "Induction of Fuzzy Decision Trees," Fuzzy Sets and Systems, vol. 69, pp. 125-139, 1995.
[41] L.A. Zadeh, "Fuzzy Sets," Information and Control, vol. 8, pp. 338-53, 1965.
[42] R.V.L. Hartley, "Transmission of Information," The Bell System Technical J., vol. 7, no. 3, pp. 535-563, 1928.
[43] O. Frank and R. Doron, "Random Sampling from Databases: A Survey," Statistics and Computing, vol. 5, no. 1, pp. 25-42, Mar. 1995.
[44] S.-T. Maytal and P. Foster, "Active Sampling for Class Probability Estimation and Ranking," Machine Learning, vol. 54, no. 2, pp. 153-178, 2004.
[45] M. Prem, M.Y. Stewart, S.-T. Maytal, and J.M. Raymond, "Active Learning for Probability Estimation Using Jensen-Shannon Divergence," Proc. 16th European Conf. Machine Learning (ECML), pp. 268-279, 2005.
[46] UCI Machine Learning Repository, http//archive.ics.uci.eduml/, 2011.

Index Terms:
Learning, uncertainty, sample selection, fuzzy decision tree.
Citation:
Xi-Zhao Wang, Ling-Cai Dong, Jian-Hui Yan, "Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction," IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 8, pp. 1491-1505, Aug. 2012, doi:10.1109/TKDE.2011.67
Usage of this product signifies your acceptance of the Terms of Use.