This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Learning Concepts in Parallel Based Upon the Strategy of Version Space
December 1994 (vol. 6 no. 6)
pp. 857-867

Applies the technique of parallel processing to concept learning. A parallel version-space learning algorithm based upon the principle of divide-and-conquer is proposed. Its time complexity is analyzed to be O(k log/sub 2/n) with n processors, where n is the number of given training instances and k is a coefficient depending on the application domains. For a bounded number of processors in real situations, a modified parallel learning algorithm is then proposed. Experimental results are then performed on a real learning problem, showing that our parallel learning algorithm works, and being quite consistent with the results of theoretical analysis. We conclude that when the number of training instances is large, it is worth learning in parallel because of its faster execution.

[1] Selim G. Akl,The Design and Analysis of Parallel Algorithms. Englewood Cliffs, NJ: Prentice-Hall, 1989.
[2] B. Arbab and D. Michie, "Generating rules from examples," inProc. 1985 Int. Joint Conf. Art. Intell., 1985, pp. 631-633.
[3] B. G. Buchanan and T. M. Mitchell, "Model-directed learning of production rules," in D. A. Waterman and F. Hayes-Roth, Eds.,Pattern-Directed Inference Systems. New York: Academic, 1978, pp. 297-312.
[4] A. Bundy, B. Silver, and D. Plummer, "An analytical comparison of some rule-learning programs,"Art. Intell., vol. 27, pp. 137-181, 1985.
[5] T. G. Dietterich and R. S. Michalski, "A comparative review of selected methods for learning from examples," in R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds.,Machine Learning: An Artificial Intelligence Approach, vol. 1. Palo Alto, CA: Toiga, 1983, pp. 41-81.
[6] R. A. Fisher, "The use of multiple measurements in taxonomic problems,"Ann. Eugenics, vol. 7, pp. 179-188, 1936.
[7] L. M. Fu and B. G. Buchanan, "Learning intermediate concepts in constructing a hierarchical knowledge base," inProc. 1985 Int. Joint Conf. Art. Intell., 1985, pp. 659-666.
[8] S. I. Gallant, "Automatic generation of expert systems from examples," inProc. 2nd Int. Conf. Art. Intell. Applications, 1985, pp. 313-319.
[9] A. Gupta, "Implementing OPS5 production systems on DADO," inProc. 1984 IEEE Int. Conf. Parallel Processing, 1984, pp. 83-91.
[10] H. Hirsh, "Incremental version-space merging: a general framework for concept learning," Ph.D. dissertation, Stanford Univ., Stanford, CA, USA, 1989.
[11] T. P. Hong and S. S. Tseng, "A parallel concept learning algorithm based upon version space strategy," inProc. 1990 IEEE Int. Phoenix Conf. Comput. Commun., 1990, pp. 734-740.
[12] T. P. Hong and S. S. Tseng, "A generalized learning problem," inProc. 1990 Int. Comput. Symp., 1990, pp. 517-522.
[13] T. P. Hong and S. S. Tseng, "Models of parallel learning systems," inProc. 1991 IEEE Int. Conf. Distrib. Computing Syst., 1991, pp. 125-132.
[14] T. P. Hong and S. S. Tseng, "Parallel perceptron learning on a single-channel broadcast communication model,"Parallel Computing, vol. 18, pp. 133-148, 1992.
[15] T. P. Hong, "A study of parallel processing and noise management on machine learning," Ph.D. dissertation, National Chiao Tung Univ., Taiwan, Republic of China, Jan. 1992.
[16] E. Horowitz and S. Sahni,Fundamentals of Computer Algorithms. Rockville, MD: Computer Sci. Press, 1978.
[17] R. S. Michalski, J. G. Carbonell, and T. M. Mitchell,Machine Learning: An Artificial Intelligence Approach, vol. 1. Palo Alto, CA: Toiga, 1983.
[18] R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds.,Machine Learning: An Artificial Intelligence Approach, vol. 2. Los Altos, CA: Morgan Kaufmann, 1986.
[19] T. M. Mitchell, "Version space: a candidate elimination approach to rule learning," inProc. 1977 Int. Joint Conf. Art. Intell., 1977, pp. 305-310.
[20] T. M. Mitchell, "Version spaces: An approach to concept learning," Ph.D. dissertation, Stanford Univ., Stanford, CA, USA, 1978.
[21] T. M. Mitchell, "An analysis of generalization as a search problem," inProc. 1979 Int. Joint Conf. Art. Intell., 1979, pp. 577-582.
[22] T. M. Mitchell, "Generalization as search,"Art. Intell., vol. 18, pp. 203-226, 1982.
[23] T. M. Mitchell, P. E. Utgoff, and R. Banerji, "Learning by experimentation: Acquiring and refining problem-solving heuristics," in R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds.,Machine Learning: An Artificial Intelligence Approach, vol. 1. Palo Alto, CA: Toiga, 1983, pp. 163-190.
[24] G. D. Plotkin, "A note on inductive generalization," in B. Meltzer and D. Michie, Eds.,Machine Intelligence, vol. 5. New York: Wiley, 1970, pp. 153-163.
[25] G. D. Plotkin, "A further note on inductive generalization," in B. Meltzer and D. Michie, Eds.,Machine Intelligence, vol. 6. New York: Wiley, 1971, pp. 101-124.
[26] M. Quinn,Designing Efficient Algorithms for Parallel Computers. New York: McGraw-Hill, 1987.
[27] R. C. T. Lee, R. C. Chang, and S. S. Tseng,Introduction to Design and Analysis of Algorithms. Englewood Cliffs, NJ: Prentice-Hall, 1991.
[28] M. J. Shaw, "Applying inductive learning to enhance knowledge-based expert systems,"Decision Support Syst., vol. 3, pp. 319-332, 1987.
[29] S. J. Stolfo and D. P. Miranker, "DADO: A parallel processor for expert systems," inProc. 1984 IEEE Int. Conf. Parallel Processing, 1984, pp. 74-82.
[30] L. H. Witten and B. A. Macdonald, "Using concept learning for knowledge acquisition,"Int. J. Man-Mach. Studies, vol. 29, pp. 171-196, 1988.

Index Terms:
learning (artificial intelligence); parallel algorithms; computational complexity; generalisation (artificial intelligence); configuration management; concept learning; parallel version-space learning algorithm; divide-and-conquer method; time complexity; training instances; application domains; bounded processor number; generalization process; hypothesis; specialization process
Citation:
T.P. Hong, S.S. Tseng, "Learning Concepts in Parallel Based Upon the Strategy of Version Space," IEEE Transactions on Knowledge and Data Engineering, vol. 6, no. 6, pp. 857-867, Dec. 1994, doi:10.1109/69.334877
Usage of this product signifies your acceptance of the Terms of Use.