This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Learning Deterministic Finite Automata with a Smart State Labeling Evolutionary Algorithm
July 2005 (vol. 27 no. 7)
pp. 1063-1074
ASCII Text
x 
 
Simon M. Lucas, T. Jeff Reynolds, "Learning Deterministic Finite Automata with a Smart State Labeling Evolutionary Algorithm," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 7, pp. 1063-1074, July, 2005.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2005.143,
author = {Simon M. Lucas and T. Jeff Reynolds},
title = {Learning Deterministic Finite Automata with a Smart State Labeling Evolutionary Algorithm},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {27},
number = {7},
issn = {0162-8828},
year = {2005},
pages = {1063-1074},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2005.143},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Learning Deterministic Finite Automata with a Smart State Labeling Evolutionary Algorithm
IS - 7
SN - 0162-8828
SP1063
EP1074
EPD - 1063-1074
A1 - Simon M. Lucas,
A1 - T. Jeff Reynolds,
PY - 2005
KW - Index Terms- Grammatical inference
KW - finite state automata
KW - random hill climber
KW - evolutionary algorithm.
VL - 27
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Learning a Deterministic Finite Automaton (DFA) from a training set of labeled strings is a hard task that has been much studied within the machine learning community. It is equivalent to learning a regular language by example and has applications in language modeling. In this paper, we describe a novel evolutionary method for learning DFA that evolves only the transition matrix and uses a simple deterministic procedure to optimally assign state labels. We compare its performance with the Evidence Driven State Merging (EDSM) algorithm, one of the most powerful known DFA learning algorithms. We present results on random DFA induction problems of varying target size and training set density. We also study the effects of noisy training data on the evolutionary approach and on EDSM. On noise-free data, we find that our evolutionary method outperforms EDSM on small sparse data sets. In the case of noisy training data, we find that our evolutionary method consistently outperforms EDSM, as well as other significant methods submitted to two recent competitions.

[1] P.J. Angeline, G.M. Saunders, and J.P. Pollack, “An Evolutionary Algorithm that Constructs Recurrent Neural Networks,” IEEE Trans. Neural Networks, vol. 5, no. 1, pp. 54-65, Jan. 1994.
[2] H.-G. Beyer, “Toward a Theory of Evolution Strategies: The (mu, lambda)-Theory,” Evolutionary Computation, vol. 2, no. 4, pp. 381-407, 1994.
[3] O. Cicchello and S.C. Kremer, “Beyond EDSM,” Proc. Int'l Colloquium Grammatical Inference, vol. 2484, pp. 37-48, 2002.
[4] O. Cicchello and S.C. Kremer, “Inducing Grammars from Sparse Data Sets: A Survey of Algorithms and Results,” J. Machine Learning Research, vol. 4 pp. 603-632, 2003.
[5] P. Dupont, “Regular Grammatical Inference from Positive and Negative Samples by Genetic Search: The GIG Method,” Proc. Grammatical Inference and Applications: Second Int'l Colloquium (ICGI-94), R.C. Carrasco and J. Oncina, eds., pp. 236-245, 1994.
[6] P. Dupont, L. Miclet, and E. Vidal, “ What Is the Search Space of the Regular Inference?” Proc. Grammatical Inference and Applications: Second Int'l Colloquium (ICGI-94), R.C. Carrasco and J. Oncina, eds., pp. 25-37, 1994.
[7] C. Giles, G. Sun, H. Chen, Y. Lee, and D. Chen, “Higher order Recurrent Neural Networks and Grammatical Inference,” Advances in Neural Information Processing Systems 2, D. Touretzky, ed., pp. 380-387, San Mateo, Calif.: Morgan Kaufman, 1990.
[8] J. Gomez, “An Incremental Learning Algorithm for Deterministic Finite Automata Using Evolutionary Algorithms (Gecco 2004 Noisy DFA Entry),” Proc. Genetic and Evolutionary Computation Conf., http://cswww.essex.ac.uk/staff/sml/gecco/ results/NoisyDFA/entries/gomezGomez.pdf , 2004.
[9] H. Juille and J.B. Pollack, “A Sampling-Based Heuristic for Tree Search Applied to Grammar Induction,” Proc. 15th Nat'l Conf. Artificial Intelligence, pp. 776-783, 1998.
[10] M. Kearns and L.G. Valiant, “Cryptographic Limitations on Learning Boolean Formulae and Finite Automata,” Proc. ACM Symp. Theory of Computation, pp. 433-444, 1989.
[11] K. Lang, “Evidence-Driven State Merging with Search,” NECI Technical Report TR98-139, citeseer.nj.nec.comlang98 evidence.html, 1998.
[12] K.J. Lang, “Random DFAs Can Be Approximately Learned from Sparse Uniform Examples,” Proc. Fifth ACM Workshop Computational Learning Theory, pp. 45-52, 1992.
[13] K.J. Lang, B. Pearlmutter, and F. Coste, “The Gowachin Server,” http://www.irisa.fr/Gowachin, 2005.
[14] K.J. Lang, B.A. Pearlmutter, and R.A. Price, “Results of the Abbadingo One DFA Learning Competition and a New Evidence-Driven State Merging Algorithm,” Proc. Int'l Colloquium Grammatical Inference, pp. 1-12, 1998.
[15] M. Lankhorst, “A Genetic Algorithm for Induction of Non-Deterministic Pushdown Automata,” Computer Science Report CS-R 9502, Univ. of Groningen, http://www.ub.rug.nl/eldoc/dis/science/m.m.lankhorst c4.pdf, 1995.
[16] S. Lucas, “Gecco 2004 Noisy DFA Results,” Proc. Genetic and Evolutionary Computation Conf., http://cswww.essex.ac.uk/staff/sml/gecco/ results/NoisyDFANoisyDFAResults.html, 2004.
[17] S.M. Lucas, “Structuring Chromosomes for Context-Free Grammar Evolution,” Proc. IEEE Int'l Conf. Evolutionary Computation, pp. 130-135, 1994.
[18] S.M. Lucas, “Evolving Finite State Transducers: Some Initial Explorations,” Proc. Sixth European Conf. Genetic Programming, pp. 130-141, 2003.
[19] S.M. Lucas and T.J. Reynolds, “Learning DFA: Evolution versus Evidence Driven State Merging,” Proc. Congress Evolutionary Computation, pp. 351-358, 2003.
[20] S. Luke, S. Hamahashi, and H. Kitano, “‘Genetic’ Programming,” Proc. Genetic and Evolutionary Computation Conf., W. Banzhaf et al., eds., pp. 1098-1105, 1999.
[21] M. Mitchell, An Introduction to Genetic Algorithms. Cambridge, Mass.: MIT Press, 1996.
[22] M. Mitchell, J. Holland, and S. Forrest, “When Will a Genetic Algorithm Outperform Hill Climbing?” Advances in Neural Information Processing Systems 6, J. Cowan, G. Tesauro, and J. Alspector, eds., pp. 51-58, San Mateo, Calif.: Morgan Kaufman, 1994.
[23] A.L. Oliveira and J.P.M. Silva, “Efficient Search Techniques for the Inference of Minimum Size Finite Automata,” String Processing and Information Retrieval, pp. 81-89, 1998.
[24] J. Oncina and P. Garcia, “Inferring Regular Languages in Polynomial Updated Time,” Pattern Recognition and Image Analysis, pp. 49-61, 1992.
[25] L. Pitt and M. Warmuth, “The Minimum Consistent DFA Problem Cannot Be Approximated within Any Polynomial,” J. ACM, vol. 40, no. 1, pp. 95-142, 1993.
[26] J.D. Schaffer, D. Whitley, and L.J. Eshelman, “Combinations of Genetic Algorithms and Neural Networks: A Survey of the State of the Art,” Proc. Int'l Workshop Combinations of Genetic Algorithms and Neural Networks, D. Whitley and J. D. Schaffer, eds., pp. 1-37, 1992.
[27] M. Sebban and J. Janodet, “On State Merging in Grammatical Inference: A Statistical Approach for Dealing with Noisy Data,” Proc. Int'l Conf. Machine Learning (ICML), 2003.
[28] M. Sebban, J. Janodet, and F. Tantini, “BLUE*: a Blue-Fringe Procedure for Learning DFA with Noisy Data (Gecco 2004 Noisy DFA Entry),” Proc. Genetic and Evolutionary Computation Conf., http://cswww.essex.ac.uk/staff/sml/gecco/ results/NoisyDFA/entries/blueBlueFringe.pdf , 2004.
[29] M. Tomita, “Dynamic Construction of Finite Automata from Examples Using Hill Climbing,” Proc. Fourth Ann. Cognitive Science Conf., pp. 105-108, 1982.
[30] B.A. Trakhtenbrot and Y.M. Barzdin, Finite Automata. Amsterdam: North-Holland, 1973.
[31] R. Watrous and G. Kuhn, “Induction of Finite-State Automata Using Second-Order Recurrent Networks,” Advances in Neural Information Processing Systems 4, J. Moody, S. Hanson, and R. Lippmann, eds., pp. 309-316, San Mateo, Calif.: Morgan Kaufman, 1992.
[32] P. Wyard, “Context-Free Grammar Induction Using Genetic Algorithms,” Proc. Fourth Int'l Conf. Genetic Algorithms, R. Belew and L. Booker, eds., pp. 514-518, 1991.

Index Terms:
Index Terms- Grammatical inference, finite state automata, random hill climber, evolutionary algorithm.
Citation:
Simon M. Lucas, T. Jeff Reynolds, "Learning Deterministic Finite Automata with a Smart State Labeling Evolutionary Algorithm," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 7, pp. 1063-1074, July 2005, doi:10.1109/TPAMI.2005.143
Usage of this product signifies your acceptance of the Terms of Use.