This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
The Meta-Pi Network: Building Distributed Knowledge Representations for Robust Multisource Pattern Recognition
July 1992 (vol. 14 no. 7)
pp. 751-769

The authors present the Meta-Pi network, a multinetwork connectionist classifier that forms distributed low-level knowledge representations for robust pattern recognition, given random feature vectors generated by multiple statistically distinct sources. They illustrate how the Meta-Pi paradigm implements an adaptive Bayesian maximum a posteriori classifier. They also demonstrate its performance in the context of multispeaker phoneme recognition in which the Meta-Pi superstructure combines speaker-dependent time-delay neural network (TDNN) modules to perform multispeaker /b,d,g/ phoneme recognition with speaker-dependent error rates of 2%. Finally, the authors apply the Meta-Pi architecture to a limited source-independent recognition task, illustrating its discrimination of a novel source. They demonstrate that it can adapt to the novel source (speaker), given five adaptation examples of each of the three phonemes.

[1] A. Barron, "Statistical learning networks: A unifying view," presented at the Symp. Interface: Stat. Comput. Sci., 1988.
[2] H. Bourlard and C. Wellekens, "Links between Markov models and multilayer perceptrons,"IEEE Trans. Patt. Anal. Machine Intell., vol. 12, no. 12, pp. 1167-1178, Dec. 1990.
[3] D. E. Rumelhartet al., Parallel Distributed Processing. Cambridge, MA: MIT Press, 1987.
[4] L. Devroye, "Automatic pattern recognition: A study of the probability of error,"IEEE Trans. Patt. Anal. Machine Intell., vol. 10, no. 4, pp. 530-543, July 1988.
[5] R. O. Duda and P. E. Hart,Pattern Classification and Scene Analysis. New York: Wiley, 1973.
[6] H. Gish, "A probabilistic approach to the understanding and training of neural network classifiers," inProc. 1990 IEEE Int. Conf. Acoustics Speech Signal Processing, Apr. 1990, pp. 1361-1364, vol. 3.
[7] J. B. Hampshire, II, B. A. Pearlmutter, and B. V. K. Vijaya Kumar, "General equivalence proofs for supervised pattern classifiers and the Bayesian discriminant function," to be published; this is a revised and extended version of "Equivalence proofs for multi-layer perceptron clasifiers and the Bayesian discriminant function," inProc. 1990 Connectionist Models Summer School, 1991, pp. 159-172.
[8] J. B. Hampshire, II and A. H. Waibel, "The Meta-Pi network: Building distributed knowledge representations for robust pattern recognition, Tech. Rep. CMU-CS-89-166-R, Carnegie Mellon Univ., School Comput. Sci., Aug. 1989.
[9] J. B. Hampshire, II and A. H. Waibel, "A novel objective function for improved phoneme recognition using time-delay neural networks,"IEEE Trans. Neural Networks, vol. 1, no. 2, pp. 216-228, June 1990.
[10] V. Hasselblad, "Estimation of parameters for a mixture of normal distributions,"Technometrics, vol. 8, pp. 431-444, 1966.
[11] W. H. Highleyman, "The design and analysis of pattern recognition experiments,"Bell Syst. Tech. J., vol. 41, pp. 723-744, Mar. 1962.
[12] G. E. Hinton, "Connectionist learning procedures," inMachine Learning: Paradigms and Methods(J. G. Carbonell, Ed.). Cambridge, MA: MIT Press, 1990, pp. 185-234.
[13] H. W. Hon, personal communication regarding the SPHINX speech recognition system, Apr. 1990.
[14] X. D. Huang, K. F. Lee, and H. W. Hon, "On semicontinuous hidden Markov modeling," inProc. IEEE 1990 Conf. Acoustics Speech Signal Processing, 1990, vol. S2VA, pp. 689-692.
[15] R. Jacobs, "Initial experiments on constructing domains of expertise and hierarchies in connectionist systems," inProc. 1988 Connectionist Models Summer School(San Mateo, CA), 1988, pp. 144-153.
[16] R. Jacobs, "Task decomposition through competition in a modular connectionist architecture," Tech. Rep. COINS-90-27, Univ. of Mass., Dept. of Comput. Inform. Sci., Mar. 1990.
[17] R. Jacobs, M. Jordan, S. Nowlan, and G. Hinton, "Adaptive mixtures of local experts,"Neural Computation, vol. 3, no. 1, Jan. 1991.
[18] B. Kämmerer and W. Küper, "Design of hierarchical perceptron structures and their application to the task of isolated word recognition," inIEEE Proc. 1989 Int. Joint Conf. Neural Networks(Washington, DC), June 1989, pp. 243-249.
[19] K. J. Lang, "A time-delay neural network architecture for speech recognition," Ph.D. thesis, Carnegie Mellon Univ., School of Comput. Sci., July 1989.
[20] K. F. Lee, "Large vocabulary speaker-independent continuous speech recognition: The SPHINX system," Tech. Rep. CMU-CS-88-148, Carnegie Mellon Univ., School of Comput. Sci., Apr. 1988.
[21] H. C. Leung and V. W. Zue, "Applications of error back-propagation to phonetic classification," inAdvances in Neural Information Processing Systems, vol. 1(D. S. Touretzky, Ed.). San Mateo, CA: Morgan Kauffman, 1989, pp. 206-214.
[22] R. Lippmann, "Pattern classification using neural networks,"IEEE Commun. Mag., vol. 27, no. 11, 1989.
[23] P. Maloney and D. Specht, "The use of probabilistic neural networks to improve solution times for hull-to-emitter correlation problems," inProc. IEEE 1989 Int. Conf. Neural Networks, June 1989, pp. 289-294, vol. 1.
[24] T. Matsuoka, H. Hamada, and R. Nakatsu, "Syllable recognition using integrated neural networks," inProc. IEEE 1989 Int. Joint Conf. Neural Networks, June 1989, pp. 251-258, vol. 1.
[25] S. J. Nowlan, "Competing experts: An experimental investigation of associative mixture models," Tech. Rep. CRG-TR-90-5, Univ. of Toronto, Dept. of Comput. Sci., Sept. 1990.
[26] S. J. Nowlan, "Maximum likelihood competitive learning," inAdvances in Neural Information Processing Systems, vol. 2(D. S. Touretzky, Ed.). San Mateo, CA: Morgan Kauffman, 1990, pp. 574-582.
[27] S. J. Nowlan, "Soft competitive adaptation: Neural network learning algorithms based on fitting statistical mixtures," Ph.D. thesis CMU-CS-91- 126, Carnegie Mellon Univ., Pittsburgh, PA, Apr. 1991.
[28] D. O'Shaugnessy,Speech Communication: Human and Machine. Reading, MA: Addison-Wesley, 1987.
[29] J. Pollack, "Cascaded back-propagation on dynamic connectionist networks," inProc. Ninth Ann. Conf. Cognitive Sci. Soc., 1987, pp. 391-404, 1987.
[30] D. A. Pomerleau, G. L. Gusciora, D. S. Touretzky, and H. T. Kung, "Neural network simulation at warp speed: How we got 17 million connections per second," inProc. Int. Conf. Neural Networks, San Diego, CA, June 1988.
[31] D. A. Pomerleau, "The meta-generalized delta rule: A new algorithm for learning in connectionist networks," Tech. Rep. CMU-CS-87-165, Carnegie Mellon Univ., School of Comput. Sci., Sept. 1987.
[32] L. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition,"Proc. IEEE, vol. 77, pp. 257-286, Feb. 1989.
[33] R. A. Redner and H. F. Walker, "Mixture densities, maximum likelihood and the EM algorithm,"SIAM Rev., vol. 26, pp. 195-239, 1984.
[34] D. Rtischev, "Speaker adaptation in a large vocabulary speech recognition system," Master's thesis, Mass. Inst. of Technol., Jan. 1989.
[35] D. W. Ruck, S. K. Rogers, M. Kabrinsky, M. E. Oxley, and B. W. Sutter, "The multilayer perceptron as an approximation to a Bayes optimal discriminant function,"IEEE Trans. Neural Networks, vol. 1, no. 4, pp. 296-298, Dec. 1990.
[36] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by backpropagation errors,"Nature, vol. 323, pp. 533-536, Oct. 1986.
[37] P. A. Shoemaker, "A Note on least-squares learning procedures and classification by neural network models,"IEEE Trans. Neural Networks, vol. 2, no. 1, pp. 158-160, Jan. 1991.
[38] D. Specht, "Probabilistic neural networks for classification, mapping, and associative memory, inProc. IEEE 1988 Int. Conf. Neural Networks, July 1988, pp. 525-532, vol. 1.
[39] R. M. Stern and M. J. Lasry, "Dynamic Speaker Adaptation for Feature-Based Isolated Word Recognition,"IEEE Trans. Acoustics Speech Signal Processing, vol. ASSP-35, pp. 751-763, June 1987.
[40] A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang, "Phoneme recognition using time-delay neural networks,"IEEE Trans. Acoust. Speech Signal Processing, vol. 37, no. 3, 1989.
[41] A. Waibel, H. Sawai, and K. Shikano, "Modularity and scaling in large phonemic neural networks,"IEEE Trans. Acoustics Speech Signal Processing, vol. ASSP-37, pp. 1888-1898, Dec. 1989.
[42] E. A. Wan, "Neural network classification: A Bayesian interpretation,"IEEE Trans. Neural Networks, vol. 1, no. 4, pp. 303-305, Dec. 1990.
[43] E. A. Wan, "Temporal backpropagation: An efficient algorithm for finite impulse response neural networks," inProc. 1990 Connectionist Models Summer School(San Mateo, CA), 1991, pp. 159-172.
[44] R. Watrous, "Context-modulated discrimination of similar vowels using second-order connectionist networks," Tech. Rep. CRG-TR-89-5, Univ. of Toronto, Dept. of Comput. Sci., Dec. 1989.
[45] H. White, "Learning in artificial neural networks: A statistical perspective,Neural Computation, vol. 1, no. 4, pp. 425-464, Winter 1989.

Index Terms:
speech recognition; Meta-Pi network; distributed knowledge representations; multisource pattern recognition; multinetwork connectionist classifier; adaptive Bayesian maximum a posteriori classifier; multispeaker phoneme recognition; speaker-dependent time-delay neural network; Bayes methods; knowledge representation; neural nets; speech recognition; statistical analysis
Citation:
J.B. Hampshire II, A. Waibel, "The Meta-Pi Network: Building Distributed Knowledge Representations for Robust Multisource Pattern Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 7, pp. 751-769, July 1992, doi:10.1109/34.142911
Usage of this product signifies your acceptance of the Terms of Use.