This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Sequential and Parallel Neural Network Vector Quantizers
January 1994 (vol. 43 no. 1)
pp. 104-109

Presents novel sequential and parallel learning techniques for codebook design in vector quantizers using neural network approaches. These techniques are used in the training phase of the vector quantizer design. These learning techniques combine the split-and-cluster methodology of the traditional vector quantizer design with neural learning, and lead to better quantizer design (with fewer distortions). The sequential learning approach overcomes the code word underutilization problem of the competitive learning network. As a result, this network only requires partial or zero updating, as opposed to full neighbor updating as needed in the self organizing feature map. The parallel learning network, while satisfying the above characteristics, also leads to parallel learning of the codewords. The parallel learning technique can be used for faster codebook design in a multiprocessor environment. It is shown that this sequential learning scheme can sometimes outperform the traditional LBG algorithm, while the parallel learning scheme performs very close to the LGB and the sequential learning algorithms.

[1] N. S. Jayant and P. Noll,Digital Coding of Waveforms. Englewood Cliffs, NJ: Prentice-Hall, 1984.
[2] A. N. Netravali and B. G. Haskell,Digital Pictures Representation and Compression. New York: Plenum, 1988.
[3] R. M. Gray, "Vector quantization,"IEEE ASSP Mag., vol. 1, pp. 4-29, Apr. 1984.
[4] Y. Linde, A. Buzo, and R. M. Gray, "An algorithm for vector quantizer design,"IEEE Trans. Commun., vol. COM-28, pp. 84-95, Jan. 1980.
[5] A. Gersho and V. Cuperman, "Vector quantization: A pattern-matching technique for speech coding,"IEEE Commun. Mag., vol. 21, pp. 15-21, Dec. 1983.
[6] J. MacQueen, "Some methods for classification and analysis of multivariate observations," inProc. 5th Berkeley Symp. Math., Stat., and Prob., vol. 1, 1967, pp. 281-296.
[7] S. Grossberg, "Adaptive pattern classification and universal recording--I: Parallel development and coding of neural feature detectors,"Biol. Cybern., vol. 23, pp. 121-134, 1976.
[8] S. Grossberg, "Adaptive pattern classification and universal recording--II: Feedback, expectation, olfaction, and illusions,"Biol. Cybern., vol. 23, pp. 187-202, 1976.
[9] S. Grossberg, "Competitive learning: From interactive activation to adaptive resonance,"Cog. Sci., vol. 11, pp. 23-63, 1987.
[10] T. Kohonen,Self-Organization and Associative Memory. Berlin, Germany: Springer-Verlag, 1988, p. 132.
[11] T. Kohonen, "An introduction to neural computing,"Neural Net., vol. 1, no. 1, pp. 3-16, 1988.
[12] T. Kohonen, "Learning vector quantization," inAbstr. 1st Ann. INNS Meeting. New York: Pergamon, 1988, p. 303.
[13] T. Kohonen,Self-Organization and Associative Memory. Berlin, Germany: Springer-Verlag, 1988, p. 132.
[14] D. E. Rumelhart, and D. Zipser, "Feature discovery by competitive learning,"Cogn. Sci., vol. 9, pp. 75-112, 1985.
[15] D.E. Rumelhart and D. McClelland, eds.,Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vols. 1-2, MIT Press, Cambridge, Mass., 1986.
[16] R. Hecht-Nielsen, "Applications of counterpropagation networks,"Neural Networks, vol. 1, no. 2, pp. 131-141, 1988.
[17] L. Steele, "Self-organization through selection," inIEEE Int. Conf. Neural Networks, San Diego, CA, 1988, pp. 1155-1162.
[18] R. P. Lippman, "An introduction to computing with neural nets,"IEEE ASSP Msg., vol. 4, pp. 4-22, 1987.
[19] J. Naylor and K. P. Li, "Analysis of a neural network algorithm for vector quantization of speech parameters," inProc. 1st Ann. INNS Meeting. New York: Pergamon, 1988, p. 310.
[20] N. M. Nasrabadi and Y. Feng, "Vector quantization of images based upon the Kohonen self-organizing feature maps," inProc. IEEE Int. Conf. Neural Networks, San Diego, CA, 1988, pp. 1101-1108.
[21] S. C. Ahalt, A. K. Krishnamurthy, P. Chen, and D. E. Melton, "Competitive learning algorithms for vector quantization," vol. 3. New York: Pergamon, 1990, pp. 277-290.
[22] F. H. Wu and K. Ganesan, "An algorithm for robust vector quantization using a neural-net model," presented at IEEE Int. Conf. Neural Networks, San Diego, CA, July 1988.
[23] F. H. Wu and K. Ganesan, "Comparative study of algorithms for VQ design using conventional (LBG) and neural-net based approaches," inProc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Scotland, UK, May 1989, pp. 751-754.
[24] F. H. Wu, K. K. Parhi, and K. Ganesan, "Neural network vector quantizer design using sequential and parallel learning techniques," inProc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Toronto, Canada, May 1991, pp. 637-640.
[25] T. C. Denk, "Final report for project to research algorithms for vector quantizer design using variations of the NNVQ algorithm," senior honors design project report, Dep. Elec. Eng., Univ. of Minnesota, Minneapolis, June 1990.

Index Terms:
learning (artificial intelligence); neural nets; parallel algorithms; parallel neural network; vector quantizers; parallel learning techniques; codebook design; neural learning; sequential learning; competitive learning; self organizing feature map; parallel learning.
Citation:
K.K. Parhi, F.H. Wu, K. Genesan, "Sequential and Parallel Neural Network Vector Quantizers," IEEE Transactions on Computers, vol. 43, no. 1, pp. 104-109, Jan. 1994, doi:10.1109/12.250614
Usage of this product signifies your acceptance of the Terms of Use.