This Article 
 Bibliographic References 
 Add to: 
A Pattern Reordering Approach Based on Ambiguity Detection for Online Category Learning
April 2003 (vol. 25 no. 4)
pp. 524-528

Abstract—Pattern reordering is proposed as an alternative to sequential and batch processing for online category learning. Upon detecting that the categorization of a new input pattern is ambiguous, the input is postponed for a predefined time, after which it is reexamined and categorized for good. This approach is shown to improve the categorization performance over purely sequential processing, while yielding a shorter input response time, or latency, than batch processing. In order to examine the response time of processing schemes, the latency of a typical implementation is derived and compared to lower bounds. Gaussian and softmax models are derived from reject option theory and are considered for detecting ambiguity and triggering pattern postponement. The average latency and Rand Adjusted clustering score of reordered, sequential, and batch processing are compared through computer simulation using two unsupervised competitive learning neural networks and a radar pulse data set.

[1] S.C. Ahalt, A.K. Krishnamurthy, P. Chen, and D.E. Melton, “Competitive Learning Algorithms for Vector Quantization,” Neural Networks, vol. 3, pp. 277-290, 1990.
[2] M.R. Anderberg, Cluster Analysis for Applications. London: Academic Press, 1973.
[3] J.S. Bridle, “Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition,” Neurocomputing—Algorithms, Architectures and Applications, F. Fogelman-Souliéand J. Hérault, eds., NATO ASI Series F68, Berlin: Springer-Verlag, pp. 227-236, 1989.
[4] G. Carpenter and S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Computer Vision, Graphics and Image Understanding, vol. 37, pp. 54–115, 1987.
[5] G.A. Carpenter, S. Grossberg, and D.B. Rosen, “Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System,” Neural Networks, vol. 4, pp. 759–771, 1991.
[6] C.K. Chow, “An Optimum Character Recognition System Using Decision Functions,” IRE Trans. Electronic Computers, vol. 6, pp. 247-254, 1957.
[7] C.K. Chow, "On Optimum Recognition Error and Reject TradeOff," IEEE Trans. Information Theory, vol. 16, no. 1, pp. 41-46, 1970.
[8] C.L. Davies and P. Hollands, “Automatic Processing for ESM,” Proc. IEE, vol. 129, no. 3, pp. 164-171, June 1982.
[9] A.K. Jain and R.C. Dubes, Algorithms for Clustering Data. Englewood Cliffs, N.J.: Prentice Hall, 1988.
[10] B. Dubuisson and M. Masson, “A Statistical Decison Rule with Incomplete Knowledge about Classes,” Pattern Recognition, vol. 26, no. 1, pp. 155-165, 1993.
[11] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis, John Wiley and Sons, 1973.
[12] T. Frank, K.-F. Kraiss, and T. Kuhlan, “Comparative Analysis of Fuzzy ART and ART-2A Network Clustering Performance,” IEEE Trans. Neural Networks, vol. 9, no. 3, pp. 544-559, 1998.
[13] K. Fukunaga and D.L. Kessell, "Application of Error-Reject Functions," IEEE Trans. Information Theory, vol. 18, pp. 814-817, 1972.
[14] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[15] A. Gersho, “On the Structure of Vector Quantizers,” IEEE Trans. Information Theory, vol. 28, pp. 157-166, 1982.
[16] E. Granger, Y. Savaria, P. Lavoie, and M.-A. Cantin, “A Comparison of Self-Organizing Neural Networks for Fast Clustering of Radar Pulses,” Signal Processing, vol. 64, no. 3, pp. 249-269, 1998.
[17] E. Granger, Y. Savaria, and P. Lavoie, “A Pattern Reordering Approach Based on Ambiguity Detection for On-Line Category Learning,” Département de GénieÉlectrique,École Polytechnique de Montréal, EPM/RT-01/02, pp. 40, Sept. 2001.
[18] R.M. Gray, "Vector Quantization," IEEE Acoustics, Speech and Signal Processing, pp. 4-29, Apr. 1984.
[19] S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: I. Parallel Development and Coding of Neural Detectors,” Biological Cybernetics, vol. 23, pp. 121-134, 1976.
[20] S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: II. Feedback, Oscillation, Olfaction, and Illusions,” Biological Cybernetics, vol. 23, pp. 187-207, 1976.
[21] J.A Hartigan,Clustering Algorithms, John Wiley and Sons, New York, N.Y., 1975.
[22] L. Hubert and P. Arabie, “Comparing Partitions,” J. Classification, vol. 2, pp. 193-218, 1985.
[23] M.E. Hellman, “The Nearest Neighbor Classification Rule with Reject Option,” IEEE Trans. Systems Science and Cybernetics, vol. 6, no. 3, pp. 179-185, 1970.
[24] T. Kohonen, "Self-Organization and Associated Memory," Berlin Heidelberg. New York: Springer-Verlag, 1988.
[25] Y. Linde, A. Buzo, R.M. Gray, An Algorithm for Vector Quantizer Design IEEE Trans. Comm., vol. 28, no. 1, pp. 84-95, 1980.
[26] S.P. Lloyd, “Least Squares Quantization in PCM,” IEEE Trans. Information Theory, vol. 28, pp. 129-137, Mar. 1982.
[27] J. MacQueen, “Some Methods for Classification Analysis of Multivariate Observations,” Proc. Fifth Berkley Symp. Math. Statistics and Probability, pp. 281-297, 1967.
[28] D.E. Rumelhart and D. Zipser, “Feature Discovery by Competitive Learning,” Cognitive Science, vol. 9, pp. 75-112, 1985.

Index Terms:
Ambiguity, online category learning, partitional clustering, pattern recognition, reject option.
Eric Granger, Yvon Savaria, Pierre Lavoie, "A Pattern Reordering Approach Based on Ambiguity Detection for Online Category Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 4, pp. 524-528, April 2003, doi:10.1109/TPAMI.2003.1190579
Usage of this product signifies your acceptance of the Terms of Use.