This Article 
 Bibliographic References 
 Add to: 
Closed-Loop Object Recognition Using Reinforcement Learning
February 1998 (vol. 20 no. 2)
pp. 139-154

Abstract—Current computer vision systems whose basic methodology is open-loop or filter type typically use image segmentation followed by object recognition algorithms. These systems are not robust for most real-world applications. In contrast, the system presented here achieves robust performance by using reinforcement learning to induce a mapping from input images to corresponding segmentation parameters. This is accomplished by using the confidence level of model matching as a reinforcement signal for a team of learning automata to search for segmentation parameters during training. The use of the recognition algorithm as part of the evaluation function for image segmentation gives rise to significant improvement of the system performance by automatic generation of recognition strategies. The system is verified through experiments on sequences of indoor and outdoor color images with varying external conditions.

[1] A.G. Barto, R.S. Sutton, and C.J.C.H. Watkins, "Learning and Sequential Decision Making," COINS Technical Report 89-95, Dept. of Computer and Information Science, Univ. of Mass., Amherst, Mass., 1989.
[2] B. Bhanu and T. Jones, "Image Understanding Research for Automatic Target Recognition," Pro. DARPA Image Understanding Workshop, pp. 249-259, 1992.
[3] B. Bhanu and S. Lee, Genetic Learning for Adaptive Image Segmentation.Boston, Mass.: Kluwer Academic Publishers, 1994.
[4] B. Bhanu and J. Ming, "Recognition of Occluded Objects: A Cluster-Structure Algorithm," Pattern Recognition, vol. 20, no. 2, pp. 199-211, 1987.
[5] B. Bhanu, S. Lee, and S. Das, "Adaptive Image Segmentation Using Genetic and Hybrid Search Methods," IEEE Trans. Aerospace and Electronic Systems, vol. 31, no. 4, pp. 1,268-1,291, Oct. 1995.
[6] B. Bhanu, S. Lee, and J. Ming, “Adaptive Image Segmentation Using a Genetic Algorithm,” IEEE Trans. Systems, Man, and Cybernetics, vol. 25, pp. 1,543-1,567, Dec. 1995.
[7] D. Chapman, "Intermediate Vision: Architecture, Implementation, and Use," Cognitive Science, vol. 16, pp. 491-537, 1992.
[8] R.T. Chin and C.R. Dyer, "Model-Based Recognition in Robot Vision," ACM Computing Surveys, pp. 67-108, Mar. 1994.
[9] M.A. Fischler, "On the Representation of Natural Scenes", Computer Vision Systems, A.R. Hanson and E.M. Riseman, eds. New York: Academic Press, 1978.
[10] K. Fukushima, S. Miyake, and T. Ito, "Neocognition: A Neural Network Model for a Mechanism of Visual Pattern Recognition," IEEE Trans. Systems, Man, and Cybernetics, vol. 13, no. 5, pp. 826-834, Sept. 1983.
[11] D.E. Goldberg and J.H. Holland Special Issue on Genetic Algorithms, Machine Learning, 2/3, 1988.
[12] R.C. Gonzalez, and P. Wintz, Digital Image Processing. Addison-Wesley Publishing Co., 1977.
[13] R.M. Haralick and L.G. Shapiro, "Image Segmentation Techniques," Computer Vision, Graphics, and Image Processing, vol. 29, pp. 100-132, 1985.
[14] H.G. John, R. Kohavi, and K. Pfleger, "Irrelevant Features and the Subet Selection Problem," Proc. 11th Int'l Conf. Machine Learning, pp. 121-129, 1994.
[15] K. Laws, "The Phoenix Image Segmentation System: Description and Evaluation," SRI Int'l Tech. Rep. TR289, Dec. 1982.
[16] Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel, "Backpropagation Applied to Handwritten Zip Code Recognition," Neural Computation, vol. 1, pp. 541-551, 1989.
[17] J.L. Marroquin and F. Girosi, "Some Extensions of the K-Means Algorithm for Image Segmentation and Pattern Classification," A.I. Memo No. 1390, MIT AI Lab, 1993.
[18] K.S. Narendra and M.A.L. Thathatchar, Learning Automata: An Introduction.Englewood Cliffs, N.J.: Prentice Hall, 1989.
[19] R. Ohlander, K. Price, and D. R. Reddy, "Picture Segmentation Using a Recursive Region Splitting Method," Computer Graphics and Image Processing, vol. 8, pp.313-333, 1978.
[20] J. Peng and B. Bhanu, "Delayed Reinforcement Learning for Closed-Loop Object Recognition," Proc. DARPA Image Understanding Workshop, pp. 1,429-1,435, Feb. 1996.
[21] V. Ramesh, "Performance Characterization of Image Understanding Algorithms," PhD Thesis, Dept. of Electrical Eng., Univ. of Washington, Seattle, Washington, 1995.
[22] S. Shafer and T. Kanade, "Recursive Region Segmentation by Analysis of Histograms," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 1,166-1,171, 1982.
[23] P. Suetens, P. Fua, and A.J. Hanson, "Computational Strategies for Object Recognition," ACM Computing Surveys, vol. 24, no. 1, pp. 5-59, 1992.
[24] S. Wang and T. Binford, "Local Step Edge Estimation—A New Algorithm, Statistical Model, and Performance Evaluation," Proc. ARPA Image Understanding Workshop, pp. 1,063-1,070, Apr. 1993.
[25] R.J. Williams, "Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning," Machine Learning, vol. 8, pp. 229-256, 1992.
[26] R.J. Williams and J. Peng, "Function Optimization Using Connectionist Reinforcement Learning Algorithms," Connection Science, vol. 3, no. 3, 1991.

Index Terms:
Adaptive color image segmentation, function optimization, generalized learning automata, learning in computer vision, model-based object recognition, multiscenario recognition, parameter learning, recognition feedback, segmentation evaluation.
Jing Peng, Bir Bhanu, "Closed-Loop Object Recognition Using Reinforcement Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 2, pp. 139-154, Feb. 1998, doi:10.1109/34.659932
Usage of this product signifies your acceptance of the Terms of Use.