This Article 
 Bibliographic References 
 Add to: 
Joint Induction of Shape Features and Tree Classifiers
November 1997 (vol. 19 no. 11)
pp. 1300-1305

Abstract—We introduce a very large family of binary features for two-dimensional shapes. The salient ones for separating particular shapes are determined by inductive learning during the construction of classification trees. There is a feature for every possible geometric arrangement of local topographic codes. The arrangements express coarse constraints on relative angles and distances among the code locations and are nearly invariant to substantial affine and nonlinear deformations. They are also partially ordered, which makes it possible to narrow the search for informative ones at each node of the tree. Different trees correspond to different aspects of shape. They are statistically weakly dependent due to randomization and are aggregated in a simple way. Adapting the algorithm to a shape family is then fully automatic once training samples are provided. As an illustration, we classify handwritten digits from the NIST database; the error rate is .7 percent.

[1] R.A. Wilkinson, J. Geist, S. Janet, P. Grother, C. Gurges, R. Creecy, B. Hammond, J. Hull, N. Larsen, T. Vogl, and C. Wilson, "The First Census Optical Character Recognition System Conference," Technical Report, Nat'l Inst. of Standards and Technology, NISTIR4912, Gaithersburg, Md., 1992.
[2] L. Bottou, C. Cortes, J.S. Denker, H. Drucker, I. Guyon, L.D. Jackel, Y. LeCun, U.A. Muller, E. Sackinger, P. Simard, and V. Vapnik, "Comparison of Classifier Methods: A Case Study in Handwritten Digit Recognition," Proc. IEEE Int'l Conf. Pattern Recognition, pp. 77-82, 1994.
[3] Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel, "Handwritten Digit Recognition With a Back-Propagation Network," D.S. Touresky, ed., Advances in Neural Information, vol. 2. San Mateo, Calif.: Morgan Kauffman, 1990.
[4] B. Boser, I. Guyon, and I. Vapnik, "A Training Algorithm for Optimal Margin Classifiers," Proc. COLT II,Philadelphia, 1992.
[5] T. Hastie, A. Buja, and R. Tibshirani, "Penalized Discriminant Analysis," Annals of Statistics, vol. 23, pp. 73-103, 1995.
[6] Z.M. Kovacs-V and R. Guerrieri, "Massively-Parallel Handwritten Character Recognition Based on the Distance Transform," Pattern Recognition, vol. 28, pp. 293-301, 1995.
[7] P.Y. Simard, Y.L. LeCun, and J.S. Denker, "Memory-Based Character Recognition Using a Transformation Invariant Metric," IEEE Int'l Conf. Pattern Recognition, pp. 262-267, 1994.
[8] S. Smith, M.O. Bourgoin, K. Sims, and H.L. Voorhees, "Handwritten Character Classification Using Nearest Neighbor in Large Databases," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, pp. 915-919, 1994.
[9] S. Shlien, "Multiple Binary Decision Tree Classifiers," Pattern Recognition, vol. 23, pp. 757-763, 1990.
[10] Q.R. Wang and C.Y. Suen, "Analysis and Design of a Decision Tree Based on Entropy Reduction and Its Application to Large Character Set Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, pp. 406-417, 1984.
[11] T.K. Ho, J.J. Hull, and S.N. Srihari, “Decision Combination in Multiple Classifiers Systems,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 1, pp. 66-75, Jan. 1994.
[12] Y.S. Huang and C.Y. Suen, A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, pp. 90-93, 1998.
[13] T.H. Reiss, "Recognizing Planar Objects Using Invariant Image Features," Lecture Notes in Computer Science 676. New York: Springer-Verlag, 1993.
[14] J.L. Mundy and A. Zisserman, Geometric Invariance in Computer Vision.Cambridge, Mass.: MIT Press, 1992.
[15] M. Sabourin and A. Mitiche, "Optical Character Recognition by a Neural Network," Neural Networks, vol. 5, pp. 843-852, 1992.
[16] K. Cho and S.M. Dunn, “Learning Shape Classes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 9, pp. 882-887, Sept. 1994.
[17] Y. Amit and D. Geman, "Shape Quantization and Recognition With Randomized Trees," Neural Computation, vol. 9, pp. 1,545-1,587, 1997.
[18] B. Jedynak and F. Fleuret, "3D Object Recognition From Geometric Queries," Proc. Image'Com 96,Bordeaux, France, 1996.
[19] K.J. Wilder, "Decision Tree Algorithms for Handwritten Digit Recognition," Dept. Mathematics and Statistics, Univ. Massachusetts, Amherst, 1997.
[20] M.D. Garris and R.A. Wilkinson, "NIST Special Database 3: Handwritten Segmented Characters," NIST, Gaithersburg, Md., no date.
[21] Y. Amit and D. Geman, "Randomized Inquiries About Shape; an Application to Handwritten Digit Recognition," Dept. Statistics, Univ. of Chicago, Technical Report 401, 1994.
[22] L. Breiman, "Bagging Predictors," Dept. of Statistics, Univ. of California, Berkeley, Technical Report 421, 1994.
[23] S.W. Kwok and C. Carter, "Multiple Decision Trees," R.D. Shachter, T.S. Levitt, L.N. Kanal, and J.F. Lemmer, eds., Uncertainty and Artificial Intelligence.New York: Elsevier Science Publishers, 1990.
[24] J.J. Oliver and D. Hand, "Averaging Over Decision Stumps," F. Bergadano and L. De Raedt, eds., Proc. ECML-1994.New York: Springer Verlag, 1994.
[25] T.G. Dietterich and G. Bakiri, "Solving Multiclass Learning Problems via Error-Correcting Output Codes," J. Artificial Intelligence Resolution, vol. 2, pp. 263-286, 1995.
[26] J.J. Hull, “A Database for Handwritten Text Recognition Research,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 5, pp. 550-554, May 1994.
[27] H. Drucker, R. Schapire, and P. Simard, "Boosting Performance in Neural Networks," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 7, pp. 705-719, 1993.
[28] L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees.Belmont, Calif.: Wadsworth, 1984.

Index Terms:
Shape quantization, feature induction, invariant arrangements, multiple decision trees, randomization, digit recognition, local topographic codes.
Yali Amit, Donald Geman, Kenneth Wilder, "Joint Induction of Shape Features and Tree Classifiers," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 11, pp. 1300-1305, Nov. 1997, doi:10.1109/34.632990
Usage of this product signifies your acceptance of the Terms of Use.