This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Fast k-Nearest Neighbor Classification Using Cluster-Based Trees
April 2004 (vol. 26 no. 4)
pp. 525-528

Abstract—Most fast k{\hbox{-}}{\rm{nearest}} neighbor (k{\hbox{-}}{\rm{NN}}) algorithms exploit metric properties of distance measures for reducing computation cost and a few can work effectively on both metric and nonmetric measures. We propose a cluster-based tree algorithm to accelerate k{\hbox{-}}{\rm{NN}} classification without any presuppositions about the metric form and properties of a dissimilarity measure. A mechanism of early decision making and minimal side-operations for choosing searching paths largely contribute to the efficiency of the algorithm. The algorithm is evaluated through extensive experiments over standard NIST and MNIST databases.

[1] T.M. Cover and P. Hart, "Nearest Neighbor Pattern Classification," Proc. IEEE Trans. Information Theory, pp. 21-27, 1967.
[2] G.L. Ritter, H.B. Woodruff, and S.R. Lowry, “An Algorithm for a Selective Nearest Neighbor Decision Rule,” IEEE Trans. Information Theory, vol. 21, no. 6, pp. 665-669, 1975.
[3] C.L. Chang, Finding Prototypes for Nearest Neighbor Decision Rule IEEE Trans. Computers, vol. 23, no. 11, pp. 1179-1184, Nov. 1974.
[4] P.E. Hart, “The Condensed Nearest Neighbor Rule,” IEEE Trans. Information Theory, vol. 14, no. 3, pp. 515-516, 1968.
[5] D.W. Jacobs, D. Weinshall, and Y. Gdalyahu, Classification with Nonmetric Distances: Image Retrieval and Class Representation IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 583-600, June 2000.
[6] A.J. Broder, Strategies for Efficient Incremental Nearest Neighbor Search Pattern Recognition, vol. 23, nos. 1/2, pp. 171-178, Nov. 1986.
[7] A. Farago, T. Linder, and G. Lubosi, "Fast Nearest-Neighbor Search in Dissimilarity Spaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 9, pp. 957-962, Sept. 1993.
[8] J.H. Friedman, F. Baskett, and L.J. Shustek, An Algorithm for Finding Nearest Neighbors IEEE Trans. Computers, vol. 24, no. 10, pp. 1000-1006, Oct. 1975.
[9] K. Fukunaga and P.M. Narendra, A Branch and Bound Algorithm for Computing k-Nearest Neighbors IEEE Trans. Computers, vol. 24, no. 7, pp. 750-753, July 1975.
[10] B.S. Kim and S.B. Park, A Fast k Nearest Neighbor Finding Algorithm Based on the Ordered Partition IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 761-766, Nov. 1986.
[11] E. Vidal, An Algorithm for Finding Nearest Neighbors in (Approxinately) Constant Average Time Pattern Recognition Letters, vol. 4, no. 3, pp. 145-157, July 1986.
[12] R.L. Brown, Accelerated Template Matching Using Template Trees Grown by Condensation IEEE Trans. Systems, Man, and Cybernetics, vol. 25, no. 3, pp. 523-528, Mar. 1995.
[13] S.A. Nene and S.K. Nayar, "A Simple Algorithm for Nearest Neighbor Search in High Dimensions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 9, Sept. 1997, pp. 989-1003.
[14] J.L. Bentley, Multidimensional Binary Search Trees Used for Associative Searching Comm. ACM, vol. 18, no. 9, pp. 509-517, Sept. 1975.
[15] M. Donahue, D. Geiger, R. Hummel, and T Liu, Sparse Representations for Image Decompositions with Occlusions Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 7-12, 1996.
[16] T. Hastie and R. Tibshirani, “Discriminant Adaptive Nearest Neighbor Classification,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 6, pp. 607-615, June 1996.
[17] J. Puzicha, Y. Rubner, C. Tomasi, and J. Buhmann, Empirical Evaluation of Dissimilarity Measures for Color and Texture Proc. Int'l Conf. Computer Vision, 1999.
[18] J.T. Favata and G. Srikantan, A Multiple Feature/Resolution Approach to Handprinted Character/Digit Recognition Proc. Int'l J. Imaging Systems and Technology, vol. 7, pp. 304-311, 1996.
[19] D. Hunttenlocher, G. Klanderman, and W. Rucklidge, Comparing Images Using Hausdorff Distance IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 1, pp. 1-14, Jan. 1997.
[20] A.K. Jain and D. Zongker, "Representation and Recognition of Handwritten Digits Using Deformable Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 12, pp. 1,386-1,390, Dec. 1997.
[21] J.D. Tubbs, A Note on Binary Template Matching Pattern Recognition, vol. 22, no. 4, pp. 359-365, 1989.
[22] B. Zhang and S.N. Srihari, Properties of Binary Vector Dissimilarity Measures Proc. JCIS Int'l Conf. Computer Vision, Pattern Recognition, and Image Processing, Sept. 2003.
[23] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-Based Learning Applied to Document Recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.

Index Terms:
Nearest neighbor classification, nonmetrics, metrics, cluster tree.
Citation:
Bin Zhang, Sargur N. Srihari, "Fast k-Nearest Neighbor Classification Using Cluster-Based Trees," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 4, pp. 525-528, April 2004, doi:10.1109/TPAMI.2004.1265868
Usage of this product signifies your acceptance of the Terms of Use.