This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Fast Nearest-Neighbor Algorithm Based on a Principal Axis Search Tree
September 2001 (vol. 23 no. 9)
pp. 964-976

—A new fast nearest-neighbor algorithm is described that uses principal component analysis to build an efficient search tree. At each node in the tree, the data set is partitioned along the direction of maximum variance. The search algorithm efficiently uses a depth-first search and a new elimination criterion. The new algorithm was compared to 16 other fast nearest-neighbor algorithms on three types of common benchmark data sets including problems from time series prediction and image vector quantization. This comparative study illustrates the strengths and weaknesses of all of the leading algorithms. The new algorithm performed very well on all of the data sets and was consistently ranked among the top three algorithms.

[1] S.H. Chen and J.S. Pan, “Fast Search Algorithm for VQ-Based Recognition of Isolated Words,” IEE Proc. I (Comm., Speech, and Vision), vol. 136, no. 6, pp. 391-396, Dec. 1989.
[2] T.-S. Chen and C.-C. Chang, “Diagonal Axes Method (DAM): A Fast Search Algorithm for Vector Quantization,” IEEE Trans. Circuits and Systems for Video Technology, vol. 7, no. 3, pp. 555-559, June 1997.
[3] C. Atkeson, W. Moore, and S. Schaal, "Locally Weighted Learning," AI Rev., vol. 11, nos. 1-5, 1997, pp. 11-73.
[4] J. McNames, “A Nearest Trajectory Strategy for Time Series Prediction,” Proc. Int'l Workshop Advanced Black-Box Techniques for Nonlinear Modeling, pp. 112-128, July 1998.
[5] C.Y. Chen, C.C. Chang, and R.C.T. Lee, “A Near Pattern-Matching Scheme Based Upon Principal Component Analysis,” Pattern Recognition Letters, vol. 16, pp. 339-345, Apr. 1995.
[6] S.G. Bakamidis, “An Exact Fast Nearest Neighbor Identification Technique,” IEEE Int'l Conf. Acoustics, Speech and Signal Processing, vol. 5, pp. 658-661, 1993.
[7] D.-Y. Cheng, A. Gersho, B. Ramamurthi, and Y. Shoham, “Fast Search Algorithms for Vector Quantization and Pattern Matching,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 1, pp. 9.11.1-9.11.4, Mar. 1984.
[8] S.A. Nene and S.K. Nayar, "A Simple Algorithm for Nearest Neighbor Search in High Dimensions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 9, Sept. 1997, pp. 989-1003.
[9] J. McNames, “Innovations in Local Modeling for Time Series Prediction,” PhD thesis, Stanford Univ., 1999.
[10] K. Fukunaga and P.M. Narendra, “A Branch and Bound Algorithm for Computing k-Nearest Neighbors,” IEEE Trans. Computers, vol. 24, no. 7, pp. 750-753, July 1975.
[11] J.H. Friedman, J.L. Bentley, and R.A. Finkel, "An Algorithm for Finding Best Matches in Logarithmic Expected Time," ACM Trans. on Math. Software, vol. 3, no. 3, pp. 209-226, Sept. 1977.
[12] B.S. Kim and S.B. Park, "A Fast k Nearest Neighbor Finding Algorithm Based on the Ordered Partition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 761-766, Nov. 1986.
[13] L. Micó, J. Oncina, and R.C. Carrasco, “A Fast Branch&Bound Nearest Neighbor Classifier in Metric Spaces,” Pattern Recognition Letters, vol. 17, pp. 731-739, 1996.
[14] I. Katsavounidis, C.-C.J. Kuo, and Z. Zhang, “Fast Tree-Structured Nearest Neighbor Encoding for Vector Quantization,” IEEE Trans. Image Processing, vol. 5, no. 2, pp. 398-404, Feb. 1996.
[15] S.C. Tai, C.C. Lai, and Y.C. Lin, “Two Fast Nearest Neighbor Searching Algorithms for Image Vector Quantization,” IEEE Trans. Comm., vol. 44, no. 12, pp. 1623-1628, Dec. 1996.
[16] Y.-C. Lin and S.-C. Tai, “Dynamic Windowed Codebook Search Algorithm in Vector Quantization,” Optical Eng., vol. 35, no. 10, pp. 2921-292, Oct. 1996.
[17] G. Golub and C. Van Loan, Matrix Computations, third ed. Baltimore: Johns Hopkins Univ. Press, 1996.
[18] P. Fränti, T. Kaukoranta, and O. Nevalainen, “On the Splitting Method for Vector Quantization Codebook Generation,” Optical Eng., vol. 36, no. 11, pp. 3043-3051, Nov. 1997.
[19] S. Lubiarz and P. Lockwood, “Evaluation of Fast Algorithms for Finding the Nearest Neighbor,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 2, pp. 1491-1494, Apr. 1997.
[20] A.S. Weigend and N.A. Gershenfeld, Time Series Prediction. Addison-Wesley, 1994.
[21] G. Poggi, "Fast Algorithm for Full-Search VQ Encoding," Electron. Lett., vol. 29, pp. 1,141-1,142, June 1993.
[22] S.J. Baek, B.K. Jeon, and K.-M. Sung, “A Fast Encoding Algorithm for Vector Quantization,” IEEE Signal Processing Letters, vol. 4, no. 12, pp. 325-327, Dec. 1997.
[23] M. Reza Soleymani and S.D. Morgera, “An Efficient Nearest Neighbor Search Method,” Proc. IEEE Trans. Comm., vol. 35, no. 6, pp. 677-679, June 1987.
[24] K.-S. Wu and J.-C. Lin, “An Efficient Nearest Neighbor Searching Algorithm with Application to LBG Codebook Generation,” J. Chinese Inst. Eng., vol. 19, no. 6, pp. 719-724, 1996.

Citation:
J. McNames, "A Fast Nearest-Neighbor Algorithm Based on a Principal Axis Search Tree," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 9, pp. 964-976, Sept. 2001, doi:10.1109/34.955110
Usage of this product signifies your acceptance of the Terms of Use.