The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - January (2010 vol.32)
pp: 105-119
Edward Rosten , Cambridge University, Cambridge
Reid Porter , Los Alamos National Laboratory, Los Alamos
Tom Drummond , Cambridge University, Cambridge
ABSTRACT
The repeatability and efficiency of a corner detector determines how likely it is to be useful in a real-world application. The repeatability is important because the same scene viewed from different positions should yield features which correspond to the same real-world 3D locations [1]. The efficiency is important because this determines whether the detector combined with further processing can operate at frame rate. Three advances are described in this paper. First, we present a new heuristic for feature detection and, using machine learning, we derive a feature detector from this which can fully process live PAL video using less than 5 percent of the available processing time. By comparison, most other detectors cannot even operate at frame rate (Harris detector 115 percent, SIFT 195 percent). Second, we generalize the detector, allowing it to be optimized for repeatability, with little loss of efficiency. Third, we carry out a rigorous comparison of corner detectors based on the above repeatability criterion applied to 3D scenes. We show that, despite being principally constructed for speed, on these stringent tests, our heuristic detector significantly outperforms existing feature detectors. Finally, the comparison demonstrates that using machine learning produces significant improvements in repeatability, yielding a detector that is both very fast and of very high quality.
INDEX TERMS
Corner detection, feature detection.
CITATION
Edward Rosten, Reid Porter, Tom Drummond, "Faster and Better: A Machine Learning Approach to Corner Detection", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 1, pp. 105-119, January 2010, doi:10.1109/TPAMI.2008.275
REFERENCES
[1] C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of Interest Point Detectors,” Int'l J. Computer Vision, vol. 37, no. 2, pp. 151-172, 2000.
[2] A. Rosenfeld and E. Johnston, “Angle Detection on Digital Curves,” IEEE Trans. Computers, vol. 22, no. 9, pp. 875-878, Sept. 1973.
[3] A. Rosenfeld and J.S. Weszka, “An Improved Method of Angle Detection on Digital Curves,” IEEE Trans. Computers, vol. 24, no. 9, pp. 940-941, Sept. 1975.
[4] H. Freeman and L.S. Davis, “A Corner-Finding Algorithm for Chain-Coded Curves,” IEEE Trans. Computers, vol. 26, no. 3, pp.297-303, Mar. 1977.
[5] H.L. Beus and S.S.H. Tiu, “An Improved Corner Detection Algorithm Based on Chain-Coded Plane Curves,” Pattern Recognition, vol. 20, no. 3, pp. 291-296, 1987.
[6] L. O'Gorman, “Curvilinear Feature Detection from Curvature Estimation,” Proc. Ninth Int'l Conf. Pattern Recognition, pp. 1116-1119, 1988.
[7] C.-H. Teh and R. Chin, “On the Detection of Dominant Points on Digital Curves,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 8, pp. 859-872, Aug. 1989.
[8] H. Ogawa, “Corner Detection on Digital Curves Based on Local Symmetry of the Shape,” Pattern Recognition, vol. 22, no. 4, pp. 351-357, 1989.
[9] A. Bandera, C. Urdiales, F. Arrebola, and E. Sandoval, “Corner Detection by Means of Adaptively Estimated Curvature Function,” Electronics Letters, vol. 36, no. 2, pp. 124-126, 2000.
[10] C. Urdiales, C. Trazegnies, A. Bandera, and E. Sandoval, “Corner Detection Based on Adaptively Filtered Curvature Function,” Electronics Letters, vol. 32, no. 5, pp. 426-428, 2003.
[11] K. Sohn, W.E. Alexander, J.H. Kim, Y. Kim, and W.E. Snyder, “Curvature Estimation and Unique Corner Point Detection for Boundary Representation,” Proc. IEEE Int'l Conf. Robotics and Automation, vol. 2, pp. 1590-1595, 1992.
[12] X. He and N. Yung, “Curvature Scale Space Corner Detector with Adaptive Threshold and Dynamic Region of Support,” Proc. 17th Int'l Conf. Pattern Recognition, pp. 791-794, 2004.
[13] N. Ansari and E.J. Delp, “On Detecting Dominant Points,” Pattern Recognition, vol. 24, no. 5, pp. 441-451, 1991.
[14] A. Rattarangsi and R.T. Chin, “Scale-Based Detection of Corners of Planar Curves,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 4, pp. 430-449, Apr. 1992.
[15] J. Lee, Y. Sun, and C. Chen, “Wavelet Transform for Corner Detection,” Proc. IEEE Conf. Systems Eng., 1992.
[16] J.-S. Lee, Y.-N. Sun, and C.-H. Chen, “Multiscale Corner Detection by Using Wavelet Transform,” IEEE Trans. Image Processing, vol. 4, no. 1, pp. 100-104, Jan. 1995.
[17] A. Quddus and M. Fahmy, “Fast Wavelet-Based Corner Detection Technique,” Electronics Letters, vol. 35, no. 4, pp. 287-288, 1999.
[18] F. Mokhtarian and R. Suomela, “Robust Image Corner Detection through Curvature Scale Space,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1376-1381, Dec. 1998.
[19] P. Saint-Marc, J.-S. Chen, and G. Medioni, “Adaptive Smoothing: A General Tool for Early Vision,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 6, pp. 514-529, June 1991.
[20] B.K. Ray and R. Pandyan, “Acord—An Adaptive Corner Detector for Planar Curves,” Pattern Recognition Letters, vol. 36, no. 3, pp.703-708, 2003.
[21] D.J. Langridge, “Curve Encoding and Detection of Discontinuities,” Computer Vision, Graphics, and Image Processing, vol. 20, no. 1, pp. 58-71, 1987.
[22] G. Medioni and Y. Yasumoto, “Corner Detection and Curve Representation Using Cubic B-Splines,” Computer Vision, Graphics, and Image Processing, vol. 39, no. 3, pp. 279-290, 1987.
[23] D.J. Beymer, “Finding Junctions Using the Image Gradient,” Proc. Sixth IEEE Conf. Computer Vision and Pattern Recognition, pp. 720-721, 1991.
[24] U. Seeger and R. Seeger, “Fast Corner Detection in Grey-Level Images,” Pattern Recognition Letters, vol. 15, no. 7, pp. 669-675, 1994.
[25] F. Arrebola, A. Bandera, P. Camacho, and F. Sandoval, “Corner Detection by Local Histograms of Contour Chain Code,” Electronics Letters, vol. 33, no. 21, pp. 1769-1771, 1997.
[26] F. Arrebola, A. Bandera, P. Camacho, and F. Sandoval, “Corner Detection and Curve Representation by Circular Histograms of Contour Chain Code,” Electronics Letters, vol. 35, no. 13, pp. 1065-1067, 1999.
[27] L. Li, “Corner Detection and Interpretation on Planar Curves Using Fuzzy Reasoning,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 11, pp. 1204-1210, Nov. 1999.
[28] P. Sankar and C. Sharma, “A Parallel Procedure for the Detection of Dominant Points on a Digital Curve,” Computer Graphics and Image Processing, vol. 7, no. 4, pp. 403-412, 1978.
[29] F.-H. Cheng and W.-H. Hsu, “Parallel Algorithm for Corner Finding on Digital Curves,” Pattern Recognition Letters, vol. 8, no. 1, pp. 47-53, 1988.
[30] D.H. Ballard, “Generalizing the Hough Transform to Detect Arbitrary Shapes,” Pattern Recognition, vol. 13, no. 2, pp. 111-122, 1981.
[31] E.R. Davies, “Application of the Generalised Hough Transform to Corner Detection,” Proc. IEE Computers and Digital Techniques, vol. 135, no. 1, pp. 49-54, 1988.
[32] R.M. Haralick and L.G. Shapiro, Computer and Robot Vision, vol. 1, Addison-Wesley, 1993.
[33] R. Mehrotra, S. Nichani, and N. Ranganathan, “Corner Detection,” Pattern Recognition, vol. 23, no. 11, pp. 1223-1233, 1990.
[34] J. Cooper, S. Venkatesh, and L. Kitchen, “The Dissimilarity Corner Detector,” Proc. Fifth Int'l Conf. Advanced Robotics, pp. 1377-1382, 1991.
[35] L. Kitchen and A. Rosenfeld, “Gray-Level Corner Detection,” Pattern Recognition Letters, vol. 1, no. 2, pp. 95-102, 1982.
[36] A. Singh and M. Shneier, “Grey Level Corner Detection: A Generalization and a Robust Real Time Implementation,” Computer Vision, Graphics, and Image Processing, vol. 51, no. 1, pp. 54-69, 1990.
[37] O. Zuniga and R. Haralick, “Corner Detection Using the Facet Model,” Proc. First IEEE Conf. Computer Vision and Pattern Recognition, pp. 30-37, 1983.
[38] R. Deriche and G. Giraudon, “A Computational Approach for Corner and Vertex Detection,” Int'l J. Computer Vision, vol. 10, no. 2, pp. 101-124, 1993.
[39] H. Wang and M. Brady, “Real-Time Corner Detection Algorithm for Motion Estimation.” Image and Vision Computing, vol. 13, no. 9, pp. 695-703, 1995.
[40] P. Beaudet, “Rotational Invariant Image Operators.” Proc. Fourth Int'l Conf. Pattern Recognition, pp. 579-583, 1978.
[41] G. Giraudon and R. Deriche, “On Corner and Vertex Detection,” Proc. Sixth IEEE Conf. Computer Vision and Pattern Recognition, pp.650-655, 1991.
[42] L. Dreschler and H.-H. Nagel, “Volumetric Model and 3D Trajectory of a Moving Car from Monocular TV Frames Sequence of a Street Scene,” Computer Graphics and Image Processing, vol. 20, no. 3, pp. 199-228, 1982.
[43] B. Luo, A.D.J. Cross, and E.R. Hancock, “Corner Detection via Topographic Analysis of Vector Potential,” Proc. Ninth British Machine Vision Conf., 1998.
[44] H. Moravec, “Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover,” Technical Report CMU-RI-TR-80-03, Robotics Inst., Carnegie Mellon Univ., 1980, also doctoral dissertation, Stanford Univ., available as Stanford AIM-340, CS-80-813.
[45] C. Harris and M. Stephens, “A Combined Corner and Edge Detector,” Proc. Alvey Vision Conf., pp. 147-151, 1988.
[46] E. Rosten, “High Performance Rigid Body Tracking,” PhD dissertation, Univ. of Cambridge, Feb. 2006.
[47] W. Förstner, “A Feature-Based Correspondence Algorithm for Image Matching,” Int'l Archive Photogrammetry and Remote Sensing, vol. 26, pp. 150-166, 1986.
[48] C. Tomasi and T. Kanade, “Detection and Tracking of Point Features,” Technical Report CMU-CS-91-132, Carnegie Mellon Univ., 1991.
[49] J. Shi and C. Tomasi, “Good Features to Track,” Proc. Ninth IEEE Conf. Computer Vision and Pattern Recognition, 1994.
[50] J.A. Noble, “Descriptions of Image Surfaces.” PhD dissertation, Dept. of Eng. Science, Univ. of Oxford, 1989.
[51] C.S. Kenney, B.S. Manjunath, M. Zuliani, M.G.A. Hewer, and A.V. Nevel, “A Condition Number for Point Matching with Application to Registration and Postregistration Error Estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 11, pp.1437-1454, Nov. 2003.
[52] M. Zuliani, C. Kenney, and B. Manjunath, “A Mathematical Comparison of Point Detectors,” Proc. Second IEEE Image and Video Registration Workshop, 2004.
[53] C. Kenney, M. Zuliani, and B. Manjunath, “An Axiomatic Approach to Corner Detection,” Proc. 18th IEEE Conf. Computer Vision and Pattern Recognition, pp. 191-197, 2005.
[54] K. Rohr, “On 3D Differential Operators for Detecting Point Landmarks,” Image and Vision Computing, vol. 15, no. 3, pp. 219-233, 1997.
[55] J.A. Noble, “Finding Corners,” Image and Vision Computing, vol. 6, no. 2, pp. 121-128, 1988.
[56] B. Triggs, “Detecting Keypoints with Stable Position, Orientation and Scale under Illumination Changes,” Proc. Eighth European Conf. Computer Vision, vol. 4, pp. 100-113, 2004.
[57] K. Mikolajczyk and C. Schmid, “An Affine Invariant Interest Point Detector,” Proc. European Conf. Computer Vision, pp. 128-142, 2002.
[58] D.G. Lowe, “Distinctive Image Features from Scale Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[59] J.L. Crowley, O. Riff, and J.H. Piater, “Fast Computation of Characteristic Scale Using a Half Octave Pyramid,” Proc. Fourth Int'l Conf. Scale-Space Theories in Computer Vision, 2003.
[60] K. Mikolajczyk and C. Schmid, “Indexing Based on Scale Invariant Interest Points,” Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 1, pp. 525-531, 2001.
[61] M. Brown and D.G. Lowe, “Invariant Features from Interest Point Groups.” Proc. 13th British Machine Vision Conf., pp. 656-665, 2002.
[62] F. Schaffalitzky and A. Zisserman, “Viewpoint Invariant Texture Matching and Wide Baseline Stereo,” Proc. Eighth IEEE Int'l Conf. Computer Vision, pp. 636-643, 2001.
[63] F. Schaffalitzky and A. Zisserman, “Multi-View Matching for Unordered Image Sets, or How Do I Organise My Holiday Snaps?” Proc. Seventh European Conf. Computer Vision, pp. 414-431, 2002.
[64] A. Guiducci, “Corner Characterization by Differential Geometry Techniques,” Pattern Recognition Letters, vol. 8, no. 5, pp. 311-318, 1988.
[65] K. Rohr, “Recognizing Corners by Fitting Parametric Models,” Int'l J. Computer Vision, vol. 9, no. 3, pp. 213-230, 1992.
[66] P.L. Rosin, “Measuring Corner Properties,” Computer Vision and Image Understanding, vol. 73, no. 2, pp. 291-307, 1999.
[67] J. Canny, “A Computational Approach to Edge Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp.679-698, Nov. 1986.
[68] K. Rangarajan, M. Shah, and D. van Brackle, “Optimal Corner Detection,” Proc. Second IEEE Int'l Conf. Computer Vision, pp. 90-94, 1988.
[69] S.-T. Liu and W.-H. Tsai, “Moment-Preserving Corner Detection,” Pattern Recognition, vol. 23, no. 5, pp. 441-460, 1990.
[70] S. Ghosal and R. Mehrotra, “Zernike Moment-Based Feature Detectors,” Proc. First Int'l Conf. Image Processing, vol. 1, pp. 934-938, 1994.
[71] F. Shen and H. Wang, “Real Time Gray Level Corner Detector,” Proc. Sixth Int'l Conf. Control, Automation, Robotics, and Vision, 2000.
[72] R.O. Duda and P.E. Hart, “Use of the Hough Transformation to Detect Lines and Curves in Pictures,” Comm. ACM, vol. 15, no. 1, pp. 11-15, 1972.
[73] F. Shen and H. Wang, “Corner Detection Based on Modified Hough Transform,” Pattern Recognition Letters, vol. 32, no. 8, pp.1039-1049, 2002.
[74] B. Luo and D. Pycock, “Unified Multi-Scale Corner Detection,” Proc. Fourth IASTED Int'l Conf. Visualisation, Imaging and Image Processing, 2004.
[75] X. Xie, R. Sudhakar, and H. Zhuang, “Corner Detection by a Cost Minimization Approach,” Pattern Recognition, vol. 26, no. 8, pp.1235-1243, 1993.
[76] S.M. Smith and J.M. Brady, “SUSAN—A New Approach to Low Level Image Processing,” Int'l J. Computer Vision, vol. 23, no. 1, pp.45-78, 1997.
[77] S.C. Bae, I.S. Kweon, and C.D. Yoo, “Cop: A New Corner Detector,” Pattern Recognition Letters, vol. 23, no. 11, pp. 1349-1360, 2002.
[78] M. Trajković and M. Hedley, “Fast Corner Detection,” Image and Vision Computing, vol. 16, no. 2, pp. 75-87, 1998.
[79] V. Lepetit and P. Fua, “Keypoint Recognition Using Randomized Trees,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1465-1479, Sept. 2006.
[80] Z.-Q. Wu and A. Rosenfeld, “Filtered Projections as an Aid in Corner Detection,” Pattern Recognition, vol. 16, no. 1, pp. 31-38, 1983.
[81] K. Paler, J. Föglein, J. Illingworth, and J. Kittler, “Local Ordered Grey Levels as an Aid to Corner Detection,” Pattern Recognition, vol. 17, no. 5, pp. 535-543, 1984.
[82] B. Robbins and R. Owens, “2D Feature Detection via Local Energy,” Image and Vision Computing, vol. 15, no. 5, pp. 353-368, 1997.
[83] G. Loy and A. Zelinsky, “A Fast Radial Symmetry Transform for Detecting Points of Interest,” Proc. Seventh European Conf. Computer Vision, pp. 358-368, 2002.
[84] P. Dias, A. Kassim, and V. Srinivasan, “A Neural Network Based Corner Detection Method,” Proc. IEEE Int'l Conf. Neural Networks, vol. 4, pp. 2116-2120, 1995.
[85] W.-C. Chen and P. Rockett, “Bayesian Labelling of Corners Using a Grey-Level Corner Image Model,” Proc. Fourth Int'l Conf. Image Processing, pp. 687-690, 1997.
[86] W. Kienzle, F.A. Wichmann, B. Schölkopf, and M.O. Franz, “Learning an Interest Operator from Human Eye Movements,” Proc. 18th IEEE Conf. Computer Vision and Pattern Recognition Workshop, 2005.
[87] L. Trujillo and G. Olague, “Synthesis of Interest Point Detectors through Genetic Programming,” Proc. Eighth Ann. Conf. Genetic and Evolutionary Computation, pp. 887-894, 2006.
[88] P. Rajan and J. Davidson, “Evaluation of Corner Detection Algorithms,” Proc. 21st Southeastern Symp. System Theory, pp. 29-33, 1989.
[89] J. Cooper, S. Venkatesh, and L. Kitchen, “Early Jump-Out Corner Detectors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 8, pp. 823-828, Aug. 1993.
[90] X. Zhang, R. Haralick, and V. Ramesh, “Corner Detection Using the Map Technique,” Proc. 12th Int'l Conf. Pattern Recognition, vol. 1, pp. 549-552, 1994.
[91] F. Mohannah and F. Mokhtarian, “Performance Evaluation of Corner Detection Algorithms under Affine and Similarity Transforms.” Proc. 12th British Machine Vision Conf., T.F. Cootes and C.Taylor, eds., 2001.
[92] P. Tissainayagam and D. Suter, “Assessing the Performance of Corner Detectors for Point Feature Tracking Applications,” Image and Vision Computing, vol. 22, no. 8, pp. 663-679, 2004.
[93] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005.
[94] P. Moreels and P. Perona, “Evaluation of Features Detectors and Descriptors Based on 3D Objects,” Int'l J. Computer Vision, pp. 263-284, 2007.
[95] E. Rosten and T. Drummond, “Fusing Points and Lines for High Performance Tracking,” Proc. 10th IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1508-1515, 2005.
[96] E. Rosten, G. Reitmayr, and T. Drummond, “Real-Time Video Annotations for Augmented Reality,” Proc. Int'l Symp. Visual Computing, 2005.
[97] E. Rosten and T. Drummond, “Machine Learning for High Speed Corner Detection,” Proc. Ninth European Conf. Computer Vision, vol. 1, pp. 430-443, 2006.
[98] J.R. Quinlan, “Induction of Decision Trees,” Machine Learning, vol. 1, pp. 81-106, 1986.
[99] W.H. Press, S.A. Teukolsky, W.H. Vetterling, and B.P. Flannery, Numerical Recipes in C. Cambridge Univ. Press, 1999.
[100] http://www.robots.ox.ac.uk/~vgg/datadata-aff.html , 2007.
[101] S.M. Smith, http://www.fmrib.ox.ac.uk/~steve/susansusan2l.c , 2005.
[102] C. Schmid, R. Mohr, and C. Bauckhage, “Comparing and Evaluating Interest Points,” Proc. Sixth IEEE Int'l Conf. Computer Vision, pp. 230-235, 1998.
[103] D.G. Lowe, “Demo Software: Sift Keypoint Detector,” http://www.cs.ubc.ca/~lowekeypoints/, 2005.
[104] B. Sklar, Digital Communications. Prentice Hall, 1988.
35 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool