This Article 
 Bibliographic References 
 Add to: 
A Condition Number for Point Matching with Application to Registration and Postregistration Error Estimation
November 2003 (vol. 25 no. 11)
pp. 1437-1454

Abstract—Selecting salient points from two or more images for computing correspondence is a well-studied problem in image analysis. This paper describes a new and effective technique for selecting these tiepoints using condition numbers, with application to image registration and mosaicking. Condition numbers are derived for point-matching methods based on minimizing windowed objective functions for 1) translation, 2) rotation-scaling-translation (RST), and 3) affine transformations. Our principal result is that the condition numbers satisfy K_Trans \leq K_RST \leq K_Affine. That is, if a point is ill-conditioned with respect to point-matching via translation, then it is also unsuited for matching with respect to RST and affine transforms. This is fortunate since K_Trans is easily computed whereas K_RST and K_Affine are not. The second half of the paper applies the condition estimation results to the problem of identifying tiepoints in pairs of images for the purpose of registration. Once these points have been matched (after culling outliers using a RANSAC-like procedure), the registration parameters are computed. The postregistration error between the reference image and the stabilized image is then estimated by evaluating the translation between these images at points exhibiting good conditioning with respect to translation. The proposed method of tiepoint selection and matching using condition number provides a reliable basis for registration. The method has been tested on a large number of diverse collection of images—multidate Landsat images, aerial images, aerial videos, and infrared images. A Web site where the users can try our registration software is available and is being actively used by researchers around the world.

[1] K. Atkinson, An Introduction to Numerical Analysis, second ed. New York: John Wiley and Sons, 1989.
[2] H.S. Baird, Model-Based Image Matching Using Location. Cambridge, Mass.: MIT Press, 1985.
[3] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43–77, 1994.
[4] S.S. Beauchemin and J.L. Barron, The Computation of Optical Flow ACM Computing Surveys, vol. 27, no. 3, pp. 433-467, 1996.
[5] M. Ben-Ezra, S. Peleg, and M. Werman, Robust Real-Time Motion Analysis Proc. DARPA Image Understanding Workshop, pp. 207-210, 1988.
[6] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hiercharchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
[7] A. Bjorck and S. Hammarling, A Schur Method for the Square Root of a Matrix Linear Algebra Application, vols. 52/53, pp. 127-140, 1983.
[8] M. Brooks, W. Chojnacki, D. Gaeley, and A. can den Hengel, What Value Covariance Information in Estimating Vision Parameters? Proc. IEEE Eighth Int'l Conf. Computer Vision, pp. 302-308, 2001.
[9] L. Brown, “A Survey of Image Registration Techniques,” ACM Computing Surveys, vol. 24, no. 4, pp. 325-376, 1992.
[10] A. Cline, C. Moler, G. Stewart, and J. Wilkinson, An Estimate for the Condition Number of a Matrix SIAM J. Numerical Analysis, vol. 16 pp. 368-375, 1979.
[11] J. Demmel, On Condition Numbers and the Distance to the Nearest Ill-Posed Problem Numerische. Mathematik, vol. 51, pp. 251-289, 1987.
[12] J. Demmel, The Probability that a Numerical Analysis Problem is Difficult Math. Computation, vol. 50, no. 182, Apr. 1988.
[13] J. Demmel, B. Diament, and G. Malajovich, On the Complexity of Computing Error Bounds Foundations of Computation Math., vol. 1, pp. 101-125, 2001.
[14] B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans, SIAM CBMS-NSF Regional Conf. Series in Applied Math., sixth printing, 1994.
[15] M.A. Fischler and R.C. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Graphics and Image Processing, vol. 24, no. 6, pp. 381–395, June 1981.
[16] L. Fonseca, G. Hewer, C. Kenney, and B. Manjunath, Registration and Fusion of Multispectral Images Using a New Control Point Assessment Method Derived from Optical Flow Ideas Proc. SPIE, vol. 3717, pp. 104-111, Apr. 1999.
[17] L. Fonseca and C. Kenney, Control Point Assessment for Image Registration Proc. XII Brazilian Symp. Computer Graphics and Image Processing, pp. 125-132, Oct. 1999.
[18] G.H. Golub and C.F. Van Loan, Matrix Computations, second ed. Baltimore: John Hopkins Univ. Press, 1989.
[19] R.M. Haralick and L.G. Shapiro, Computer and Robot Vision. New York: Addison-Wesley, 1993.
[20] C. Harris and M. Stephens, A Combined Corner and Edge Detector Proc. Alvey Vision Conf., pp. 147-151, 1988.
[21] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[22] G. Hewer and C. Kenney, The Sensitivity of the Stable Lyapunov Equation SIAM J. Control and. Optimization, vol. 26, pp. 321-344, 1988.
[23] G. Hewer, C. Kenney, and W. Kuo, A Survey of Optical Flow Methods for Tracking Problems Proc. SPIE Conf., vol. 2242, pp. 561-572, Apr. 1994.
[24] G. Hewer, C. Kenney, W. Kuo, and L. Peterson, Curvature and Aggregate Velocity for Optical Flow Proc. SPIE Conf., July 1995.
[25] G. Hewer, C. Kenney, B. Manjunath, and A. Van Nevel, Condition Theory for Image Processing ECE techincal report, Univ. of Calif., Santa Barbara, Mar. 2002.
[26] E.C. Hildreth, Computations Underlying the Measurement of Visual Motion Artificial Intelligence, vol. 23, pp. 309-354, 1984.
[27] D. Hinrichsen and A.J. Pritchard, Stability Radii of Linear Systems System Control Letters, vol. 7, pp. 1-10, 1986.
[28] B.K.P. Horn and B. Schunck, Determining Optical Flow Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[29] B.K. Horn, Robot Vision. Cambridge, Mass.: MIT Press, 1986.
[30] R.A. Horn and C.R. Johnson, Matrix Analysis. Cambridge Univ. Press, 1990.
[31] M. Irani, B. Rousso, and S. Peleg, “Computing Occluding and Transparent Motions,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 5-16, Jan. 1994.
[32] M. Irani and P. Anandan, Robust Multi-Sensor Image Alignment Proc. IEEE Int'l Conf. Computer Vision, pp. 959-966, 1998.
[33] Y. Kanazawa and K. Kanatani, Do We Really Have to Consider Covariance Matrices for Image Features? Proc. IEEE Int'l Conf. Computer Vision, pp. 301-306, 2001.
[34] J. K. Kearney,W. B. Thompson,, and D. L. Boley,“Optical flow estimation: An error analysis of gradient-based methods with local optimization,” Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 2, pp 229-244, 1987.
[35] C. Kenney and G. Hewer, The Sensitivity of the Algebraic and Differential Riccati Equations SIAM J. Control and Optimization, vol. 28, pp. 50-69, 1990.
[36] C.S. Kenney and A.J. Laub, Condition Estimates for Matrix Functions SIAM J. Matrix Analysis and Application, vol. 10, pp. 191-209, 1989.
[37] C.S. Kenney and A.J. Laub, Small-Sample Statistical Condition Estimates for General Matrix Functions SIAM J. Scientific Computing, vol. 15, pp. 36-61, 1994.
[38] H. Li, B.S. Manjunath, and S.K. Mitra, A Contour Based Approach to Multisensor Image Registration IEEE Trans. Image Processing, pp. 320-334, 1995.
[39] B. Lucas and T. Kanade, An Iterative Image Registration Technique with an Application to Stereo Vision Proc. Image Understanding Workshop, pp. 121-130, 1981.
[40] D.G. Luenberger, Linear and Nonlinear Programming, second ed. Menlo Park, Calif.: Addison-Wesley, 1984.
[41] J. Tanner and C. Mead, Optical Motion Sensor Analog VLSI and Neural Systems, Carver Mead, ed., chapter 14, pp. 229-255, Reading, Mass.: Addison-Wesley Publishing Company, 1989.
[42] C.B. Moler and C.F. Van Loan, Nineteen Dubious Ways to Compute the Exponential of a Matrix SIAM Rev., vol. 20, pp. 801-836, 1978.
[43] J.R. Rice, A Theory of Condition SIAM J. Numerical Analysis, vol. 3, pp. 287-310, 1966.
[44] C. Schmid, R. Mohr, and C. Baukhage, Evaluation of Interest Point Detectors Int'l J. Computer Vision, vol. 27, pp. 151-172, 2000.
[45] J. Shi and C. Tomasi, Good Features to Track Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[46] T. Tommasini, A. Fusiello, E. Trucco, and V. Roberto, Making Good Features Track Better IEEE Proc. Conf. Computer Vision and Pattern Recognition, pp. 178-183, 1998.
[47] G.W. Stewart, Error and Perturbation Bounds for Subspaces Associated with Certain Eigenvalue Problems SIAM Rev., vol. 15, pp. 727-764, 1973.
[48] V. Sundareswaran and S. Mallat, Multiscale Optical Flow Computation with Wavelets Preprint. Courant Inst. of Math. Sciences, New York Univ. 1992.
[49] E. Trucco and A. Verri, Introductory Techniques for 3D Computer Vision. Upper Saddle River, N.J.: Prentice-Hall, 1998.
[50] B.C. Vemuri, S. Huang, S. Sahni, C.M. Leonard, C. Mohr, R. Gilmore, and J. Fitzsimmons, An Efficient Motion Estimator with Application to Medical Image Registration Medical Image Analysis, vol. 2, no. 1, pp. 79-98, 1998.
[51] R.C. Ward, Numerical Computation of the Matrix Exponential with Accuracy Estimate SIAM J. Numerical Analysis, vol. 14, pp. 600-614, 1977.
[52] R. Wildes, D. Horvonen, S. Hsu, R. Kumar, W. Lehman, B. Matei, and W. Zhao, Video Georegistration: Algorithm and Quantitative Evaluation Proc. Int'l Conf. Computer Vision, pp. 343-350, 2001.
[53] C. Van Loan, How Near Is a Stable Matrix to an Unstable Matrix? Contemporary Math., vol. 47, pp. 465-477, 1985.
[54] J.H. Wilkinson, "Error Analysis of Direct Methods of Matrix Inversion," J. ACM, Vol. 8, 1961, pp. 281-330.
[55] J.H. Wilkinson,The Algebraic Eigenvalue Problem, Oxford Univ. Press, Oxford, UK, 1965.
[56] Q. Zheng and R. Chellappa, “A Computational Vision Approach to Image Registration,” IEEE Trans. Image Processing, vol. 2, no. 3, pp. 311-326, 1993.
[57] http://vision.ece.ucsb.edudemos.html.Follow the link to the Image Registration demo, 2003.

Index Terms:
Registration, conditioning, feature representation, motion.
Charles S. Kenney, B.S. Manjunath, Marco Zuliani, Gary A. Hewer, Alan Van Nevel, "A Condition Number for Point Matching with Application to Registration and Postregistration Error Estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 11, pp. 1437-1454, Nov. 2003, doi:10.1109/TPAMI.2003.1240118
Usage of this product signifies your acceptance of the Terms of Use.