This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Point Matching under Large Image Deformations and Illumination Changes
June 2004 (vol. 26 no. 6)
pp. 674-688

Abstract—To solve the general point correspondence problem in which the underlying transformation between image patches is represented by a homography, a solution based on extensive use of first order differential techniques is proposed. We integrate in a single robust M-estimation framework the traditional optical flow method and matching of local color distributions. These distributions are computed with spatially oriented kernels in the 5D joint spatial/color space. The estimation process is initiated at the third level of a Gaussian pyramid, uses only local information, and the illumination changes between the two images are also taken into account. Subpixel matching accuracy is achieved under large projective distortions significantly exceeding the performance of any of the two components alone. As an application, the correspondence algorithm is employed in oriented tracking of objects.

[1] S. Baker and I. Matthews, “Equivalence and Efficiency of Image Alignment Algorithms,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, Dec. 2001.
[2] J. Barron and R. Klette, Quantitative Color Optical Flow Proc. 16th Int'l Conf. Pattern Recognition, vol. 4, pp. 251-255, Aug. 2002.
[3] A.C. Berg and J. Malik, Geometric Blur for Template Matching Proc. 2001 IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 607-614, Dec. 2001.
[4] J. Bergen, P. Anandan, K. Hanna, and R. Hingorani, Hierarchical Model-Based Motion Estimation Proc. European Conf. Computer Vision, pp. 237-252, May 1992.
[5] M.J. Black and P. Anandan, The Robust Estimation of Multiple Motions: Parameteric and Piecewise-Smooth Flow Fields Proc. Conf. Computer Vision and Image Understanding, vol. 63, pp. 75-104, 1996.
[6] J.W. Brandt, Improved Accuracy in Gradient-Based Optical Flow Estimation Int'l J. Computer Vision, vol. 25, pp. 5-22, 1997.
[7] J. W. Brewer, “Kronecker Products and Matrix Calculus in System Theory,” IEEE Trans. Circuits and Systems, vol. 25, pp. 772-781, Sept. 1978.
[8] J. Bride and P. Meer, Registration via Direct Methods: A Statistical Approach Proc. 2001 IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 984-989, Dec. 2001.
[9] M. Brooks, W. Chojnacki, D. Gaeley, and A. can den Hengel, What Value Covariance Information in Estimating Vision Parameters? Proc. IEEE Eighth Int'l Conf. Computer Vision, pp. 302-308, 2001.
[10] W. Chojnacki, M. J. Brooks, A. van den Hengel, and D. Gawley, From FNS to HEIV: A Link between Two Vision Parameter Estimation Methods IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, pp. 264-268, 2004.
[11] D. Comaniciu, V. Ramesh, and P. Meer, Real-Time Tracking of Non-Rigid Objects Using Mean Shift Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 142-149, 2000.
[12] M.S. Drew, J. Wei, and Z.N. Li, Illumination-Invariant Image Retrieval and Video Segmentation Pattern Recognition, vol. 32, pp. 1369-1388, 1999.
[13] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. Wiley, 2000.
[14] C. Fermuller, D. Shulman, and Y. Aloimonos, The Statistics of Optical Flow Computer Vision and Image Understanding, vol. 82, pp. 1-32, Apr. 2001.
[15] G.D. Finlayson and S.D. Hordley, Color Constancy at a Pixel J. Optical Soc. of Am. A, vol. 18, pp. 253-264, 2001.
[16] M.A. Fischler and R.C. Bolles, Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography Comm. Assoc. Computing Machinery, vol. 24, pp. 381-395, 1981.
[17] K. Fukunaga, Introduction to Statistical Pattern Recognition. second ed., Academic Press, 1990.
[18] A. Fusiello, E. Trucco, T. Tommasini, and V. Roberto, Improving Feature Tracking with Robust Statistics Pattern Analysis and Applications, vol. 2, pp. 312-320, 1999.
[19] T. Gevers and A.W.M. Smeulders, Content-Based Image Retrieval by Viewpoint-Invariant Color Indexing Image and Vision Computing, vol. 17, pp. 475-488, 1999.
[20] V. Govindu and C. Shekhar, Alignment Using Distributions of Local Geometric Properties IEEE Trans. Pattern Analysis Machine Intelligence, vol. 21, pp. 1031-1043, 1999.
[21] A. Griffin and J. Kittler, An Active Mesh Based Tracker for Improved Feature Correspondences Pattern Recognition Letters, vol. 23, pp. 443-449, 2002.
[22] A.B. Hadiashar and D. Suter, Robust Optic Flow Computation Int'l J. Computer Vision, vol. 29, pp. 59-77, 1998.
[23] G. Hager and P. Belhumeur, “Real-Time Tracking of Image Regions with Changes in Geometry and Illumination,” Proc. Computer Vision and Pattern Recognition, 1996.
[24] C. Harris and M. Stephens, A Combined Corner and Edge Detector Proc. Alvey Vision Conf., pp. 147-151, 1988.
[25] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[26] P.J. Huber, Robust Statistical Procedures, second ed. Soc. Industrial and Applied Math., 1996.
[27] M. Irani and P. Anandan, Robust Multi-Sensor Image Alignment Proc. IEEE Int'l Conf. Computer Vision, pp. 959-966, 1998.
[28] K. Kanatani, Statistical Optimization for Geometric Computation: Theory and Practice. Elsevier, 1996.
[29] K. Kanatani and Y. Kanazawa, Automatic Thresholding for Correspondence Detection Proc. Statistical Methods in Video Processing Workshop, pp. 19-24, June 2002.
[30] Y. Kanazawa and K. Kanatani, Do We Really Have to Consider Covariance Matrices for Image Features? Proc. IEEE Int'l Conf. Computer Vision, pp. 301-306, 2001.
[31] Y.G. Leclerc, Q.T. Luong, and P. Fua, Self-Consistency and MDL: A Paradigm for Evaluating Point Correspondence Algorithms and Detecting Change Proc. Int'l J. Computer Vision, vol. 51, pp. 63-83, 2003.
[32] G. Li, Robust Regression Exploring Data Tables, Trends, and Shapes, pp. 281-343, D.C. Hoaglin, F. Mosteller, and J.W. Tukey, eds., John Wiley and Sons, 1985.
[33] D.G. Lowe, “Object Recognition from Local Scale-Invariant Features,” Proc. Seventh Int'l. Conf. Computer Vision, pp. 1150-1157, Sept. 1999.
[34] B. Lucas and T. Kanade, An Iterative Image Registration Technique with Application to Stereo Vision Proc. Int'l Joint Conf. Artificial Intelligence, pp. 674-679, Aug. 1981.
[35] J. Matas, S. Obdrzalek, and O. Chum, Local Affine Frames for Wide-Baseline Stereo Proc. 16th Int'l Conf. Pattern Recognition, vol. 4, pp. 363-366, Aug. 2002.
[36] B. Matei, Heteroscedastic Errors-In-Variables Models in Computer Vision PhD thesis, Dept. of Electrical and Computer Eng., Rutgers Univ., 2001. Available athttp://www.caip.rutgers.edu/riul/research theses.html.
[37] B. Matei and P. Meer, “A General Method for Errors-in-Variables Problems in Computer Vision,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 18-25, June 2000.
[38] P. Meer and I. Weiss, Smoothed Differentiation Filters for Images J. Visual Comm. and Image Representation, vol. 3, pp. 58-72, 1992.
[39] K. Mikolajczyk and C. Schmid, An Affine Invariant Interest Point Detector Proc. European Conf. Computer Vision, vol. 1, pp. 128-142, May 2002.
[40] F. Mindru, T. Moons, and L.V. Gool, Comparing Intensity Transformations and Their Invariants in the Context of Color Pattern Recognition Proc. European Conf. Computer Vision, vol. 4, pp. 448-460, May 2002.
[41] A. Mitiche and P. Bouthemy, Computation and Analysis of Image Motion: A Synopsis of Current Problems and Methods Proc. Int'l J. Computer Vision, vol. 19, pp. 29-55, 1996.
[42] H. Nagel, Optical Flow Estimation and the Interaction between Measurement Errors at Adjacent Pixel Positions Proc. Int'l J. Computer Vision, vol. 15, pp. 271-288, 1995.
[43] L. Ng and V. Solo, Errors-in-Variables Modeling in Optical Flow Estimation IEEE Trans. Image Processing, vol. 10, pp. 1528-1540, 2001.
[44] K. Nickels and S. Hutchinson, Estimating Uncertainty in SSD-Based Feature Tracking Image and Vision Computing, vol. 20, pp. 47-58, 2002.
[45] M. Nicolescu and G. Medioni, Perceptual Grouping from Motion Cues Using Tensor Voting in 4-D Proc. European Conf. Computer Vision, vol. 3, pp. 423-437, May 2002.
[46] J.M. Odobez and P. Bouthemy, Robust Multiresolution Estimation of Parametric Motion Models Applied to Complex Scenes J. Visual Comm. and Image Representation, vol. 6, pp. 348-365, 1995.
[47] P. Pritchett and A. Zisserman, Wide Baseline Stereo Matching IEEE Proc. Int'l Conf., pp. 754-760, Jan. 1998.
[48] F. Rothganger, S. Lazebnik, C. Schmidt, and J. Ponce, 3D Object Modeling and Recognition Using Affine-Invariant Patches and Multi-View Spatial Constraints Proc. 2003 IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 272-280, June 2003.
[49] P.J. Rousseeuw and A.M. Leroy, Robust Regression and Outlier Detection. Wiley, 1987.
[50] F. Schaffalitzky and A. Zisserman, Multi-View Matching for Unordered Image Sets, or How Do I Organize My Holiday Snaps? Proc. European Conf. Computer Vision, vol. 1, pp. 414-431, May 2002.
[51] C. Schmid, R. Mohr, and C. Bauckhage, Evaluation of Interest Point Detectors Computer Vision and Image Understanding, vol. 78, pp. 151-172, 2000.
[52] J. Shi and C. Tomasi, Good Features to Track Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[53] B.W. Silverman, Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986.
[54] R. Szeliski and J Coughlan, Spline-Based Image Registration Proc. Int'l J. Computer Vision, vol. 22, pp. 199-218, 1997.
[55] D. Tell and S. Carlsson, Wide Baseline Point Matching Using Affine Invariants Computed from Intensity Profiles Proc. European Conf. Computer Vision, pp. 814-828, June 2000.
[56] D. Tell and S. Carlsson, Combining Appearance and Topology for Wide Baseline Matching Proc. European Conf. Computer Vision, vol. 1, pp. 68-81, May 2002.
[57] T. Tommasini, A. Fusiello, E. Trucco, and V. Roberto, Making Good Features Track Better IEEE Proc. Conf. Computer Vision and Pattern Recognition, pp. 178-183, 1998.
[58] P.H.S. Torr and C. Davidson, IMPSAC: Synthesis of Importance Sampling and Random Sample Consensus IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, pp. 354-364, 2003.
[59] P.H.S. Torr and A. Zisserman, MLESAC: A New Robust Estimator with Application to Estimating Image Geometry Computer Vision and Image Understanding, vol. 78, pp. 138-156, 2000.
[60] C.J. Tsai, N.P. Galatsanos, and A.K. Katsaggelos, Optical Flow Estimation from Noisy Data Using Differential Techniques Proc. 1999 IEEE Int'l Conf. Acoustics, Speech and Signal Processing, vol. 6, pp. 3393-3396, Mar. 1999.
[61] T. Tuytelaars and L. Van Gool, Wide Baseline Stereo Matching Based on Local, Affinely Invariant Regions Proc. 11th British Machine Vision Conf., pp. 412-425, Sept. 2000.
[62] T. Tuytelaars, A. Turina, and L. Van Gool, Noncombinatorial Detection of Regular Repetitions under Perspective Skew IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, pp. 418-432, 2003.
[63] M.P. Wand and M.C. Jones, Kernel Smoothing. Chapman&Hall, 1995.
[64] B. Wang, K.K. Sung, and T.K. Ng, The Localized Consistency Principle for Image Matching under Non-Uniform Illumination Variation and Affine Distortion Proc. European Conf. Computer Vision, vol. 1, pp. 205-219, May 2002.
[65] J. Weber and J. Malik, Robust Computation of Optical-Flow in a Multiscale Differential Framework Int'l J. Computer Vision, vol. 14, pp. 67-81, 1995.

Index Terms:
Correspondence problem, optical flow, color distribution matching, motion tracking, wide-baseline stereo.
Citation:
Bogdan Georgescu, Peter Meer, "Point Matching under Large Image Deformations and Illumination Changes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 674-688, June 2004, doi:10.1109/TPAMI.2004.2
Usage of this product signifies your acceptance of the Terms of Use.