This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Registration of Challenging Image Pairs: Initialization, Estimation, and Decision
November 2007 (vol. 29 no. 11)
pp. 1973-1989
Our goal is an automated 2d-image-pair registration algorithm capable of aligning images taken of a wide variety of natural and man-made scenes as well as many medical images. The algorithm should handle low overlap, substantial orientation and scale differences, large illumination variations, and physical changes in the scene. An important component of this is the ability to automatically reject pairs that have no overlap or have too many differences to be aligned well.We propose a complete algorithm, including techniques for initialization, for estimating transformation parameters, and for automatically deciding if an estimate is correct. Keypoints extracted and matched between images are used to generate initial similarity transform estimates, each accurate over a small region. These initial estimates are rank-ordered and tested individually in succession. Each estimate is refined using the Dual-Bootstrap ICP algorithm, driven by matching of multiscale features. A three-part decision criteria, combining measurements of alignment accuracy, stability in the estimate, and consistency in the constraints, determines whether the refined transformation estimate is accepted as correct. Experimental results on a data set of 22 challenging image pairs show that the algorithm effectively aligns 19 of the 22 pairs and rejects 99.8% of the misalignments that occur when all possible pairs are tried. The algorithm substantially out-performs algorithms based on keypoint matching alone.

[1] S. Baker and I. Matthews, “Lucas-Kanade 20 Years On: A Unifying Framework,” Int'l J. Computer Vision, vol. 56, no. 3, pp. 221-255, 2004.
[2] S. Belongie, C. Carson, H. Greenspan, and J. Malik, “Color- and Texture-Based Image Segmentation Using EM and Its Application to Content-Based Image Retrieval,” Proc. IEEE Int'l Conf. Computer Vision, pp. 675-682, 1998.
[3] S. Belongie, J. Malik, and J. Puzicha, “Shape Matching and Object Recognition Using Shape Contexts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002.
[4] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hierarchical Model-Based Motion Estimation,” Proc. Second European Conf. Computer Vision, pp. 237-252, 1992.
[5] P. Besl and N. McKay, “A Method for Registration of 3D Shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[6] L.G. Brown, “A Survey of Image Registration Techniques,” ACM Computing Surveys, vol. 24, no. 4, pp. 325-376, Dec. 1992.
[7] M. Brown and D. Lowe, “Recognising Panoramas,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[8] M. Brown, R. Szeliski, and S. Winder, “Multi-Image Matching Using Multi-Scale Oriented Patches,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 510-517, 2005.
[9] K. Bubna and C.V. Stewart, “Model Selection Techniques and Merging Rules for Range Data Segmentation Algorithms,” Computer Vision and Image Understanding, vol. 80, pp. 215-245, 2000.
[10] K.P. Burnham and D.R. Anderson, Model Selection and Inference: A Practical Information-Theorectic Approach, first ed. Springer, 1998.
[11] A. Can, C. Stewart, B. Roysam, and H. Tanenbaum, “A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 347-364, Mar. 2002.
[12] G. Champleboux, S. Lavallee, R. Szeliski, and L. Brunie, “From Accurate Range Imaging Sensor Calibration to Accurate Model-Based 3-D Object Localization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 83-89, 1992.
[13] Y. Chen and G. Medioni, “Object Modeling by Registration of Multiple Range Images,” Image and Vision Computing, vol. 10, no. 3, pp. 145-155, 1992.
[14] H. Chui, A. Rangarajan, J. Zhang, and C.M. Leonard, “Unsupervised Learning of an Atlas from Unlabeled Point-Sets,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp.160-172, Feb. 2004.
[15] Y. Dufournaud, C. Schmid, and R. Horaud, “Image Matching with Scale Adjustment,” Computer Vision and Image Understanding, vol. 93, pp. 175-194, 2004.
[16] R. Fergus, P. Perona, and A. Zisserman, “Object Class Recognition by Unsupervised Scale-Invariant Learning,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[17] V. Ferrari, T. Tuytelaars, and L.V. Gool, “Simultaneous Object Recognition and Segmentation by Image Exploration,” Proc. Eighth European Conf. Computer Vision, 2004.
[18] R. Fransens, C. Strecha, and L.V. Gool, “Multimodal and Multiband Image Registration Using Mutual Information,” Proc. Theory and Applications of Knowledge-Driven Image Information Mining with Focus on Earth Observation (ESA-EUSC), 2004.
[19] W.T. Freeman and E.H. Adelson, “The Design and Use of Steerable Filters,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 9, pp. 891-906, Sept. 1991.
[20] L.V. Gool, T. Moons, and D. Ungureanu, “Affine/Photometric Invariants for Planar Intensity Patterns,” Proc. Fourth European Conf. Computer Vision, 1996.
[21] S. Granger and X. Pennec, “Multi-Scale EM-ICP: A Fast and Robust Approach for Surface Registration,” Proc. Seventh European Conf. Computer Vision, pp. 418-432, 2002.
[22] W. Grimson, T. Lozano-Perez, W. Wells, G. Ettinger, and S. White, “An Automatic Registration Method for Frameless Stereotaxy, Image, Guided Surgery and Enhanced Reality Visualization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 430-436, 1994.
[23] C. Harris and M. Stephens, “A Combined Corner and Edge Detector,” Proc. Fourth Alvey Vision Conf., pp. 147-151, 1988.
[24] R. Hartley and A. Zisserman, Multiple View Geometry. Cambridge Univ. Press, 2000.
[25] D.L.G. Hill, P.G. Batchelor, M. Holden, and D.J. Hawkes, “Medical Image Registration,” Physics in Medicine and Biology, vol. 46, no. 3, 2001.
[26] P.W. Holland and R.E. Welsch, “Robust Regression Using Iteratively Reweighted Least-Squares,” Comm. Statistics: Theory and Methods, vol. A6, pp. 813-827, 1977.
[27] M. Irani and P. Anandan, “Robust Multisensor Image Alignment,” Proc. IEEE Int'l Conf. Computer Vision, pp. 959-966, 1998.
[28] T. Kadir, A. Zisserman, and M. Brady, “An Affine Invariant Salient Region Detector,” Proc. Eighth European Conf. Computer Vision, 2004.
[29] K. Kanatani, Statistical Optimization for Geometric Computation: Theory and Practice. Elsevier, 1996.
[30] T. Lindeberg, Scale-Space Theory in Computer Vision. Kluwer Academic, 1994.
[31] D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004.
[32] B. Luo and E.R. Hancock, “Iterative Procrustes Alignment with the EM Algorithm,” Image and Vision Computing, vol. 20, nos. 5-6, pp. 377-396, Apr. 2002.
[33] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality Image Registration by Maximization of Mutual Information,” IEEE Trans. Medical Imaging, vol. 16, no. 2, pp. 87-198, 1997.
[34] J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust Wide-Baseline Stereo from Maximally Stable Extremal Regions,” Image and Vision Computing, vol. 22, no. 10, pp. 761-767, Sept. 2004.
[35] P. Meer, “Robust Techniques for Computer Vision,” Emerging Topics in Computer Vision, G. Medioni and S.B. Kang, eds., Prentice Hall, 2004.
[36] K. Mikolajczyk and C. Schmid, “Scale and Affine Invariant Interest Point Detectors,” Int'l J. Computer Vision, vol. 60, no. 1, pp. 63-86, 2004.
[37] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005.
[38] K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L.V. Gool, “A Comparison of Affine Region Detectors,” Int'l J. Computer Vision, vol. 65, nos. 1-2, pp. 43-72, 2005.
[39] J.V. Miller, “Regression-Base Surface Reconstruction: Coping with Noise, Outliers, and Discontinuities,” PhD thesis, Rensselaer Polytechnic Inst., Aug. 1997.
[40] J.P.W. Pluim, J.B.A. Maintz, and M.A. Vierveger, “Image Registration by Maximization of Combined Mutual Information and Gradient Information,” IEEE Trans. Medical Imaging, vol. 19, no. 8, pp. 809-814, Aug. 2000.
[41] J.P.W. Pluim, J.B.A. Maintz, and M.A. Vierveger, “Mutual-Information-Based Registration of Medical Images: A Survey,” IEEE Trans. Medical Imaging, vol. 22, no. 8, pp. 986-1004, 2003.
[42] H. Sawhney, S. Hsu, and R. Kumar, “Robust Video Mosaicing through Topology Inference and Local to Global Alignment,” Proc. Fifth European Conf. Computer Vision, vol. II, pp. 103-119, 1998.
[43] F. Schaffalitzky and A. Zisserman, “Multi-View Matching for Unordered Image Sets, or How Do I Organize My Holiday Snaps,” Proc. Seventh European Conf. Computer Vision, vol. 1, pp. 414-431, 2002.
[44] C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of Interest Point Detectors,” Int'l J. Computer Vision, vol. 37, no. 2, pp. 151-172, 2000.
[45] Y. Shan, H.S. Sawhney, and R. Kumar, “Unsupervised Learning of Discriminative Edge Measures for Vehicle Matching between Non-Overlapping Cameras,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[46] D. Shen and C. Davatzikos, “Hammer: Hierarchical Attribute Matching Mechanism for Elastic Registration,” IEEE Trans. Medical Imaging, vol. 21, no. 11, pp. 1421-1439, 2002.
[47] K.L. Steele and P.K. Egbert, “Correspondence Expansion for Wide Baseline Stereo,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[48] C. Stewart, C.-L. Tsai, and B. Roysam, “The Dual-Bootstrap Iterative Closest Point Algorithm with Application to Retinal Image Registration,” IEEE Trans. Medical Imaging, vol. 22, no. 11, pp. 1379-1394, 2003.
[49] C.V. Stewart, “Robust Parameter Estimation in Computer Vision,” SIAM Rev., vol. 41, no. 3, pp. 513-537, 1999.
[50] P. Torr, “An Assessment of Information Criteria for Motion Model Selection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 47-52, 1997.
[51] P. Torr and A. Zisserman, “MLESAC: A New Robust Estimator with Application to Estimating Image Geometry,” Computer Vision and Image Understanding, vol. 78, no. 1, pp. 138-156, Apr. 2000.
[52] T. Tuytelaars and L.V. Gool, “Matching Widely Separated Views Based on Affine Invariant Regions,” Int'l J. Computer Vision, vol. 1, no. 59, pp. 61-85, 2004.
[53] P. Viola and W.M. Wells III, “Alignment by Maximization of Mutual Information,” Int'l J. Computer Vision, vol. 24, no. 2, pp.137-154, 1997.
[54] B. Zitova and J. Flusser, “Image Registration Methods: A Survey,” Image and Vision Computing, vol. 21, pp. 977-1000, 2003.

Index Terms:
Image registration, feature extraction, iterative closest point, radial lens distortion,, decision criteria, keypoint
Citation:
Gehua Yang, Charles V. Stewart, Michal Sofka, Chia-Ling Tsai, "Registration of Challenging Image Pairs: Initialization, Estimation, and Decision," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1973-1989, Nov. 2007, doi:10.1109/TPAMI.2007.1116
Usage of this product signifies your acceptance of the Terms of Use.