This Article 
 Bibliographic References 
 Add to: 
A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina
March 2002 (vol. 24 no. 3)
pp. 347-364

This paper describes a robust hierarchical algorithm for fully-automatic registration of a pair of images of the curved human retina photographed by a fundus microscope. Accurate registration is essential for mosaic synthesis, change detection, and design of computer-aided instrumentation. Central to the new algorithm is a 12-parameter interimage transformation derived by modeling the retina as a rigid quadratic surface with unknown parameters, imaged by an uncalibrated weak perspective camera. The parameters of this model are estimated by matching vascular landmarks extracted by an algorithm that recursively traces the blood vessel structure. The parameter estimation technique, which could be generalized to other applications, is a hierarchy of models and methods: an initial match set is pruned based on a zeroth order transformation estimated as the peak of a similarity-weighted histogram; a first order, affine transformation is estimated using the reduced match set and least-median of squares; and the final, second order, 12-parameter transformation is estimated using an M-estimator initialized from the first order estimate. This hierarchy makes the algorithm robust to unmatchable image features and mismatches between features caused by large interframe motions. Before final convergence of the M-estimator, feature positions are refined and the correspondence set is enhanced using normalized sum-of-squared differences matching of regions deformed by the emerging transformation. Experiments involving 3,000 image pairs (1,024\times1,024 pixels) from 16 different healthy eyes were performed. Starting with as low as 20 percent overlap between images, the algorithm improves its success rate exponentially and has a negligible failure rate above 67 percent overlap. The experiments also quantify the reduction in errors as the model complexities increase. Final registration errors less than a pixel are routinely achieved. The speed, accuracy, and ability to handle small overlaps compare favorably with retinal image registration techniques published in the literature.

[1] E. Balduf, “Real-Time Algorithms for a Laser Retinal Surgery System—Implementation on the Silicon Graphics Octane System,” Master's thesis, Rensselaer Polytechnic Inst., 1998.
[2] A.E. Beaton and J.W. Tukey, “The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data,” Technometrics, vol. 16, pp. 147-185, 1974.
[3] D.E. Becker, H. Ancin, D.H. Szarowski, J.N. Turner, and B. Roysam, “Automated 3-D Montage Synthesis from Laser-Scanning Confocal Images: Application to Quantitative Tissue-Level Cytological Analysis,” Cytometry, vol. 25, no. 3, pp. 235-245, Nov. 1996.
[4] D.E. Becker, A. Can, H.L. Tanenbaum, J.N. Turner, and B. Roysam, “Image Processing Algorithms for Retinal Montage Synthesis, Mapping, and Real-Time Location Determination,” IEEE Trans. Biomedical Eng., vol. 45, no. 1, pp. 105-118, 1998.
[5] D.E. Becker, A. Can, H.L. Tanenbaum, J.N. Turner, and B. Roysam, “Image Processing Algorithms for Retinal Montage Synthesis, Mapping, and Real-Time Location Determination,” IMIA Yearbook of Medical Informatics, Int'l Medical Informatics Association, Bemmel et al. eds., Germany: Schattauer Press, 1999.
[6] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hiercharchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
[7] J.W. Berger, “Quantitative Image Sequence Analysis of Fundus Fluorescein Angiography,” Ophthalmic Surgery Lasers, vol. 30, no. 1, pp. 72-73, Jan. 1999.
[8] J.W. Berger, M.E. Leventon, N. Hata, W. Wells, and R. Kikinis, “Design Considerations for a Computer-Vision-Enabled Ophthalmic Augmented Reality Environment,” Lecture Notes in Computer Science, vol. 1205, pp. 399-408, 1997.
[9] J.W. Berger and D.S. Shin, “Computer Vision Enabled Augmented Reality Fundus Biomicroscopy,” Ophthalmology, vol. 106, no. 10, Oct. 1999.
[10] P.J. Besl and N.D. McKay, "A Method for Registration of 3D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[11] G. Borgefors, “Distance Transforms in Digital Images,” Computer Vision, Graphics, and Image Processing, vol. 34, pp. 344-371, 1986.
[12] L. Brown, “A Survey of Image Registration Techniques,” ACM Computing Surveys, vol. 24, no. 4, pp. 325-376, 1992.
[13] A. Can, “Robust Computer Vision Algorithms for Registering Images from the Curved Human Retina,” PhD thesis, Rensselaer Polytechnic Inst., 2000.
[14] A. Can, H. Shen, J. N. Turner, H. L. Tanenbaum, and B. Roysam, “Rapid Automated Tracing and Feature Extraction from Live High-Resolution Retinal Fundus Images Using Direct Exploratory Algorithms,” IEEE Trans. Information Technology for Biomedicine, vol. 3, no. 2, pp. 125-138, 1999.
[15] A. Can, C. Stewart, and B. Roysam, “Robust Hierarchical Algorithm for Constructing a Mosaic from Images of the Curved Human Retina,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 286-292, 1999.
[16] A. Can, C. Stewart, B. Roysam, and H. Tanenbaum, “A Feature-Based Algorithm for Joint, Linear Estimation of High-Order Image-to-Mosaic Transformations: Mosaicing the Curved Human Retina,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 412-419, Mar. 2002.
[17] T. Cham and R. Cipolla, “A Statistical Framework for Long-Range Feature Matching in Uncalibrated Image Mosaicing,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 442-447, 1998.
[18] Y. Chen and G. Medioni, “Object Modeling from Multiple Range Images,” Image and Vision Computing, vol. 10, no. 3, pp. 145-155, Apr. 1992.
[19] H. Chui and A. Rangarajan, “A New Algorithm for Non-Rigid Point Matching,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 44-51, June 2000.
[20] A.V. Cideciyan, “Registration of Ocular Fundus Images,” IEEE Eng. in Medicine and Biology, vol. 14, no. 1, pp. 52-58, Jan. 1995.
[21] J. Duncan and N. Ayache, Medical Image Analysis: Progress over Two Decades and the Challenges Ahead IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 85-106, Jan. 2000.
[22] Retina and Vitreous, J.L. Federman, ed., St. Louis, Mo.: The C.V. Mosby Company, 1988.
[23] M.A. Fischler and R.C. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Graphics and Image Processing, vol. 24, no. 6, pp. 381–395, June 1981.
[24] M. Gleicher, “Projective Registration with Difference Decomposition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[25] M. Goldbaum, N. Katz, S. Chaudhuri, M. Nelson, and P. Kube, “Digital Image Processing for Ocular Fundus Images,” Ophthalmology Clinics of North Am., vol. 3, no. 3, pp. 447-466, Sept. 1990.
[26] M.H. Goldbaum, V. Kouznetsova, B.L. Cot, W.E. Hart, and M. Nelson, “Automated Registration of Digital Ocular Fundus Images for Comparison of Lesions,” SPIE: Ophthalmic Technologies III, vol. 1877, pp. 94-99, 1993.
[27] F.R. Hampel, P.J. Rousseeuw, E. Ronchetti, and W.A. Stahel, Robust Statistics: The Approach Based on Influence Functions. John Wiley&Sons, 1986.
[28] P.W. Holland and R.E. Welsch, “Robust Regression Using Iteratively Reweighted Least-Squares,” Comm. Statistical-Theory Method, vol. A6, pp. 813-827, 1977.
[29] M. Irani, B. Rousso, and S. Peleg, “Computing Occluding and Transparent Motions,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 5-16, Jan. 1994.
[30] P.K. Jensen and E. Scherfig, “Resolution of Retinal Digital Colour Images,” Acta Ophthalmology Scandinavia, vol. 77, no. 5, pp. 526-529, Oct. 1999.
[31] M. Kashiwase, T. Sata, Y. Yamauchi, H. Minoda, N. Usui, T. Iwasaki, T. Kurata, and M. Usui., “Progressive Outer Retinal Necrosis Caused by Herpes Simplex Virus Type 1 in a Patient with Acquired Immunodeficiency Syndrome,” Ophthalmology, vol. 107, no. 4, pp. 790-794, Apr. 2000.
[32] H. Lester and S. Arridge, “A Survey of Hierarchical Non-Linear Medical Image Registration,” Pattern Recognition, vol. 32, no. 1, pp. 129-149, 1999.
[33] A.A. Mahurkar, M.A. Vivino, B.L. Trus, E.M. Kuehl, M.B. Datiles, and M.I. Kaiser-Kupfer, “Constructing Retinal Fundus Photomontages,” Investigative Ophthalmology and Visual Science, vol. 37, no. 8, pp. 1675-1683, July 1996.
[34] M.S. Markow, H.G. Rylander, and A.J. Welch, “Real-Time Algorithm for Retinal Tracking,” IEEE Trans. Biomedical Eng., vol. 40, no. 12, pp. 1269-1281, Dec. 1993.
[35] G.K. Matsopoulos, N.A. Mouravliansky, K.K. Delibasis, and K.S. Nikita, “Automatic Retinal Image Registration Scheme Using Global Optimization Techniques,” IEEE Trans. Information Technology in Biomedicine, vol. 3, no. 1, pp. 47-60, 1999.
[36] J.V. Miller and C.V. Stewart, "MUSE: Robust Surface Fitting Using Unbiased Scale Estimates," Proc. Computer Vision and Pattern Recognition '96, pp. 300-306,San Francisco, June 1996.
[37] F. Neumann, W. Schreiner, and M. Neumann, “Computer Simulation of Coronary Arterial Trees,” Advances in Eng. Software, vol. 28, pp. 353-357, 1997.
[38] T. Onuki and S. Nitta, “Computer Simulation of Geometry and Hemodynamics of Canine Pulmonary Arteries,” Annals of Biomedical Eng., vol. 21, pp. 107-115, 1993.
[39] Eye Movements and Visual Cognition: Scene Perception and Reading, K.E. Rayner, ed., New York: Springer-Verlag, 1992.
[40] N. Ritter, R. Owens, J. Cooper, R. Eikelboom, and P. van Saarloos, “Registration of Stereo and Temporal Images of the Retina,” IEEE Trans. Medical Imaging, vol. 18, no. 5, pp. 404-418, 1999.
[41] P.J. Rousseeuw, “Least Median of Squares Regression,” J. Am. Statistical Assoc., vol. 79, pp. 871-880, 1984.
[42] P. Rousseeuw and A. Leory, Robust Regression and Outlier Detection. Wiley Series in Probability and Statistics, 1987.
[43] H. Sawhney, S. Hsu, and R. Kumar, “Robust Video Mosaicing through Topology Inference and Local to Global Alignment,” Proc. Fifth European Conf. Computer Vision, vol. II, pp. 103-119, 1998.
[44] H.S. Sawhney and R. Kumar, “True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 3, pp. 235-243, Mar. 1999.
[45] C. Schmid, R. Mohr, and C. Bauckhage, “Comparing and Evaluating Interest Points,” IEEE Proc. Int'l Conf. Computer Vision, pp. 230-235, 1998.
[46] A. Shashua and S. Toelg, “The Quadric Reference Surface: Theory and Applications,” Int'l J. Computer Vision, vol. 23, no. 2, pp. 185-198, 1997.
[47] H. Shen, B. Roysam, C. Stewart, J. Turner, and H. Tanenbaum, “Optimal Scheduling of Tracing Computations for Real-Time Vascular Landmark Extraction from retinal Fundus Images,” IEEE Trans. Information Technology in Biomedicine, vol. 5, no. 1, pp. 77-91, Mar. 2001.
[48] D.S. Shin, N.B. Javornik, and J.W. Berger, “Computer-Assisted Interactive Fundus Image Processing for Macular Drusen Quantitation,” Ophthalmology, vol. 106, no. 6, pp. 1119-1125, June 1999.
[49] H.-Y. Shum and R. Szeliski, “Construction and Refinement of Panoramic Mosaics with Global and Local Alignment,” Proc. IEEE Int'l Conf. Computer Vision, pp. 953-958, 1998.
[50] C.V. Stewart, “Robust Parameter Estimation in Computer Vision,” SIAM Review, vol. 41, no. 3, pp. 513-537, 1999.
[51] G. Szeliski and S. Lavalée, “Matching 3D Anatomical Surface with Non-Rigid Deformations Using Octree-Splines,” Int'l J. Computer Vision, vol. 18, no. 2, pp. 171-186, 1996.
[52] D. Terzopoulos and D. Metaxas, “Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 703-714, July 1991.
[53] P. Torr and D. Murray, “The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix,” Int'l J. Computer Vision, vol. 3, no. 24, pp. 271-300, 1997.
[54] Y. Wexler and A. Shashua, “Q-Warping: Direct Computation of Quadratic Reference Surfaces,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 333-338, 1999.
[55] C.H. Wright, R.D. Ferguson, H.G. Rylander, III, A.J. Welch, and S.F. Barrett, “Hybrid Approach to Retinal Tracking and Laser Aiming for Photocoagulation,” J. Biomedical Optics, vol. 2, no. 2, pp. 195-203, 1997.
[56] F. Zana and J.C. Klein, “A Multimodal Registration Algorithm of Eye Fundus Images Using Vessels Detection and Hough Trransform,” IEEE Trans. Medical Imaging, vol. 18, no. 5, pp. 419-428, 1999.
[57] Z. Zhang, R. Deriche, O. Faugeras, and Q.T. Luong, “A Rubust Technique for Matching Two Uncalibrated Images through the Recovery of the Unknown Epipolar Geometry,” Artificial Intelligence J., vol. 78, pp. 87-119, 1995.

Index Terms:
robust estimation, registration, transformation estimation, image mosaic, retinal imaging, feature extraction, feature refinement, multiscale methods, ophthalmic image processing, biomedical image processing
A. Can, C.V. Stewart, B. Roysam, H.L. Tanenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 347-364, March 2002, doi:10.1109/34.990136
Usage of this product signifies your acceptance of the Terms of Use.