This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction
March 1999 (vol. 21 no. 3)
pp. 235-243
ASCII Text
x 
 
Harpreet S. Sawhney, Rakesh Kumar, "True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 3, pp. 235-243, March, 1999.
 
 
BibTex
x
 
@article{ 10.1109/34.754589,
author = {Harpreet S. Sawhney and Rakesh Kumar},
title = {True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {21},
number = {3},
issn = {0162-8828},
year = {1999},
pages = {235-243},
doi = {http://doi.ieeecomputersociety.org/10.1109/34.754589},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction
IS - 3
SN - 0162-8828
SP235
EP243
EPD - 235-243
A1 - Harpreet S. Sawhney,
A1 - Rakesh Kumar,
PY - 1999
KW - Image sequence analysis
KW - video mosaics
KW - lens distortion correction.
VL - 21
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 

Abstract—Multiple images of a scene are related through 2D/3D view transformations and linear and non-linear camera transformations. In the traditional techniques to compute these transformations, especially the ones relying on direct intensity gradients, one image and its coordinate system have been assumed to be ideal and distortion free. In this paper, we present an algorithm for true multi-image alignment that does not rely on the measurements of a reference image being distortion free. The algorithm is developed to specifically align and mosaic images using parametric transformations in the presence of lens distortion. When lens distortion is present, none of the images can be assumed to be ideal. In our formulation, all the images are modeled as intensity measurements represented in their respective coordinate systems, each of which is related to an ideal coordinate system through an interior camera transformation and an exterior view transformation. The goal of the accompanying algorithm is to compute an image in the ideal coordinate system while solving for the transformations that relate the ideal system with each of the data images. Key advantages of the technique presented in this paper are: (i) no reliance on one distortion free image, (ii) ability to register images and compute coordinate transformations even when the multiple images are of an extended scene with no overlap between the first and last frame of the sequence, and (iii) ability to handle linear and non-linear transformations within the same framework. Results of applying the algorithm are presented for the correction of lens distortion, and creation of video mosaics.

[1] S. Ayer and H. Sawhney, "Layered Representation of Motion Video Using Robust Maximum-Likelihood Estimation of Mixture Models and mdl Encoding," Int'l Conf. Computer Vision, pp. 777-784,Cambridge, Mass., June 1995.
[2] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hiercharchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
[3] M. Black and P. Anandan, The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields J. Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
[4] P. Fua and Y.G. Leclerc, “Object-Centered Surface Reconstruction: Combining Multi-Image Stereo and Shading,” Int'l J. Computer Vision, vol. 16, pp. 35-56, Sept. 1995.
[5] K.J. Hanna and N.E. Okamoto, “Combining Stereo and Motion for Direct Estimation of Scene Structure,” Proc. Int'l Conf. Computer Vision, pp. 357-365, 1993.
[6] R.I. Hartley, “In Defense of the 8-Point Algorithm,” Proc. Fifth Int'l Conf. Computer Vision, pp. 1,064-1,070, June 1995.
[7] B.K.P. Horn and B.G. Schunck, "Determining Optical Flow," Artificial Intelligence, vol. 17, no. 1-3, pp. 185-203, 1981.
[8] S. Hsu, P. Anandan, and S. Peleg, "Accurate Computation of Optical Flow by Using Layered Motion Representation," Proc. ICPR, pp. 743-746,Jerusalem, Oct. 1994.
[9] S. Hsu and H.S. Sawhney, "Influence of Global Constraints and Lens Distortion on Pose and Appearance Recovery From a Purely Rotating Camera," WACV'98, pp. 154-159,Princeton, N.J., Oct. 1998.
[10] M. Irani, P. Anandan, and S. Hsu, “Mosaic Based Representations of Video Sequences and Their Applications,” Proc. Fifth Int'l Conf. Computer Vision, pp. 605-611, June 1995.
[11] M. Irani, B. Rousso, and S. Peleg, “Detecting and Tracking Multiple Moving Objects Using Temporal Integration,” Proc. European Conf. Computer Vision, pp. 282-287, May 1992.
[12] R. Kumar, P. Anandan, and K. Hanna, “Direct Recovery of Shape from Multiple Views: A Parallax Based Approach,” Proc. Int'l Conf. Pattern Recognition, pp. 685-688, Oct. 1994.
[13] R. Kumar, P. Anandan, M. Irani, J. Bergen, and K. Hanna, “Representation of Scenes from Collections of Images,“ Proc. IEEE Workshop Representation of Visual Scenes, pp. 10-17, June 1995.
[14] S. Mann and R. Picard, “Virtual Bellows: Constructing High Quality Stills from Video,” Proc. First IEEE Int'l Conf. Image Processing, vol. I, pp. 363-367, Nov. 1994.
[15] S. Peleg and J. Herman, “Panoramic Mosaics by Manifold Projection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 338-343, June 1997.
[16] H.S. Sawhney, "Simplifying Motion and Structure Analysis Using Planar Parallax and Image Warping," Proc. Int'l Conf. Pattern Recognition, 1994.
[17] H.S. Sawhney, S. Ayer, and M. Gorkani, “Model-Based 2D and 3D Dominant Motion Estimation for Mosaicing and Video Representation,” Proc. Fifth Int'l Conf. Computer Vision, pp. 583-590, June 1995.
[18] H. Sawhney, S. Hsu, and R. Kumar, “Robust Video Mosaicing through Topology Inference and Local to Global Alignment,” Proc. Fifth European Conf. Computer Vision, vol. II, pp. 103-119, 1998.
[19] S.M. Seitz and C.R. Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring,” Computer Vision and Pattern Recognition, pp. 1067-1073, 1997.
[20] H.-Y. Shum and R. Szeliski, “Construction and Refinement of Panoramic Mosaics with Global and Local Alignment,” Proc. IEEE Int'l Conf. Computer Vision, pp. 953-958, 1998.
[21] G.P. Stein, “Lens Distortion Calibration Using Point Correspondences,” Proc. 1997 Conf. Computer Vision and Pattern Recognition, pp. 143-148, June 1997.
[22] R. Szeliski, “Image Mosaicing for Tele-Reality Applications,” IEEE Computer Graphics and Applications, 1996.
[23] R.Y. Tsai, "An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision," Proc. Computer Vision and Pattern Recognition Conf., pp. 364-374, 1986.
[24] Z. Zhang, "On the Epipolar Geometry Between Two Images With Lens Distortion," Int'l Conf. Pattern Recognition, vol. 1, pp. 407-411,Vienna, Aug. 1996.

Index Terms:
Image sequence analysis, video mosaics, lens distortion correction.
Citation:
Harpreet S. Sawhney, Rakesh Kumar, "True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 3, pp. 235-243, March 1999, doi:10.1109/34.754589
Usage of this product signifies your acceptance of the Terms of Use.