Subscribe
Issue No.04 - April (2009 vol.31)
pp: 649-660
Hui Ji , National University of Singapore, Singapore
Cornelia Fermüller , University of Maryland at College Park, College Park
ABSTRACT
We present an analysis and algorithm for the problem of super-resolution imaging, that is the reconstruction of HR (high-resolution) images from a sequence of LR (low-resolution) images. Super-resolution reconstruction entails solutions to two problems. One is the alignment of image frames. The other is the reconstruction of a HR image from multiple aligned LR images. Both are important for the performance of super-resolution imaging. Image alignment is addressed with a new batch algorithm, which simultaneously estimates the homographies between multiple image frames by enforcing the surface normal vectors to be the same. This approach can handle longer video sequences quite well. Reconstruction is addressed with a wavelet-based iterative reconstruction algorithm with an efficient de-noising scheme. The technique is based on a new analysis of video formation. At a high level our method could be described as a better-conditioned iterative back projection scheme with an efficient regularization criteria in each iteration step. Experiments with both simulated and real data demonstrate that our approach has better performance than existing super-resolution methods. It can remove even large amounts of mixed noise without creating artifacts.
INDEX TERMS
Image processing software, Enhancement
CITATION
Hui Ji, Cornelia Fermüller, "Robust Wavelet-Based Super-Resolution Reconstruction: Theory and Algorithm", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 4, pp. 649-660, April 2009, doi:10.1109/TPAMI.2008.103
REFERENCES
 [1] S. Baker and T. Kanade, “Limits on Super-Resolution and How to Break Them,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 372-379, 2000. [2] B. Bascle, A. Blake, and A. Zisserman, “Motion Deblurring and Super-Resolution from an Image Sequence,” Proc. Fourth European Conf. Computer Vision, vol. 2, pp. 573-582, 1996. [3] C.D. Boor, A Practical Guide to Splines, first hardcover printing ed. Spring, 2001. [4] N. Bose and K. Boo, “High-Resolution Image Recognition with Multisensors,” J. Imaging Systems and Technology, vol. 9, pp. 294-304, 1998. [5] A. Chambolle, R. Devore, N. Lee, and B. Lucier, “Nonlinear Wavelet Image Processing: Variational Problems, Compression and Noise Removal through Wavelets,” IEEE Trans. Image Processing, vol. 7, 1998. [6] R. Chan, T. Chan, L. Shen, and Z. Shen, “Wavelet Deblurring Algorithms for Spatially Varying Blur from High-Resolution Image Reconstruction,” Linear Algebra and Its Applications, vol. 366, pp. 139-155, 2003. [7] R. Chan, S. Riemenschneider, L. Shen, and Z. Shen, “High-Resolution Image Reconstruction with Displacement Errors: A Framelet Approach,” Int'l J. Imaging System and Technology, vol. 14, pp. 91-104, 2004. [8] R. Chan, S. Riemenschneider, L. Shen, and Z. Shen, “Tight Frame: An Efficient Way for High-Resolution Image Reconstruction,” Applied and Computational Harmonic Analysis, vol. 17, pp. 91-115, 2004. [9] M. Eland and A. Feuer, “Restoration of a Signal Super-Resolution Image from Several Blurred, Noisy and Undersampled Measured Images,” IEEE Trans. Image Processing, pp. 1646-1658, 1997. [10] S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Robust Shift and Add Approach to Super-Resolution,” Proc. Int'l Soc. Optical Eng. (SPIE), 2003. [11] C. Fermüller, P. Baker, and Y. Aloimonos, “Visual Space Time Geometry: A Tool for Perception and Imagination,” Proc. IEEE, vol. 90, no. 5, pp. 1113-1135, 2002. [12] V.M. Govindu, “Combining Two-View Constraints for Motion Estimation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001. [13] B.K.P. Horn, Robot Vision. MacGraw-Hill, 1986. [14] M. Irani and F. Peleg, “Motion Analysis for Image Enhancement: Resolution, Occlusion and Transparency,” J. Visual Comm. and Image Representation, vol. 4, pp. 324-335, 1993. [15] H. Ji and C. Fermüller, “A 3D Shape Constraint on Video,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp.1018-1023, June 2006. [16] A. Makadia and K. Daniilidis, “Rotation Estimation from Spherical Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, pp. 1170-1175, 2006. [17] A. Makadia, C. Geyer, and K. Daniilidis, “Correspondenceless Structure from Motion,” Int'l J. Computer Vision, 2007. [18] S. Mallat, A Wavelet Tour of Signal Processing. Academic Press, 1999. [19] L.J. Mondell, Diophantine Equations. Academic Press, 1969. [20] P. Mrazek, J. Weickert, and G. Steidl, “Correspondences between Wavelet Shrinkage and Nonlinear Diffusion,” Proc. Scale-Space Methods in Computer Vision, pp. 101-116, 2003. [21] N. Nguyen and N.P. Milanfar, “A Wavelet-Based Interpolation-Restoration Method for Superresolution,” Circuits, Systems, and Signal Processing, vol. 19, pp. 321-338, 2000. [22] A. Shashua and S. Avidan, “The Rank 4 Constraint in Multiple ($\geq$ 3) View Geometry,” Proc. Fourth European Conf. Computer Vision, 1996. [23] A. Tekalp, M. Ozkan, and M. Sezan, “High-Resolution Image Reconstruction from Low-Resolution Image Sequences and Space-Varying Image Restoration,” Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 169-172, 1992. [24] J. Weickert, Anisotropic Diffusion in Image Processing. Teubner, 1998. [25] C. Youla, “Generalized Image Restoration by the Method of Alternating Orthogonal Projections,” IEEE Trans. Circuits and System, 1978. [26] L. Zelnik-Manor and M. Irani, “Multi-Frame Estimation of Planar Motion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, pp. 1105-1114, 2000. [27] W. Zhao and H.S. Sawhney, “Is Super-Resolution with Optical Flow Feasible,” Proc. Seventh European Conf. Computer Vision, pp.599-613, 2002. [28] A. Zomet, A. Rav-Acha, and S. Peleg, “Robust Super-Resolution,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.