This Article 
 Bibliographic References 
 Add to: 
Video Super-Resolution Using Controlled Subpixel Detector Shifts
June 2005 (vol. 27 no. 6)
pp. 977-987
Video cameras must produce images at a reasonable frame-rate and with a reasonable depth of field. These requirements impose fundamental physical limits on the spatial resolution of the image detector. As a result, current cameras produce videos with a very low resolution. The resolution of videos can be computationally enhanced by moving the camera and applying super-resolution reconstruction algorithms. However, a moving camera introduces motion blur, which limits super-resolution quality. We analyze this effect and derive a theoretical result showing that motion blur has a substantial degrading effect on the performance of super-resolution. The conclusion is that, in order to achieve the highest resolution, motion blur should be avoided. Motion blur can be minimized by sampling the space-time volume of the video in a specific manner. We have developed a novel camera, called the "jitter camera," that achieves this sampling. By applying an adaptive super-resolution algorithm to the video produced by the jitter camera, we show that resolution can be notably enhanced for stationary or slowly moving objects, while it is improved slightly or left unchanged for objects with fast and complex motions. The end result is a video that has a significantly higher resolution than the captured one.

[1] S. Baker and T. Kanade, “Limits on Super-Resolution and How to Break Them,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1167-1183, Sept. 2002.
[2] B. Bascle, A. Blake, and A. Zisserman, “Motion Deblurring and Super-Resolution from an Image Sequence,” Proc. European Conf. Computer Vision, vol. 2, pp. 573-582, 1996.
[3] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hierarchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
[4] D. Capel and A. Zisserman, “Super-Resolution Enhancement of Text Image Sequences,” Proc. Int'l Conf. Pattern Recognition, vol. I, pp. 600-605, Sept. 2000.
[5] M.C. Chiang and T.E. Boult, “Efficient Super-Resolution via Image Warping,” Image and Vision Computing, vol. 18, no. 10 pp. 761-771, July 2000.
[6] Pixera Corporation, “Diractor,” http://www.outex.oulu.fihttp:/ .
[7] M. Elad and A. Feuer, “Restoration of a Single Superresolution Image from Several Blurred, Noisy, and Undersampled Measured Images,” IEEE Trans. Image Processing, vol. 6, no. 12, pp. 1646-1658, Dec. 1997.
[8] Physik Instrumente, “M-111 Micro Translation Stage,” http:/, 2005.
[9] M. Irani and S. Peleg, “Improving Resolution by Image Registration,” Graphical Models and Image Processing, vol. 53, pp. 231-239, 1991.
[10] Z. Lin and H.Y. Shum, “Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 1 pp. 83-97, Jan. 2004.
[11] P. Meer, D. Mintz, D.Y. Kim, and A. Rosenfeld, “Robust Regression Methods for Computer Vision: A Review,” Int'l J. Computer Vision, vol. 6, no. 1, pp. 59-70, 1991.
[12] Mi nolta “Dimage-a1,” /, 2005.
[13] A.J. Patti, M.I. Sezan, and A.M. Tekalp, “Superresolution Video Reconstruction with Arbitrary Sampling Lattices and Nonzero Aperture Time,” IEEE Trans. Image Processing, vol. 6, no. 8, pp. 1064-1076, Aug. 1997.
[14] P. Pudlák, “A Note on the Use of Determinant for Proving Lower Bounds on the Size of Linear Circuits,” Information Processing Letters, vol. 74, nos. 5-6, pp. 197-201, 2000.
[15] R. Ramanath, W. Snyder, G. Bilbro, and W. Sander, “Demosaicking Methods for Bayer Color Arrays,” J. Electronic Imaging, vol. 11, no. 3, July 2002.
[16] A. Rav-Acha and S. Peleg, “Restoration of Multiple Images with Motion Blur in Different Directions,” Proc. IEEE Workshop Applications of Computer Vision, pp. 22-28, 2000.
[17] Point Grey Research, “Dragonfly Camera,” http:/www.ptgrey. com, 2005.
[18] R.R. Schultz and R.L. Stevenson, “Extraction of High-Resolution Frames from Video Sequences,” IEEE Trans. Image Processing, vol. 5, no. 6, pp. 996-1011, June 1996.
[19] E. Shechtman, Y. Caspi, and M. Irani, “Increasing Space-Time Resolution in Video,” Proc. European Conf. Computer Vision, vol. I, p. 753, 2002.
[20] H. Shekarforoush and R. Chellappa, “Data-Driven Multichannel Superresolution with Application to Video Sequences,” J. Optical Soc. Am., vol. 16, no. 3, pp. 481-492, Mar. 1999.
[21] E.P. Simoncelli, “Modeling the Joint Statistics of Images in the Wavelet Domain,” SPIE, vol. 3813, pp. 188-195, July 1999.

Index Terms:
Sensors, jitter camera, jitter video, super-resolution, motion blur.
Moshe Ben-Ezra, Assaf Zomet, Shree K. Nayar, "Video Super-Resolution Using Controlled Subpixel Detector Shifts," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 977-987, June 2005, doi:10.1109/TPAMI.2005.129
Usage of this product signifies your acceptance of the Terms of Use.