The Community for Technology Leaders
RSS Icon
Issue No.05 - May (2012 vol.34)
pp: 972-986
Tom E. Bishop , Heriot-Watt University, Edinburgh
Portable light field (LF) cameras have demonstrated capabilities beyond conventional cameras. In a single snapshot, they enable digital image refocusing and 3D reconstruction. We show that they obtain a larger depth of field but maintain the ability to reconstruct detail at high resolution. In fact, all depths are approximately focused, except for a thin slab where blur size is bounded, i.e., their depth of field is essentially inverted compared to regular cameras. Crucial to their success is the way they sample the LF, trading off spatial versus angular resolution, and how aliasing affects the LF. We show that applying traditional multiview stereo methods to the extracted low-resolution views can result in reconstruction errors due to aliasing. We address these challenges using an explicit image formation model, and incorporate Lambertian and texture preserving priors to reconstruct both scene depth and its superresolved texture in a variational Bayesian framework, eliminating aliasing by fusing multiview information. We demonstrate the method on synthetic and real images captured with our LF camera, and show that it can outperform other computational camera systems.
Computational photography, superresolution, deconvolution, blind deconvolution, multiview stereo, shape from defocus.
Tom E. Bishop, "The Light Field Camera: Extended Depth of Field, Aliasing, and Superresolution", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 5, pp. 972-986, May 2012, doi:10.1109/TPAMI.2011.168
[1] E.H. Adelson and J.Y. Wang, "Single Lens Stereo with a Plenoptic Camera," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 99-106, Feb. 1992.
[2] T. Georgeiv and C. Intwala, "Light Field Camera Design for Integral View Photography," technical report, Adobe Systems, 2006.
[3] R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, "Light Field Photography with a Hand-Held Plenoptic Camera," Technical Report CSTR 2005-02, Stanford Univ., Apr. 2005.
[4] A. Veeraraghavan, R. Raskar, A.K. Agrawal, A. Mohan, and J. Tumblin, "Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing," ACM Trans. Graphics, vol. 26, no. 3, p. 69, 2007.
[5] M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, "Light Field Microscopy," ACM Trans. Graphics, vol. 25, no. 3, pp. 924-934, 2006.
[6] A. Lumsdaine and T. Georgiev, "The Focused Plenoptic Camera," Proc. IEEE Int'l Conf. Computational Photography, Apr. 2009.
[7] A. Levin, W.T. Freeman, and F. Durand, "Understanding Camera Trade-Offs through a Bayesian Analysis of Light Field Projections," Proc. European Conf. Computer Vision, pp. 619-624, 2008.
[8] G. Lippmann, "Epreuves Reversibles Donnant la Sensation du Relief," J. Physics, vol. 7, no. 4, pp. 821-825, 1908.
[9] K. Fife, A. El Gamal, and H.-S. Wong, "A 3D Multi-Aperture Image Sensor Architecture," Proc. IEEE Custom Integrated Circuits Conf., pp. 281-284, 2006.
[10] C.-K. Liang, G. Liu, and H.H. Chen, "Light Field Acquisition Using Programmable Aperture Camera," Proc. IEEE Int'l Conf. Image Processing, pp. V233-236, 2007.
[11] M. Ben-Ezra, A. Zomet, and S. Nayar, "Jitter Camera: High Resolution Video from a Low Resolution Detector," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, 2004.
[12] W. Cathey and E. Dowski, "New Paradigm for Imaging Systems," Applied Optics, vol. 41, no. 29, pp. 6080-6092, 2002.
[13] H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, "Flexible Depth of Field Photography," Proc. 10th European Conf. Computer Vision, Oct. 2008.
[14] S.C. Park, M.K. Park, and M.G. Kang, "Super-Resolution Image Reconstruction: A Technical Overview," IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21-36, May 2003.
[15] A. Katsaggelos, R. Molina, and J. Mateos, Super Resolution of Images and Video. Morgan & Claypool, 2007.
[16] S. Borman and R. Stevenson, "Super-Resolution from Image Sequences—A Review," Proc. Midwest Symp. Circuits and Systems, pp. 374-378, 1999.
[17] M.K. Ng and A.C. Yau, "Super-Resolution Image Restoration from Blurred Low-Resolution Images," J. Math. Imaging and Vision, vol. 23, pp. 367-378, 2005.
[18] S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and Challenges in Super-Resolution," Int'l J. Imaging Systems and Technology, vol. 14, pp. 47-57, 2004.
[19] B.R. Hunt, "Super-Resolution of Images: Algorithms, Principles, Performance," Int'l J. Imaging Systems and Technology, vol. 6, no. 4, pp. 297-304, 2005.
[20] W.-S. Chan, E. Lam, M. Ng, and G. Mak, "Super-Resolution Reconstruction in a Computational Compound-Eye Imaging Systems," Multidimensional Systems and Signal Processing, vol. 18, no. 2, pp. 83-101, Sept. 2007.
[21] A. Lumsdaine and T. Georgiev, "Full Resolution Lightfield Rendering," technical report, Indiana Univ. and Adobe Systems, 2008.
[22] T. Georgiev and A. Lumsdaine, "Depth of Field in Plenoptic Cameras," Proc. Eurographics 2009, 2009.
[23] M. Levoy and P. Hanrahan, "Light Field Rendering," Proc. ACM Siggraph, pp. 31-42, 1996.
[24] J. Stewart, J. Yu, S.J. Gortler, and L. McMillan, "A New Reconstruction Filter for Undersampled Light Fields," Proc. 14th Eurographics Workshop Rendering, pp. 150-156, 2003.
[25] J.-X. Chai, S.-C. Chan, H.-Y. Shum, and X. Tong, "Plenoptic Sampling," Proc. ACM Siggraph, pp. 307-318, 2000.
[26] A. Isaksen, L. McMillan, and S.J. Gortler, "Dynamically Reparameterized Light Fields," Proc. ACM Siggraph, pp. 297-306, 2000.
[27] R. Ng, "Fourier Slice Photography," Proc. ACM Siggraph, vol. 24, no. 3, pp. 735-744, 2005.
[28] V. Vaish, M. Levoy, R. Szeliski, C. Zitnick, and S.B. Kang, "Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures," Proc. 26th IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2331-2338, 2006.
[29] T.E. Bishop, S. Zanetti, and P. Favaro, "Light Field Superresolution," Proc. IEEE Int'l Conf. Computational Photograph, Apr. 2009.
[30] T.E. Bishop and P. Favaro, "Plenoptic Depth Estimation from Multiple Aliased Views," Proc. 12th IEEE Int'l Conf. Conf. Computer Vision Workshops, 2009.
[31] T.E. Bishop, R. Molina, and J.R. Hopgood, "Blind Restoration of Blurred Photographs via AR Modelling and MCMC," Proc. IEEE 15th Int'l Conf. Image Processing, 2008.
[32] T.E. Bishop, "Blind Image Deconvolution: Nonstationary Bayesian Approaches to Restoring Blurred Photos," PhD dissertation, Univ. of Edinburgh, 2008.
[33] M. Born and E. Wolf, Principles of Optics. Pergamon, 1986.
[34] Z. Wang and F. Qi, "Analysis of Multiframe Super-Resolution Reconstruction for Image Anti-Aliasing and Deblurring," Image and Vision Computing, vol. 23, no. 4, pp. 393-404, Apr. 2005.
[35] D. Robinson and P. Milanfar, "Fundamental Performance Limits in Image Registration," IEEE Trans. Image Processing, vol. 13, no. 9, pp. 1185-1199, Sept. 2004.
[36] S. Baker and T. Kanade, "Limits on Super-Resolution and How to Break Them," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1167-1183, Sept, 2002.
[37] A. Papoulis, "Generalized Sampling Expansion," IEEE Trans. Circuits and Systems, vol. 24, no. 11, pp. 652-654, Nov. 1977.
[38] E.P. Simoncelli, "Statistical Modeling of Photographic Images," Handbook of Image and Video Processing, A. Bovik, ed., second ed., Academic Press, Jan. 2005.
[39] C. Zhou and S. Nayar, "What Are Good Apertures for Defocus Deblurring?" Proc. IEEE Int'l Conf. Computational Photography, 2009.
5 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool