| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
Simultaneous Estimation of Super-Resolved Scene and Depth Map from Low Resolution Defocused Observations
September 2003 (vol. 25 no. 9)
pp. 1102-1117
Abstract—This paper presents a novel technique to simultaneously estimate the depth map and the focused image of a scene, both at a super-resolution, from its defocused observations. Super-resolution refers to the generation of high spatial resolution images from a sequence of low resolution images. Hitherto, the super-resolution technique has been restricted mostly to the intensity domain. In this paper, we extend the scope of super-resolution imaging to acquire depth estimates at high spatial resolution simultaneously. Given a sequence of low resolution, blurred, and noisy observations of a static scene, the problem is to generate a dense depth map at a resolution higher than one that can be generated from the observations as well as to estimate the true high resolution focused image. Both the depth and the image are modeled as separate Markov random fields (MRF) and a maximum a posteriori estimation method is used to recover the high resolution fields. Since there is no relative motion between the scene and the camera, as is the case with most of the super-resolution and structure recovery techniques, we do away with the correspondence problem.
[1] 1102 T. Komatsu, T. Igarashi, K. Aizawa, and T. Saito, Very High Resolution Imaging Scheme with Multiple Different-Aperture Cameras Signal Processing: Image Comm., vol. 5, pp. 511-526, Dec. 1993.[2] R.Y. Tsai and T.S. Huang, Multiframe Image Restoration and Registration Advances in Computer Vision and Image Processsing, pp. 317-339, JAI Press Inc., 1984.[3] M. Irani and S. Peleg, “Improving Resolution by Image Restoration,” Computer Vision, Graphics, and Image Processing, vol. 53, pp. 231-239, 1991.[4] H. Shekarforoush, M. Berthod, J. Zerubia, and M. Werman, “Sub-Pixel Bayesian Estimation of Albedo and Height,” Int'l J. Computer Vision, vol. 19, pp. 289-300, 1996.[5] R.S. Schultz and R.L. Stevenson, “Extraction of High-Resolution Frames from Video Sequences,” IEEE Trans. Image Processing, vol. 5, pp. 996-1,011, June 1996.[6] Super-Resolution Imaging, S. Chaudhuri, ed. Boston: Kluwer Academic, 2001.[7] A.P. Pentland, Depth of Scene from Depth of Field Proc. Image Understanding Workshop, pp. 253-259, 1982.[8] S. Chaudhuri and A.N. Rajagopalan, Depth from Defocused Images: A Real Aperture Imaging Approach. New York: Springer-Verlag, 1999.[9] M. Subbarao, Efficient Depth Recovery Through Inverse Optics Machine Vision for Inspection and Measurement, H. Freeman, ed., Academic Press, 1989.[10] A. Pentland, "A New Sense for Depth of Field, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 4, pp. 523-531, July 1987.[11] M. Subbarao, "Parallel Depth Recovery by Changing Camera Parameters," Proc. Int'l Conf. Computer Vision, pp. 149-155, Dec. 1988.[12] K.V. Prasad, R.J. Mammone, and J. Yogeshwar, 3D Image Restoration Using Constrained Optimization Techniques Optical Eng., vol. 29, no. 4, pp. 277-288, Apr. 1990.[13] W.N. Klarquist, W.S. Geisler, and A.C. Bovik, Maximum-Likelihood Depth-from-Defocus for Active Vision Proc. IEEE Int'l Conf. Intelligent Robots and Systems, 1995.[14] M. Gokstorp, "Computing Depth from Out-of-Focus Blur Using a Local Frequency Representation," Proc. Int'l Conf. Pattern Recognition, Oct. 1994.[15] M. Watanabe and S.K. Nayar, "Minimal Operator Set for Passive Depth From Defocus," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 431-438,San Francisco, June 1996.[16] Y.Y. Schechner and N. Kiryati, Depth from Defocus vs. Stereo: How Different Really Are They? Technical Report EE PUB No. 1155, Technion Israel Inst. of Tech nology, May 1998.[17] S.K. Nayar, M. Watanabe, and M. Noguchi, "Real-Time Focus Range Sensor," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, pp. 1,186-1,198, Dec. 1996.[18] A.N. Rajagopalan and S. Chaudhuri, “Space-Variant Approaches to Recovery of Depth from Defocused Images,” Computer Vision and Image Understanding, vol. 68, no. 3, pp. 309-329, Dec. 1997.[19] A.N. Rajagopalan and S. Chaudhuri, “A Variational Approach to Recovering Depth from Defocused Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 10, pp. 1,158-1,165 Oct. 1997.[20] A.N. Rajagopalan and S. Chaudhuri, An MRF Model Based Approach to Simultaneous Recovery of Depth and Restoration from Defocused Images IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 7, pp. 577-589, July 1999.[21] M.K. Ozkan, A.M. Tekalp, and M.I. Sezan, POCS Based Restoration of Space Varying Blurred Images IEEE Trans. Image Processing, vol. 3, no. 4, pp. 450-454, July 1994.[22] S. Koch, H. Kaufmann, and J. Biemond, Restoration of Spatially Varying Blurred Images Using Multiple Model-Based Extended Kalman Filter IEEE Trans. Image Processing, vol. 4, no. 4, pp. 520-523, Apr. 1995.[23] H. Trussel and S. Fogel, Identification and Restoration of Spatially Variant Motion Blurs in Sequential Images IEEE Trans. Image Processing, vol. 1, no. 1, pp. 123-126, Jan. 1992.[24] S. Kim, N. Bose, and H. Valenzuela, “Recursive Reconstruction of High Resolution Image from Noisy Undersampled Multiframes,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 38, pp. 1013-1027, 1990.[25] C. Srinivas, M.D. Srinath, A Stochastic Model Based Approach for Simultaneous Restoration of Multiple Mis-Registered Images Int'l Soc. Optical Eng., vol. 1360, pp. 1416-1427, 1990.[26] H. Ur and D. Gross, “Improved Resolution from Subpixel Shifted Pictures,” Computer Vision, Graphics, and Image Processing, vol. 54, no. 2, pp. 181-186, 1992.[27] D. Jackson, "Structuring Z Specifications with Views, ACM Trans. on Software Engineering and Methodology, vol. 4, no. 4, pp. 365-389, Oct. 1995.[28] R.R. Schultz and R.L. Stevenson, "A Bayesian Approach to Image Expansion for Improved Definition," IEEE Trans. Image Processing, vol. 3, 1994.[29] P. Ciaccia, M. Patella, and P. Zezula, “Processing Complex Similarity Queries with Distance-Based Access Methods,” Proc. Sixth Int'l Conf. Extending Database Technology, Mar. 1998.[30] M. Elad and A. Feuer, “Super-Resolution Restoration of Continuous Image Sequence—Adaptive Filtering Approach,” IEEE Trans. Image Processing, vol. 8, no. 3, pp. 387-395, Marc. 1999.[31] M.-C. Chiang and T.E. Boult, “Local Blur Estimation and Super-Resolution,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 821-826, 1997.[32] D. Rajan and S. Chaudhuri, A Generalized Interpolation Scheme for Image Scaling and Super-Resolution Proc. Erlangen Workshop Vision, Modeling, and Visualization, Univ. of Erlangen-Nuremberg, Germany, pp. 301-308, Nov. 1999.[33] D. Rajan and S. Chaudhuri, Generalized Interpolation and Its Application in Super-Resolution Imaging Image and Vision Computing, vol. 19, no. 13, pp. 957-969, Nov. 2001.[34] P. Cheeseman, B. Kanefsky, R. Kraft, J. Stutz, and R. Hanson, Super-Resolved Surface Reconstruction from Multiple Images Technical Report FIA-94-12, NASA Ames Research Center, Moffet Field, Calif., Dec. 1994.[35] R.C. Dubes and A.K. Jain, Random Field Models in Image Analysis J. Applied Statistics, vol. 16, no. 2, pp. 131-164, 1989.[36] J. Besag, Spatial Interaction and the Statistical Analysis of Lattice Systems J. Royal Statistical Soc., Series B, vol. 36, pp. 192-236, 1974.[37] S. Geman and D. Geman, Stochastic Relaxation, Gibbs Distribution and the Bayesian Restoration of Image IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721-741, 1984.[38] A. Blake and A. Zisserman, Visual Reconstruction. MIT Press, 1987.[39] J. Subrahmonia, Y.P. Hung, and D.B. Cooper, Model-Based Segmentation and Estimation of 3D Surfaces from Two or More Intensity Images Using Markov Random Fields Proc. IEEE Int'l Conf. Pattern Recognition, pp. 390-397, 1990.[40] D.B. Cooper, J. Subrahmonia, Y.P. Hung, and B. Cernuschi-Frias, The Use of Markov Random Fields in Estimating and Recognizing Objects in 3D Space Markov Random Fields: Theory and Applications, R. Chellappa and A. Jain, eds. Academic Press, 1993.[41] D. Rajan, Some New Approaches to Generation of Super-Resolution Images PhD thesis, School of Biomedical Eng., Indian Inst. of Technology, Bombay, 2001.
Index Terms:
Super-resolution, depth from defocus, space-variant blur identification, restoration, Markov random field.
Citation:
Deepu Rajan, Subhasis Chaudhuri, "Simultaneous Estimation of Super-Resolved Scene and Depth Map from Low Resolution Defocused Observations," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1102-1117, Sept. 2003, doi:10.1109/TPAMI.2003.1227986
