This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Geometric Approach to Shape from Defocus
March 2005 (vol. 27 no. 3)
pp. 406-417
We introduce a novel approach to shape from defocus, i.e., the problem of inferring the three-dimensional (3D) geometry of a scene from a collection of defocused images. Typically, in shape from defocus, the task of extracting geometry also requires deblurring the given images. A common approach to bypass this task relies on approximating the scene locally by a plane parallel to the image (the so-called equifocal assumption). We show that this approximation is indeed not necessary, as one can estimate 3D geometry while avoiding deblurring without strong assumptions on the scene. Solving the problem of shape from defocus requires modeling how light interacts with the optics before reaching the imaging surface. This interaction is described by the so-called point spread function (PSF). When the form of the PSF is known, we propose an optimal method to infer 3D geometry from defocused images that involves computing orthogonal operators which are regularized via functional singular value decomposition. When the form of the PSF is unknown, we propose a simple and efficient method that first learns a set of projection operators from blurred images and then uses these operators to estimate the 3D geometry of the scene from novel blurred images. Our experiments on both real and synthetic images show that the performance of the algorithm is relatively insensitive to the form of the PSF. Our general approach is to minimize the Euclidean norm of the difference between the estimated images and the observed images. The method is geometric in that we reduce the minimization to performing projections onto linear subspaces, by using inner product structures on both infinite and finite-dimensional Hilbert spaces. Both proposed algorithms involve only simple matrix-vector multiplications which can be implemented in real-time.

[1] N. Asada, H. Fujiwara, and T. Matsuyama, “Edge and Depth from Focus,” Int'l J. Computer Vision, vol. 26, no. 2, pp. 153-163, 1998.
[2] M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging. Inst. of Physics Publications, 1998.
[3] S.S. Bhasin and S. Chaudhuri, “Depth from Defocus in Presence of Partial Self Occlusion,” Proc. Int'l Conf. Computer Vision, vol. 1, no. 2, pp. 488-93, 2001.
[4] A. Blake and A. Zisserman, Visual Reconstruction. MIT Press, 1987.
[5] M. Born and E. Wolf, Principles of Optics. Pergamon Press, 1980.
[6] S. Chaudhuri and A. Rajagopalan, Depth from Defocus: A Real Aperture Imaging Approach. Springer Verlag, 1999.
[7] J. Ens and P. Lawrence, “An Investigation of Methods for Determining Depth from Focus,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, pp. 97-108, 1993.
[8] O. Faugeras, Three Dimensional Vision, a Geometric Viewpoint. MIT Press, 1993.
[9] P. Favaro, A. Mennucci, and S. Soatto, “Observing Shape from Defocused Images,” Int'l J. Computer Vision, vol. 52, no. 1, pp. 25-43, Apr. 2003.
[10] P. Favaro and S. Soatto, “Learning Shape from Defocus,” Proc. European Conf. Computer Vision, vol. 2, pp. 735-745, 2002.
[11] P. Favaro and S. Soatto, “Seeing beyond Occlusions (and Other Marvels of a Finite Lens Aperture),” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '03), pp. II: 579-586, 2003.
[12] P. Favaro and S. Soatto, “Shape and Radiance Estimation from the Information Divergence of Blurred Images,” Proc. European Conf. Computer Vision, pp. 755-768, June 2000.
[13] D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Prentice Hall, 2002.
[14] B. Girod and S. Scherock, “Depth from Focus of Structured Light,” Proc. SPIE, pp. 209-15, 1989.
[15] M. Gokstorp, “Computing Depth from Out-of-Focus Blur Using a Local Frequency Representation,” Proc. Int'l Conf. Pattern Recognition, pp. A:153-158, 1994.
[16] G. Golub and V. Pereyra, “The Differentiation of Pseudo-Inverses and Nonlinear Least-Squares Problems Whose Variables Separate,” SIAM J. Numerical Analysis, vol. 10, no. 2, pp. 413-532, 1973.
[17] D. Luenberger, Optimization by Vector Space Methods. Wiley, 1968.
[18] Y. Ma, S. Soatto, J. Kosecka, and S. Sastry, An Invitation to 3D Vision, from Images to Models. Springer Verlag, 2003.
[19] J. Marshall, C. Burbeck, and D. Ariely, “Occlusion Edge Blur: A Cue to Relative Visual Depth,” Intl J. Optical Soc. Am. A, vol. 13, pp. 681-688, 1996.
[20] H.N. Nair and C.V. Stewart, “Robust Focus Ranging,” Computer Vision and Pattern Recognition, pp. 309-314, 1992.
[21] S.K. Nayar and Y. Nakagawa, “Shape from Focus: An Effective Approach for Rough Surfaces,” IEEE Int'l Conf. Robotics and Automation, pp. 218-225, 1990.
[22] S.K. Nayar, M. Watanabe, and M. Noguchi, “Real-Time Focus Range Sensor,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 12, pp. 1186-1198, Dec. 1996.
[23] S.K. Nayar and S.G. Narasimhan, “Vision in Bad Weather,” Proc. Int'l Conf. Computer Vision (ICCV '99), pp. 820-827, 1999.
[24] M. Noguchi and S.K. Nayar, “Microscopic Shape from Focus Using Active Illumination,” Proc. Int'l Conf. Pattern Recognition, pp. 147-152, 1994.
[25] A. Pentland, “A New Sense for Depth of Field,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, pp. 523-531, 1987.
[26] A. Pentland, T. Darrell, M. Turk, and W. Huang, “A Simple, Real-Time Range Camera,” Computer Vision and Pattern Recognition, pp. 256-261, 1989.
[27] A. Pentland, S. Scherock, T. Darrell, and B. Girod, “Simple Range Cameras Based on Focal Error,” J. Optical Soc. Am. A, vol. 11, no. 11, pp. 2925-2934, Nov. 1994.
[28] A.N. Rajagopalan and S. Chaudhuri, “A Block Shift-Variant Blur Model for Recovering Depth from Defocused Images,” Proc. Int'l Conf. Image Processing, pp. 636-39, 1995.
[29] A.N. Rajagopalan and S. Chaudhuri, “Optimal Selection of Camera Parameters for Recovery of Depth from Defocused Images,” Computer Vision and Pattern Recognition, pp. 219-224, 1997.
[30] A.N. Rajagopalan and S. Chaudhuri, “Optimal Recovery of Depth from Defocused Images Using an MRF Model,” Proc. Int'l Conf. Computer Vision, pp. 1047-1052, 1998.
[31] Y. Schechner and N. Kiryati, “The Optimal Axial Interval in Estimating Depth from Defocus,” Proc. Int'l Conf. Computer Vision, pp. 843-848, 1993.
[32] G. Schneider, B. Heit, J. Honig, and J. Bremont, “Monocular Depth Perception by Evaluation of the Blur in Defocused Images,” Proc. Int'l Conf. Image Processing, vol. 2, pp. 116-9, 1994.
[33] S. Soatto and P. Favaro, “A Geometric Approach to Blind Deconvolution with Application to Shape from Defocus,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 10-17, June 2000.
[34] M. Subbarao and N. Gurumoorthy, “Depth Recovery from Blurred Edges,” Computer Vision and Pattern Recognition, pp. 498-503, 1988.
[35] M. Subbarao and G. Surya, “Depth from Defocus: A Spatial Domain Approach,” Int'l J. Computer Vision, vol. 13, pp. 271-294, 1994.
[36] M. Watanabe and S. Nayar, “Rational Filters for Passive Depth from Defocus,” Int'l J. Computer Vision, vol. 27, no. 3, pp. 203-225, 1998.
[37] M. Watanabe and S.K. Nayar, “Minimal Operator Set for Passive Depth from Defocus,” Computer Vision and Pattern Recognition, pp. 431-438, 1996.
[38] Y. Xiong and S. Shafer, “Depth from Focusing and Defocusing,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 68-73, 1993.
[39] Y. Xiong and S.A. Shafer, “Moment Filters for High Precision Computation of Focus and Stereo,” Proc. Int'l Conf. Intelligent Robots and Systems, pp. 108-113, Aug. 1995.
[40] L. Yen-Fu, “A Unified Approach to Image Focus and Defocus Analysis,” Dept. of Electrical and Computer Eng., State Univ. of New York at Stony Brook, 1998.
[41] D. Ziou and F. Deschenes, “Depth from Defocus Estimation in Spatial Domain,” Computer Vision and Image Understanding, vol. 81, no. 2, pp. 143-165, Feb. 2001.

Index Terms:
Shape from defocus, depth from defocus, blind deconvolution, image processing, deblurring, shape, 3D reconstruction, shape estimation, image restoration, learning subspaces.
Citation:
Paolo Favaro, Stefano Soatto, "A Geometric Approach to Shape from Defocus," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 406-417, March 2005, doi:10.1109/TPAMI.2005.43
Usage of this product signifies your acceptance of the Terms of Use.