This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Real-Time Focus Range Sensor
December 1996 (vol. 18 no. 12)
pp. 1186-1198

Abstract—Structures of dynamic scenes can only be recovered using a real-time range sensor. Depth from defocus offers an effective solution to fast and dense range estimation. However, accurate depth estimation requires theoretical and practical solutions to a variety of problems including recovery of textureless surfaces, precise blur estimation, and magnification variations caused by defocusing. Both textured and textureless surfaces are recovered using an illumination pattern that is projected via the same optical path used to acquire images. The illumination pattern is optimized to maximize accuracy and spatial resolution in computed depth. The relative blurring in two images is computed using a narrow-band linear operator that is designed by considering all the optical, sensing, and computational elements of the depth from defocus system. Defocus invariant magnification is achieved by the use of an additional aperture in the imaging optics. A prototype focus range sensor has been developed that has a workspace of 1 cubic foot and produces up to 512 × 480 depth estimates at 30 Hz with an average RMS error of 0.2%. Several experimental results are included to demonstrate the performance of the sensor.

[1] M. Born and E. Wolf, Principles of Optics.London: Permagon, 1965.
[2] R. Kingslake, Optical System Design. Academic Press, 1983.
[3] R.N. Bracewell, The Fourier Transform and Its Applications. McGraw Hill, 1965.
[4] B.K. Horn, Robot Vision. Cambridge, Mass.: MIT Press, 1986.
[5] B.K.P. Horn, Focusing, AI Lab, Memo 160. MIT, Cambridge, Mass, 1968.
[6] T. Kanade, "Development of a Video-Rate Stereo Machine," Proc. ARPA Image Understanding Workshop, Nov. 1994, pp. 549-557"
[7] B. Girod and S. Scherock, "Depth from Focus of Structured Light," Proc. SPIE: Optics, Illum., and Image Sng for Mach. Vis. IV, vol. 1194, Nov. 1989,Philadelphia, Penn.
[8] R.A. Jarvis, "A Perspective on Range Finding Techniques for Computer Vision," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 2, pp. 122-139, March 1983.
[9] K. L. Boyer and A. C. Kak, "Color-Encoded Structured Light for Rapid Active Sensing," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 1, pp. 14-28, Jan. 1987.
[10] A. Pentland, "A New Sense for Depth of Field, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 4, pp. 523-531, July 1987.
[11] A. Pentland, S. Scherock, T. Darrell, and B. Girod, "Simple Range Cameras Based on Focal Error," J. Optical Society of America, vol. 11, no. 11, pp. 2,925-2,935, Nov. 1994
[12] A. Pentland, T. Darrell, M. Turk, and W. Huang, "A Simple, Real-Time Range Camera," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 256-261, June 1989,San Diego, Calif.
[13] M. Subbarao, "Parallel Depth Recovery by Changing Camera Parameters," Proc. Int'l Conf. Computer Vision, pp. 149-155, Dec. 1988.
[14] M. Gokstorp, "Computing Depth from Out-of-Focus Blur Using a Local Frequency Representation," Proc. Int'l Conf. Pattern Recognition, Oct. 1994.
[15] M. Subbarao and G. Surya, "Application of Spatial-Domian Convolution/Deconvolution Transform for Determining Distance from Image Defocus," Proc. OE/BOSTON '92 SPIE Conf., vol. 1,822, Boston, Mass., Nov. 1992
[16] Y. Xiong and S.A. Shafer, Moment and Hypergeometric Filters for High Precision Computation of Focus, Stereo and Optical Flow. The Robotics Institute: Carnegie Mellon University, no. CMU-RI-TR-94-28, Pittsburgh, Penn., Sept. 1994.
[17] P.J. Besl, Range Imaging Sensors. General Motors Research Laboratories no. GMR-6090, March 1988.
[18] Y. Xiong and S.A. Shafer, "Variable Window Gabor Filters and Their Use in Focus and Correspondence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 668-671, June 1994.
[19] S.K. Nayar, M. Watanabe, and M. Noguchi, "Real-Time Focus Range Sensor," Proc. Int'l Conf. Computer Vision, pp. 995-1,001, June 1995.
[20] M. Watanabe, S.K. Nayar, and M. Noguchi, "Real-Time Implementation of Depth from Defocus," Proc. SPIE Conf.,Philadelphia, Penn., Oct. 1995.
[21] M. Watanabe and S. K. Nayar, "Telecentric Optics for Computational Vision," Proc. European Conf. Computer Vision,Cambridge, U.K., Apr. 1996
[22] M. Watanabe and S.K. Nayar, "Minimal Operator Set for Passive Depth From Defocus," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 431-438,San Francisco, June 1996.
[23] S. Inokuchi, K. Sato, and F. Matsuda, "Range Imaging System for 3D Object Recognition," Proc. Seventh Int'l Conf. Pattern Recognition, pp. 806-808, July 1984.
[24] A. Gruss, S. Tada, and T. Kanade, "A VLSI Smart Sensor for Fast Range Imaging," Proc. ARPA Image Understanding Workshop, pp. 977-986, Apr. 1993,Washington, D.C.
[25] T. Kanade, A. Gruss, and L.R. Carley, "A Very Fast VLSI Rangefinder," Proc. Int'l Conf. Robotics and Automation, pp. 1,322-1,329, Apr. 1991,Sacramento, Calif.
[26] T. Kanade, "Development of a Video-Rate Stereo Machine," Proc. ARPA Image Understanding Workshop, pp. 549-557, Nov. 1994.
[27] E. Krotkov, "Focusing," Int'l J. Computer Vision vol. 1, pp. 223-237, 1987.
[28] T. Darrell and K. Wohn, "Pyramid Based Depth From Focus," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 504-509, June 1988.
[29] S.K. Nayar and Y. Nakagawa, "Shape from Focus," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 8, pp. 824-831, Aug. 1994.
[30] H.N. Nair and C.V. Stewart, "Robust Focus Ranging," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 309-314, June 1988.
[31] A. Krishnan and N. Ahuja, "Range Estimation From Focus Using a Non-Frontal Imaging Camera," Proc. AAAI Conf., pp. 830-835, July 1993.
[32] N. Asada, H. Fujiwara, and T. Matsuyama, "Edge and Depth from Focus," Proc. Asian Conf. Computer Vision pp. 83-86,Osaka, Japan, Nov. 1993.
[33] P. Grossman, "Depth from Focus," Pattern Recognition, vol. 9, no. 1, pp. 63-69, 1987.
[34] V.M. Bove, Jr., "Discrete Fourier Transform Based Depth-From-Focus," Proc. OSA Topical Meeting Image Understanding and Machine Vision, 1989.
[35] J. Ens and P. Lawrence, "A Matrix Based Method for Determining Depth from Focus," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 600-609, June 1991.
[36] M. Gokstorp, "Computing Depth from Out-of-Focus Blur Using a Local Frequency Representation," Proc. Int'l Conf. Pattern Recognition, Oct. 1994.
[37] R.G. Willson and S.A. Shafer, "Modeling and Calibration of Automated Zoom Lenses," PhD dissertation, The Robotics Institute, Carnegie Mellon University, CMU-RI-TR-94-03, Jan. 1994.

Index Terms:
Depth from defocus, constant magnification defocusing, active illumination pattern, optical transfer function, image sensing, tuned focus operator, depth estimation, real-time range sensor.
Citation:
Shree K. Nayar, Masahiro Watanabe, Minori Noguchi, "Real-Time Focus Range Sensor," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 12, pp. 1186-1198, Dec. 1996, doi:10.1109/34.546256
Usage of this product signifies your acceptance of the Terms of Use.