This Article 
 Bibliographic References 
 Add to: 
Surface Shape Reconstruction of a Nonrigid Transport Object Using Refraction and Motion
October 1992 (vol. 14 no. 10)
pp. 1045-1052

The appearance of a pattern behind a transparent, moving object is distorted by refraction at the moving object's surface. An algorithm for reconstructing the surface shape of a nonrigid transparent object, such as water, from the apparent motion of the observed pattern is described. This algorithm is based on the optical and statistical analysis of the distortions. It consists of four steps: extraction of optical flow, averaging of each point trajectory obtained from the optical flow sequence, calculation of the surface normal using optical characteristics, and reconstruction of the surface. The algorithm is applied to both synthetic and real images to demonstrate its performance.

[1] D. H. Ballard and C. M. Brown,Computer Vision. Englewood Cliffs, NJ: Prentice-Hall, 1982.
[2] W. E. L. Grimson,From Images to Surfaces: A Computational Study of the Human Early visual System. Cambridge, MA: MIT Press, 1981.
[3] B. Julesz, "Binocular depth perception of computer-generated patterns,"Bel Syst. Tech. J., vol. 39, pp. 1125-1162, 1960.
[4] B. K. P. Horn,Robot Vision. Cambridge, MA: M.I.T. Press, 1986.
[5] K. Ikeuchi and B. K. P. Horn, "Numerical shape from shading and occluding boundaries,"Artificial Intell., vol. 17, pp. 141-184, 1981.
[6] M. Brady and A. Yuille, "An extremum principle from shape from contour,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-6, pp. 288-301, 1984.
[7] A. P. Witkin, "Recovering surface shape and orientation from texture,"Artificial Intell., vol. 17, pp. 17-45, 1981.
[8] K. A. Stevens, "The visual interpretation of surface contours,"Artificial Intell., vol. 17, pp. 47-74, 1981.
[9] J. Aloimonos, "Shape from texture,"Biol. Cybern., vol. 58, pp. 345-360, 1988.
[10] H. C. Longuet-Higgins, "The visual ambiguity of a moving plane," inProc. Roy. Soc. London Ser. B, vol. 223, pp. 165-175, 1984.
[11] R. Y. Tsai and T. S. Huang, "Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-6, pp. 13-27, 1984.
[12] M. Subbaro, "Interpretation of image flow: Rigid curved surface in motion,"Int. J. Comput. Vision, vol. 2, pp. 77-96, 1988.
[13] K. Sugihara and N. Sugie, "Recovery of rigid structure from orthographically projected optical flow,"Comp. Vision Graphics Image Processing, vol. 27, pp. 309-320, 1984.
[14] K. ichi Kanatani, "Structure and motion from optical flow under perspective projection,"Comput. Vision Graphics Image Processing, vol. 38, pp. 122-146, May 1987.
[15] H. -H. Nagel, "Representation of moving rigid objects based on visual observations,"IEEE Comput., vol. C-14, pp. 29-39, 1981.
[16] O. D. Faugeras, F. Lustman, and G. Toscani, "Motion and structure from motion from point and line matches," inProc. First Int. Conf. Comput. Vision(London, UK), 1987, pp. 25-34.
[17] S. Ullman,The Interpretation of Visual Motion. Cambridge, MA: MIT Press, 1979.
[18] J. J. Gibson,The Ecological Approach to Visual Perception. Boston, MA: Houghton Mifflin, 1979.
[19] D. Marr,Vision. New York: W. H. Freeman, 1982.
[20] B. L. Gotwols and G. B. Irani, "Charge-coupled device camera system for remotely measuring the dynamics of ocean waves,"Appl. Opt., vol. 21, no. 5, pp. 851-860, Mar. 1982
[21] B. Jahne, "Image sequence analysis of complex physical objects: Nonlinear small water surface waves," inProc. 1st Int. Conf. Comput. Vision, 1987, pp. 191-200.
[22] P. Y. Ts'o and B. A. Brasky, "Modeling and rendering waves: Wave-tracing using beta-splines and reflective and refractive texture mapping,"ACM Trans. Graphics, vol. 6, no. 3, pp. 191-214, July 1987.
[23] B. K. P. Horn, "Determining optical flow,"Artificial Intell., vol. 17, pp. 185-203, 1981.
[24] J. M. Prager and M. A. Arbib, "Computing the optic flow: The MATCH algorithm and prediction,"Comput. Vision Graphics Image Processing, vol. 24, pp. 213-237, 1984.

Index Terms:
surface shape reconstruction; image reconstruction; nonrigid transport object; refraction; apparent motion; statistical analysis; optical flow sequence; image reconstruction; light refraction; optical information processing; statistical analysis
H. Murase, "Surface Shape Reconstruction of a Nonrigid Transport Object Using Refraction and Motion," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 10, pp. 1045-1052, Oct. 1992, doi:10.1109/34.159906
Usage of this product signifies your acceptance of the Terms of Use.