The Community for Technology Leaders
RSS Icon
Issue No.11 - November (2010 vol.32)
pp: 2054-2070
Yuriy Vasilyev , Harvard University, Cambridge
Todd Zickler , Harvard University, Cambridge
Ohad Ben-Shahar , Ben-Gurion University of the Negev, Beer-Sheva
An image of a specular (mirror-like) object is nothing but a distorted reflection of its environment. When the environment is unknown, reconstructing shape from such an image can be very difficult. This reconstruction task can be made tractable when, instead of a single image, one observes relative motion between the specular object and its environment, and therefore, a motion field—or specular flow—in the image plane. In this paper, we study the shape from specular flow problem and show that observable specular flow is directly related to surface shape through a nonlinear partial differential equation. This equation has the key property of depending only on the relative motion of the environment while being independent of its content. We take first steps toward understanding and exploiting this PDE, and we examine its qualitative properties in relation to shape geometry. We analyze several cases in which the surface shape can be recovered in closed form, and we show that, under certain conditions, specular shape can be reconstructed when both the relative motion and the content of the environment are unknown. We discuss numerical issues related to the proposed reconstruction algorithms, and we validate our findings using both real and synthetic data.
Specular objects, specular flow, shape reconstruction, environment motion field, Gaussian curvature, parabolic points, specular curvature.
Yuriy Vasilyev, Todd Zickler, Ohad Ben-Shahar, "Shape from Specular Flow", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 11, pp. 2054-2070, November 2010, doi:10.1109/TPAMI.2010.126
[1] S. Waldon and C. Dyer, "Dynamic Shading, Motion Parallax and Qualitative Shape," Proc. IEEE Workshop Qualitative Vision, pp. 61-70, 1993.
[2] S. Roth and M. Black, "Specular Flow and the Recovery of Surface Structure," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1869-1876, 2006.
[3] S. Baker, D. Scharstein, J. Lewis, S. Roth, M. Black, and R. Szeliski, "A Database and Evaluation Methodology for Optical Flow," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[4] M. Osadchy, D. Jacobs, and R. Ramamoorthi, "Using Specularities for Recognition," Proc. IEEE Int'l Conf. Computer Vision, pp. 1512-1519, 2003.
[5] O. Drbohlav and R. Sára, "Specularities Reduce Ambiguity of Uncalibrated Photometric Stereo," Proc. European Conf. Computer Vision, pp. 46-60, 2000.
[6] O. Drbohlav and M. Chantler, "Can Two Specular Pixels Calibrate Photometric Stereo?" Proc. IEEE Int'l Conf. Computer Vision, pp. 1850-1857, 2005.
[7] J. Solem, H. Aanaes, and A. Heyden, "A Variational Analysis of Shape from Specularities Using Sparse Data," Proc. Conf. 3D Data Processing, Visualization, and Transmission, pp. 26-33, 2004.
[8] K. Ikeuchi, "Determining Surface Orientations of Specular Surfaces by Using the Photometric Stereo Method," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 6, pp. 661-669, Nov. 1981.
[9] S. Nayar, K. Ikeuchi, and T. Kanade, "Determining Shape and Reflectance of Hybrid Surfaces by Photometric Sampling," IEEE Trans. Robotics and Automation, vol. 6, no. 4, pp. 418-431, Aug. 1990.
[10] S. Savarese, M. Chen, and P. Perona, "Local Shape from Mirror Reflections," Int'l J. Computer Vision, vol. 64, no. 1, pp. 31-67, 2005.
[11] T. Bonfort, P. Sturm, and P. Gargallo, "General Specular Surface Triangulation," Proc. Asian Conf. Computer Vision, pp. 872-881, 2006.
[12] M. Knauer, J. Kaminski, and G. Häusler, "Phase Measuring Deflectometry: A New Approach to Measure Specular Free-Form Surfaces," Proc. Conf. SPIE, pp. 366-376, 2004.
[13] M. Tarini, H. Lensch, M. Goesele, and H. Seidel, "3D Acquisition of Mirroring Objects Using Striped Patterns," Graphical Models, vol. 67, pp. 233-259, 2005.
[14] S. Rozenfeld, I. Shimshoni, and M. Lindenbaum, "Dense Mirroring Surface Recovery from 1D Homographies and Sparse Correspondence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[15] J. Lellmann, J. Blazer, A. Reider, and J. Beyerer, "Shape from Specular Reflection and Optical Flow," Int'l J. Computer Vision, vol. 80, no. 2, pp. 226-241, Nov. 2008.
[16] K. Kutulakos and E. Steger, "A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation," Proc. IEEE Int'l Conf. Computer Vision, pp. 1448-1455, 2005.
[17] A. Blake, "Specular Stereo," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 973-976, 1985.
[18] A. Blake and G. Brelstaff, "Geometry from Specularities," Proc. IEEE Int'l Conf. Computer Vision, 1988.
[19] A. Zisserman, P. Giblin, and A. Blake, "The Information Available to a Moving Observer from Specularities," Image and Vision Computing, vol. 7, pp. 38-42, 1989.
[20] A. Blake and H. Bülthoff, "Does the Brain Know the Physics of Specular Reflection?" Nature, vol. 343, pp. 165-168, 1990.
[21] A. Blake and H. Bülthoff, "Shape from Specularities: Computation and Psychophysics," Philosophical Trans. Biological Sciences, vol. 331, no. 1260, pp. 237-252, 1991.
[22] J. Koenderink and A. van Doorn, "Photometric Invariants Related to Solid Shape," Optical Acta, vol. 27, no. 7, pp. 981-996, 1980.
[23] M. Longuet-Higgins, "Reflection and Refraction at a Random Moving Surface. I. Pattern and Paths of Specular Points," J. Optical Soc. Am., vol. 50, no. 9, pp. 838-844, 1960.
[24] M. Oren and S. Nayar, "A Theory of Specular Surface Geometry," Int'l J. Computer Vision, vol. 24, no. 2, pp. 105-124, 1997.
[25] J. Zheng, Y. Fukagawa, and N. Abe, "Shape and Model from Specular Motion," Proc. IEEE Int'l Conf. Computer Vision, pp. 72-79, 1995.
[26] J. Zheng and A. Murata, "Acquiring a Complete 3D Model from Specular Motion under the Illumination of Circular-Shaped Light Sources," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 913-920, Aug. 2000.
[27] S. Savarese, F. Li, and P. Perona, "What Do Reflections Tell Us about the Shape of a Mirror?" Proc. Symp. Applied Perception in Graphics and Visualization, 2004.
[28] R. Fleming, A. Torralba, and E. Adelson, "Specular Reflections and the Perception of Shape," J. Vision, vol. 4, pp. 798-820, 2004.
[29] U. Weidenbacher, P. Bayerl, H. Neumann, and R. Fleming, "Sketching Shiny Surfaces: 3D Shape Extraction and Depiction of Specular Surfaces," ACM Trans. Applied Perception, vol. 3, no. 3, pp. 262-285, 2006.
[30] A. DelPPozo and S. Savarese, "Detecting Specular Surfaces on Natural Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[31] D. Nister, H. Stewenius, and E. Grossmann, "Non-Parametric Self-Calibration," Proc. IEEE Int'l Conf. Computer Vision, vol. 1, 2005.
[32] S. Ramalingam, P. Sturm, and S. Lodha, "Towards Generic Self-Calibration of Central Cameras," Proc. Workshop Omnidirectional Vision, pp. 20-27, 2005.
[33] "Light Probe Gallery," http://www.debevec.orgProbes/, 2008.
[34] K. Ikeuchi and B. Horn, "Numerical Shape from Shading and Occluding Boundaries," Artificial Intelligence, vol. 17, pp. 141-184, 1981.
[35] O. Ben-Shahar and S. Zucker, "The Perceptual Organization of Texture Flows: A Contextual Inference Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 4, pp. 401-417, Apr. 2003.
[36] M. do Carmo, Differential Geometry of Curves and Surfaces. Prentice-Hall, Inc., 1976.
[37] B. Horn and B. Schunck, "Determining Optical Flow," Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[38] S. Agarwal, S. Mallick, D. Kriegman, and S. Belongie, "On Refractive Optical Flow," Proc. European Conf. Computer Vision, pp. 483-494, 2004.
[39] B. Hartung and D. Kersten, "Distinguishing Shiny from Matte," J. Vision, vol. 2, no. 7, p. 551, 2002.
[40] S. Roth, F. Domini, and M. Black, "Specular Flow and the Perception of Surface Reflectance," J. Vision, vol. 3, no. 9 p. 413, 2003.
32 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool