The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2013 vol.35)
pp: 639-652
A. Netz , Comput. Sci. Dept., Univ. of Haifa, Ariel, Israel
M. Osadchy , Comput. Sci. Dept., Univ. of Haifa, Haifa, Israel
ABSTRACT
We present a novel approach to pose estimation and model-based recognition of specular objects in difficult viewing conditions, such as low illumination, cluttered background, large highlights, and shadows that appear on the object of interest. In such challenging conditions, conventional features are unreliable. We show that under the assumption of a dominant light source, specular highlights produced by a known object can be used to establish correspondence between its image and the 3D model, and to verify the hypothesized pose and the identity of the object. Previous methods that use highlights for recognition make limiting assumptions such as known pose, scene-dependent calibration, simple shape, etc. The proposed method can efficiently recognize free-form specular objects in arbitrary pose and under unknown lighting direction. It uses only a single image of the object as its input and outputs object identity and the full pose. We have performed extensive experiments for both recognition and pose estimation accuracy on synthetic images and on real indoor and outdoor images.
INDEX TERMS
Lighting, Estimation, Shape, Solid modeling, Three dimensional displays, Image recognition, Databases, specularities, Object recognition, varying illumination, pose estimation, invariants
CITATION
A. Netz, M. Osadchy, "Recognition Using Specular Highlights", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 3, pp. 639-652, March 2013, doi:10.1109/TPAMI.2012.127
REFERENCES
[1] T.D. Alter, "3D Pose from 3 Corresponding Points under Weak-Perspective Projection," technical report, Massachusetts Inst. of Technology, Artificial Intelligence Laboratory, 1992.
[2] B. Barrois and C. Wohler, "3D Pose Estimation Based on Multiple Monocular Cues," Proc. Workshop towards Benchmarking Automated Calibration, Orientation and Surface Reconstruction from Images, pp. 1-8, 2007.
[3] R. Basri and D.W. Jacobs, "Lambertian Reflectance and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[4] J.S. Beis and D.G. Lowe, "Indexing without Invariants in 3D Object Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 1000-1015, Oct. 1999.
[5] J.W. Birk, R.B. Kelly, and H.A.S. Martines, "An Orienting Robot for Feeding Workpieces Stored in Bins," IEEE Trans. Systems, Man, and Cybernetics, vol. 11, no. 2, pp. 151-160, Feb. 1981.
[6] A. Blake and G. Brelstaff, "Geometry from Specularities," Proc. IEEE Int'l Conf. Computer Vision, pp. 394-403, 1988.
[7] A. Blake and H. Bulthoff, "Shape from Specularities: Computation and Psychophysics," Philosophical Trans. Royal Soc. B: Biological Sciences, vol. 331, no. 1260, pp. 237-252, 1991.
[8] J.F. Blinn, "Models of Light Reflection for Computer Synthesized Pictures," SIGGRAPH Computer Graphics, vol. 11, pp. 192-198, 1977.
[9] T.A. Cass, "Polynomial-Time Object Recognition in the Presence of Clutter, Occlusion, and Uncertainty," Proc. European Conf. Computer Vision, pp. 834-842, 1992.
[10] J.Y. Chang, R. Raskar, and A.K. Agrawal, "3D Pose Estimation and Segmentation Using Specular Cues," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1706-1713, 2009.
[11] A. Collet, M. Martinez, and S.S. Srinivasa, "The Moped Framework: Object Recognition and Pose Estimation for Manipulation," The Int'l J. Robotics Research, vol. 30, pp. 1284-1306, 2011.
[12] P. David and D. DeMenthon, "Object Recognition in High Clutter Images Using Line Features," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1581-1588, 2005.
[13] A. DelPozo and S. Savarese, "Detecting Specular Surfaces on Natural Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, 2007.
[14] J. Flusser and T. Suk, "Pattern Recognition by Affine Moment Invariants," Pattern Recognition, vol. 26, no. 1, pp. 167-174, 1993.
[15] Y. Fukada, H. Doi, K. Nagamine, and T. Inari, "Relationships-Based Recognition of Structural Industrial Parts Stacked in Bin," Robotica, vol. 2, pp. 147-154, 1984.
[16] A.S. Georghiades, "Incorporating the Torrance and Sparrow Model of Reflectance in Uncalibrated Photometric Stereo," Proc. IEEE Int'l Conf. Computer Vision, pp. 816-825, 2003.
[17] R. Gershon, A.D. Jepson, and J.K. Tsotsos, "The Use of Color in Highlight Identification," Proc. 10th Int'l Joint Conf. Artificial Intelligence, vol. 2, pp. 752-754, 1987.
[18] K.D. Gremban and K. Ikeuchi, "Planning Multiple Observations for Object Recognition," Int'l J. Computer Vision, vol. 12, no. 2, pp. 137-172, 1994.
[19] B.M. Haralick, C.N. Lee, K. Ottenberg, and M. Nölle, "Review and Analysis of Solutions of the Three Point Perspective Pose Estimation Problem," Int'l J. Computer Vision, vol. 13, no. 3, pp. 331-356, 1994.
[20] E. Hsiao, A. Collet, and M. Hebert, "Making Specific Features Less Discriminative to Improve Point-Based 3D Object Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2653-2660, 2010.
[21] D.P. Huttenlocher and S. Ullman, "Object Recognition Using Alignment," Proc. IEEE Int'l Conf. Computer Vision, vol. 87, pp. 102-111, 1987.
[22] D. Jacobs and R. Basri, "3-D to 2-D Pose Determination with Regions," Int'l J. Computer Vision, vol. 34, no. 2, pp. 123-145, 1999.
[23] U. Klank, D. Pangercic, R.B. Rusu, and M. Beetz, "Real-Time CAD Model Matching for Mobile Manipulation and Grasping," Proc. Ninth IEEE-RAS Int'l Conf. Humanoid Robots, Dec. 2009.
[24] G.J. Klinker, S.A. Shafer, and T. Kanade, "The Measurement of Highlights in Color Images," Int'l J. Computer Vision, vol. 2, no. 1, pp. 7-32, 1988.
[25] K. Koshikawa and Y. Shirai, "A 3-D Modeler for Vision Research," Proc. Int'l Conf. Advanced Robotics, pp. 185-190, 1985.
[26] H.W. Kuhn, "The Hungarian Method for the Assignment Problem," Naval Research Logistic Quarterly, vol. 2, pp. 83-97, 1955.
[27] P. Lagger, M. Salzmann, V. Lepetit, and P. Fua, "3D Pose Refinement from Reflections," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[28] Y. Lamdan and H.J. Wolfson, "Geometric Hashing: A General and Efficient Model-Based Recognition Scheme," Proc. IEEE Int'l Conf. Computer Vision, pp. 238-249, 1988.
[29] D.G. Lowe, "Fitting Parameterized Three-Dimensional Models to Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 5, pp. 441-450, May 1991.
[30] K. McHenry, J. Ponce, and D. Forsyth, "Finding Glass," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 973-979, 2005.
[31] J.F. Norman, J.T. Todd, and G.A. Orban, "Perception of Three-Dimensional Shape from Specular Highlights, Deformations of Shading, and Other Types of Visual Information," Psychological Science, vol. 15, no. 8, pp. 565-570, 2004.
[32] M. Oren and S.K. Nayar, "A Theory of Specular Surface Geometry," Int'l J. Computer Vision, vol. 24, no. 2, pp. 105-124, 1997.
[33] F. Ortiz and F. Torres, "Automatic Detection and Elimination of Specular Reflectance in Color Images by Means of MS Diagram and Vector Connected Filters," IEEE Trans. Systems, Man, and Cybernetics, Part C: Applications and Rev., vol. 36, no. 5, pp. 681-687, Sept. 2006.
[34] M. Osadchy, D. Jacobs, R. Ramamoorthi, and D. Tucker, "Using Specularities in Comparing 3D Models and 2D Images," Computer Vision and Image Understanding, vol. 111, no. 3, pp. 275-294, 2008.
[35] M.J.D. Powell, "A Fast Algorithm for Nonlinearly Constrained Optimization Calculations," Numerical Analysis, G.A. Watson, ed., pp. 144-157, Springer Verlag, 1978.
[36] R. Ramamoorthi and P. Hanrahan, "On the Relationship between Radiance and Irradiance: Determining the Illumination from Images of a Convex Lambertian Object," J. Optical Soc. Am., vol. 18, no. 10, pp. 2448-2459, 2001.
[37] S. Roth and M.J. Black, "Specular Flow and the Recovery of Surface Structure," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1869-1876, 2006.
[38] F. Rothganger, S. Lazebnik, C. Schmid, and J. Ponce, "3D Object Modeling and Recognition from Photographs and Image Sequences," Toward Category-Level Object Recognition, pp. 105-126, Springer, 2006.
[39] K. Sato, K. Ikeuchi, and T. Kanade, "Model-Based Recognition of Specular Objects Using Sensor Models," Proc. IEEE Workshop Directions in Automated CAD-Based Vision, pp. 2-10, 1991.
[40] K. Sato, K. Ikeuchi, and T. Kanade, "Model Based Recognition of Specular Objects Using Sensor Models," Proc. Workshop Directions in Automated CAD-Based Vision, pp. 2-10, 1991.
[41] S. Savarese, M. Chen, and P. Perona, "Local Shape from Mirror Reflections," Int'l J. Computer Vision, vol. 64, no. 1, pp. 31-67, 2005.
[42] C. Schmid, R. Mohr, and C. Bauckhage, "Evaluation of Interest Point Detectors," Int'l J. Computer Vision, vol. 37, no. 2, pp. 151-172, 2000.
[43] H. Schultz, "Retrieving Shape Information from Multiple Images of a Specular Surface," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 2, pp. 195-201, Feb. 1994.
[44] S. Shirdhonkar and D.W. Jacobs, "Non-Negative Lighting and Specular Object Recognition," Proc. IEEE Int'l Conf. Computer Vision, pp. 1323-1330, 2005.
[45] D.G. Sim, O.K. Kwon, and R.H. Park, "Object Matching Algorithms Using Robust Hausdorff Distance Measures," IEEE Trans. Image Processing, vol. 8, no. 3, pp. 425-429, Mar. 1999.
[46] T. Suk and J. Flusser, "Graph Method for Generating Affine Moment Invariants," Proc. Int'l Conf. Pattern Recognition, vol. 2, 2004.
[47] R. Swaminathan, S.B. Kang, R. Szeliski, A. Criminisi, and S.K. Nayar, "On the Motion and Appearance of Specularities in Image Sequences," Proc. European Conf. Computer Vision, vol. 1, pp. 508-523, 2002.
[48] K.K. Thornber and D.W. Jacobs, "Broadened, Specular Reflection and Linear Subspaces," Technical Report 2001-033, NEC, 2001.
[49] J. Wang and K.J. Dana, "A Novel Approach for Texture Shape Recovery," Proc. IEEE Int'l Conf. Computer Vision, p. 1374, 2003.
[50] L.B. Wolff, "Using Polarization to Separate Reflection Components," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 363-369, 1989.
122 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool