The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - January (2010 vol.32)
pp: 87-104
Adrien Bartoli , LASMEA, UMR, CNRS/UBP, France
Vincent Gay-Bellile , CEA Saclay and LASMEA, UMR, CNRS/UBP, France
ABSTRACT
The registration problem for images of a deforming surface has been well studied. External occlusions are usually well handled. In 2D image-based registration, self-occlusions are more challenging. Consequently, the surface is usually assumed to be only slightly self-occluding. This paper is about image-based nonrigid registration with self-occlusion reasoning. A specific framework explicitly modeling self-occlusions is proposed. It is combined with an intensity-based, “direct” data term for registration. Self-occlusions are detected as shrinkage areas in the 2D warp. Experimental results on several challenging data sets show that our approach successfully registers images with self-occlusions while effectively detecting the self-occluded regions.
INDEX TERMS
Nonrigid registration, self-occlusion, direct method, image retexturing.
CITATION
Adrien Bartoli, Vincent Gay-Bellile, "Direct Estimation of Nonrigid Registrations with Image-Based Self-Occlusion Reasoning", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 1, pp. 87-104, January 2010, doi:10.1109/TPAMI.2008.265
REFERENCES
[1] http://www.cise.ufl.edu/research/sparsecholmod , 2009.
[2] S. Baker, R. Gross, I. Matthews, and T. Ishikawa, “Lucas-Kanade 20 Years On: A Unifying Framework: Part 2,” Technical Report CMU-RI-TR-03-01, Robotics Inst., Carnegie Mellon Univ., 2003.
[3] S. Baker and I. Matthews, “Lucas-Kanade 20 Years On: A Unifying Framework,” Int'l J. Computer Vision, vol. 56, pp. 221-255, 2004.
[4] A. Bartoli, “Groupwise Geometric and Photometric Direct Image Registration,” Proc. British Machine Vision Conf., 2006.
[5] A. Bartoli, “Maximizing the Predictivity of Smooth Deformable Image Warps through Cross-Validation,” Math. Imaging and Vision, vol. 31, pp. 133-145, 2008.
[6] A. Bartoli and A. Zisserman, “Direct Estimation of Non-Rigid Registrations,” Proc. British Machine Vision Conf., 2004.
[7] S. Benhimane and E. Malis, “Real-Time Image-Based Tracking of Planes Using Efficient Second-Order Minimization,” Proc. Int'l Conf. Intelligent Robots and Systems, 2004.
[8] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hierarchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, 1992.
[9] M.J. Black and P. Anandan, “Robust Dynamic Motion Estimation over Time,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 1991.
[10] F.L. Bookstein, “Principal Warps: Thin-Plate Splines and the Decomposition of Deformations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 6, pp. 567-585, June 1989.
[11] C. Bregler, A. Hertzmann, and H. Biermann, “Recovering Non-Rigid 3D Shape from Image Streams,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 2000.
[12] P.J. Burt and E.H. Adelson, “The Laplacian Pyramid as a Compact Image Code,” IEEE Trans. Comm., vol. 31, no. 4, pp. 532-540, Apr. 1983.
[13] T.F. Cootes, S. Marsland, C.J. Twining, K. Smith, and C.J. Taylor, “Groupwise Diffeomorphic Non-Rigid Registration for Automatic Model Building,” Proc. European Conf. Computer Vision, 2004.
[14] A.W. Fitzgibbon, “Robust Registration of 2D and 3D Point Sets,” Proc. British Machine Vision Conf., 2001.
[15] V. Gay-Bellile, A. Bartoli, and P. Sayd, “Feature-Driven Non-Rigid Image Registration,” Proc. British Machine Vision Conf., 2007.
[16] G.D. Hager and P.N. Belhumeur, “Efficient Region Tracking with Parametric Models of Geometry and Illumination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 10, pp. 1025-1039, Oct. 1998.
[17] K.P. Horn and G. Schunck, “Determining Optical Flow,” Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[18] X. Huang, N. Paragios, and D. Metaxas, “Shape Registration in Implicit Spaces Using Information Theory and Free Form Deformations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1303-1318, Aug. 2006.
[19] S. Ilic, M. Salzmann, and P. Fua, “Implicit Meshes for Effective Silhouette Handling,” Int'l J. Computer Vision, vol. 72, pp. 159-178, 2007.
[20] V. Lepetit and P. Fua, “Monocular Model-Based 3D Tracking of Rigid Objects: A Survey,” Foundations and Trends in Computer Graphics and Vision, vol. 1, pp. 1-89, 2005.
[21] J. Lim and M.-H. Yang, “A Direct Method for Non-Rigid Motion with Thin-Plate Spline,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[22] W.C. Lin and Y. Liu, “Tracking Dynamic Near-Regular Textures under Occlusions and Rapid Movements,” Proc. European Conf. Computer Vision, 2006.
[23] E. Malis, “An Efficient Unified Approach to Direct Visual Tracking of Rigid and Deformable Surfaces,” Proc. Int'l Conf. Intelligent Robots and Systems, 2007.
[24] J. Pilet, V. Lepetit, and P. Fua, “Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation,” Int'l J. Computer Vision, vol. 76, pp. 109-122, 2008.
[25] D. Pizarro and A. Bartoli, “Shadow Resistant Direct Image Registration,” Proc. Scandinavian Conf. Image Analysis, 2007.
[26] D. Pizarro, J. Peyras, and A. Bartoli, “Light-Invariant Fitting of Active Appearance Models,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[27] M. Prasad, A. Zisserman, and A. Fitzgibbon, “Single View Reconstruction of Curved Surfaces,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[28] D. Rueckert, L. Sonoda, C. Hayes, D. Hill, M. Leach, and D. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Medical Imaging, vol. 18, no. 8, pp. 712-721, Aug. 1999.
[29] M. Salzmann, S. Ilic, and P. Fua, “Physically Valid Shape Parameterization for Monocular 3-D Deformable Surface Tracking,” Proc. British Machine Vision Conf., 2005.
[30] G. Silveira and E. Malis, “Real-Time Visual Tracking under Arbitrary Illumination Changes,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 2007.
[31] R. White and D. Forsyth, “Retexturing Single Views Using Texture and Shading,” Proc. European Conf. Computer Vision, 2006.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool