The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2009 vol.31)
pp: 1210-1224
Jianke Zhu , The Chinese University of Hong Kong, Hong Kong
Michael R. Lyu , The Chinese University of Hong Kong, Hong Kong
Thomas S. Huang , University of Illinois at Urbana-Champaign, Urbana
ABSTRACT
In this paper, we present a fusion approach to solve the nonrigid shape recovery problem, which takes advantage of both the appearance information and the local features. We have two major contributions. First, we propose a novel progressive finite Newton optimization scheme for the feature-based nonrigid surface detection problem, which is reduced to only solving a set of linear equations. The key is to formulate the nonrigid surface detection as an unconstrained quadratic optimization problem that has a closed-form solution for a given set of observations. Second, we propose a deformable Lucas-Kanade algorithm that triangulates the template image into small patches and constrains the deformation through the second-order derivatives of the mesh vertices. We formulate it into a sparse regularized least squares problem, which is able to reduce the computational cost and the memory requirement. The inverse compositional algorithm is applied to efficiently solve the optimization problem. We have conducted extensive experiments for performance evaluation on various environments, whose promising results show that the proposed algorithm is both efficient and effective.
INDEX TERMS
Image processing and computer vision, nonrigid detection, real-time deformable registration, nonrigid augmented reality, medical image registration.
CITATION
Jianke Zhu, Michael R. Lyu, Thomas S. Huang, "A Fast 2D Shape Recovery Approach by Fusing Features and Appearance", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 7, pp. 1210-1224, July 2009, doi:10.1109/TPAMI.2008.151
REFERENCES
[1] L. Brown, “A Survey of Image Registration Techniques,” ACM Computing Surveys, vol. 24, no. 4, pp. 325-376, Dec. 1992.
[2] A. Bartoli and A. Zisserman, “Direct Estimation of Non-Rigid Registration,” Proc. 15th British Machine Vision Conf., Sept. 2004.
[3] L.V. Tsap, D.B. Goldgof, and S. Sarkar, “Nonrigid Motion Analysis Based on Dynamic Refinement of Finite Element Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 5, pp. 526-543, May 2000.
[4] H. Fang and J.C. Hart, “Rototexture: Automated Tools for Texturing Raw Video,” IEEE Trans. Visualization and Computer Graphics, vol. 12, no. 6, pp. 1580-1589, Nov./Dec. 2006.
[5] H. Fang and J.C. Hart, “Textureshop: Texture Synthesis as a Photograph Editing Tool,” ACM Trans. Graphics, vol. 23, no. 3, pp.354-359, 2004.
[6] R. White and D. Forsyth, “Retexturing Single Views Using Texture and Shading,” Proc. Ninth European Conf. Computer Vision, pp. 70-81, 2006.
[7] R. White and D.A. Forsyth, “Combining Cues: Shape from Shading and Texture,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1809-1816, 2006.
[8] J. Pilet, V. Lepetit, and P. Fua, “Real-Time Non-Rigid Surface Detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 822-828, 2005.
[9] M.J. Black and Y. Yacoob, “Tracking and Recognizing Rigid and Non-Rigid Facial Motions Using Local Parametric Models of Image Motion,” Proc. Fifth Int'l Conf. Computer Vision, pp. 374-381, 1995.
[10] M. Brand and R. Bhotika, “Flexible Flow for 3D Nonrigid Tracking and Shape Recovery,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 315-322, 2001.
[11] L. Torresani, D. Yang, E. Alexander, and C. Bregler, “Tracking and Modeling Non-Rigid Objects with Rank Constraints,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 493-500, 2001.
[12] T. Cootes, G. Edwards, and C. Taylo, “Active Appearance Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 981-685, June 2001.
[13] J. Zhu, S.C. Hoi, and M.R. Lyu, “Real-Time Non-Rigid Shape Recovery via Active Appearance Models for Augmented Reality,” Proc. Ninth European Conf. Computer Vision, pp. 186-197, 2006.
[14] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, Sept. 2003.
[15] D. DeCarlo and D.N. Metaxas, “Optical Flow Constraints on Deformable Models with Applications to Face Tracking,” Int'l J. Computer Vision, vol. 38, no. 2, pp. 99-127, 2000.
[16] J. Pilet, V. Lepetit, and P. Fua, “Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation,” Int'l J. Computer Vision, vol. 76, no. 2, pp. 109-122, 2008.
[17] M.A. Fischler and R.C. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Comm. ACM, vol. 24, no. 6, pp. 381-395, 1981.
[18] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[19] S. Belongie, J. Malik, and J. Puzicha, “Shape Matching and Object Recognition Using Shape Contexts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002.
[20] H. Chui and A. Rangarajan, “A New Point Matching Algorithm for Non-Rigid Registration,” Computer Vision and Image Understanding, vol. 89, nos. 2-3, pp. 114-141, 2003.
[21] G. McNeill and S. Vijayakumar, “Part-Based Probabilistic Point Matching Using Equivalence Constraints,” Advances in Neural Information Processing Systems, vol. 19, pp. 969-976, 2007.
[22] M. Salzmann, J. Pilet, S. Ilic, and P. Fua, “Surface Deformation Models for Non-Rigid 3-D Shape Recovery,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 8, pp. 1481-1487, Aug. 2007.
[23] S. Sclaroff and J. Isidoro, “Active Blobs: Region-Based, Deformable Appearance Models,” Computer Vision and Image Understanding, vol. 89, nos. 2-3, pp. 197-225, 2003.
[24] O. Chum and J. Matas, “Matching with PROSAC—Progressive Sample Consensus,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 220-226, 2005.
[25] S. Baker and I. Matthews, “Lucas-Kanade 20 Years On: A Unifying Framework,” Int'l J. Computer Vision, vol. 56, no. 3, pp. 221-255, Mar. 2004.
[26] W.-C. Lin and Y. Liu, “Tracking Dynamic Near-Regular Texture under Occlusion and Rapid Movements,” Proc. Ninth European Conf. Computer Vision, pp. 44-55, 2006.
[27] I. Matthews and S. Baker, “Active Appearance Models Revisited,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 135-164, 2004.
[28] V. Gay-Bellile, A. Bartoli, and P. Sayd, “Feature-Driven Direct Non-Rigid Image Registration,” Proc. 18th British Machine Vision Conf., 2007.
[29] S. Granger and X. Pennec, “Multi-Scale EM-ICP: A Fast and Robust Approach for Surface Registration,” Proc. Seventh European Conf. Computer Vision, pp. 418-432, 2002.
[30] D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[31] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005.
[32] H. Ling and D.W. Jacobs, “Deformation Invariant Image Matching,” Proc. 10th IEEE Int'l Conf. Computer Vision, pp. 1466-1473, 2005.
[33] V. Lepetit and P. Fua, “Keypoint Recognition Using Randomized Trees,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1465-1479, Sept. 2006.
[34] B.D. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proc. Seventh Int'l Joint Conf. Artificial Intelligence, pp. 674-679, 1981.
[35] A. Bartoli, “Groupwise Geometric and Photometric Direct Image Registration,” Proc. 17th British Machine Vision Conf., Sept. 2006.
[36] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active Contour Models,” Int'l J. Computer Vision, vol. 1, no. 4, pp. 321-331, Jan. 1988.
[37] P. Fua and Y. Leclerc, “Object-Centered Surface Reconstruction: Combining Multi-Image Stereo and Shading,” Int'l J. Computer Vision, vol. 16, no. 1, pp. 35-56, Sept. 1995.
[38] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
[39] O.L. Mangasarian, “A Finite Newton Method for Classification,” Optimization Methods and Software, vol. 17, no. 5, pp. 913-929, 2002.
[40] S.S. Keerthi and D. DeCoste, “A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs,” J. Machine Learning Research, vol. 6, pp. 341-361, 2005.
[41] S. Ilic, M. Salzmann, and P. Fua, “Implicit Meshes for Effective Silhouette Handling,” Int'l J. Computer Vision, vol. 72, no. 2, pp.159-178, 2007.
[42] A. Bartoli, “Groupwise Geometric and Photometric Direct Image Registration,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 12, pp. 2098-2108, Dec. 2008.
[43] S. Periaswamy and H. Farid, “Medical Image Registration with Partial Data,” Medical Image Analysis, no. 10, pp. 452-464, 2006.
[44] J. Zhu and M.R. Lyu, “Progressive Finite Newton Approach to Real-Time Nonrigid Surface Detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[45] S. Benhimane and E. Malis, “Real-Time Image-Based Tracking of Planes Using Efficient Second-Order Minimization,” Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, pp. 943-948, 2004.
24 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool