The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.06 - June (2011 vol.33)
pp: 1098-1115
Samuel Dambreville , Georgia Institute of Technology, Atlanta and the Boston Consulting Group
Anthony Yezzi , Georgia Institute of Technology, Atlanta
Romeil Sandhu , Georgia Institute of Technology, Atlanta
ABSTRACT
In this work, we present a nonrigid approach to jointly solving the tasks of 2D-3D pose estimation and 2D image segmentation. In general, most frameworks that couple both pose estimation and segmentation assume that one has exact knowledge of the 3D object. However, under nonideal conditions, this assumption may be violated if only a general class to which a given shape belongs is given (e.g., cars, boats, or planes). Thus, we propose to solve the 2D-3D pose estimation and 2D image segmentation via nonlinear manifold learning of 3D embedded shapes for a general class of objects or deformations for which one may not be able to associate a skeleton model. Thus, the novelty of our method is threefold: First, we present and derive a gradient flow for the task of nonrigid pose estimation and segmentation. Second, due to the possible nonlinear structures of one's training set, we evolve the preimage obtained through kernel PCA for the task of shape analysis. Third, we show that the derivation for shape weights is general. This allows us to use various kernels, as well as other statistical learning methodologies, with only minimal changes needing to be made to the overall shape evolution scheme. In contrast with other techniques, we approach the nonrigid problem, which is an infinite-dimensional task, with a finite-dimensional optimization scheme. More importantly, we do not explicitly need to know the interaction between various shapes such as that needed for skeleton models as this is done implicitly through shape learning. We provide experimental results on several challenging pose estimation and segmentation scenarios.
INDEX TERMS
3D pose estimation, image segmentation, statistical learning, kernel PCA.
CITATION
Samuel Dambreville, Anthony Yezzi, Romeil Sandhu, "A Nonrigid Kernel-Based Framework for 2D-3D Pose Estimation and 2D Image Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 6, pp. 1098-1115, June 2011, doi:10.1109/TPAMI.2010.162
REFERENCES
[1] A. Balan, L. Sigal, M. Black, J. Davis, and H. Haussecker, “Detailed Human Shape and Pose from Images,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[2] P.J. Besl and N.D. McKay, “A Method for Registration of 3D Shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[3] Active Contours. A. Blake and M. Isard, eds. Springer, 1998.
[4] V. Blanz and T. Vetter, “A Morphable Model for the Synthesis of 3D Faces,” Proc. ACM SIGGRAPH, pp. 187-194, 1999.
[5] R. Bowden, T. Mitchell, and M. Sarhadi, “Reconstructing 3D Pose and Motion from a Single Camera View,” Proc. British Machine Vision Conf., vol. 2, pp. 904-913, 1998.
[6] M. Bray, P. Kohli, and P. Torr, “Posecut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts,” Proc. European Conf. Computer Vision, pp. 642-655, 2006.
[7] V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic Active Contours,” Proc. Int'l J. Computer Vision, vol. 22, pp. 61-79, 1997.
[8] T. Chan and L. Vese, “Active Contours without Edges,” IEEE Trans. Image Processing, vol. 10, no. 2, pp. 266-277, Feb. 2001.
[9] G. Charpiat, O. Faugeras, and R. Keriven, “Approximations of Shape Metrics and Application to Shape Warping and Empirical Shape Statistics,” Foundations of Computational Math., vol. 5, no. 1, pp. 1-58, 2005.
[10] G. Charpiat, O. Faugeras, and R. Keriven, “Shape Statistics for Image Segmentation with Prior,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-6, 2007.
[11] G. Charpiat, P. Maurel, J. Pons, R. Keriven, and O. Faugeras, “Generalized Gradients: Priors on Minimization Flows,” Int'l J. Computer Vision, vol. 73, no. 3, pp. 325-344, 2007.
[12] D. Cremers, T. Kohlberger, and C. Schnoerr, “Shape Statistics in Kernel Space for Variational Image Segmentation,” Pattern Recognition, vol. 36, no. 9, pp. 1929-1943, 2003.
[13] S. Dambreville, “Statistical and Geometric Methods for Shape-Driven Segmentation and Tracking,” doctoral dissertation, Georgia Inst. of Tech nology, 2008.
[14] S. Dambreville, Y. Rathi, and A. Tannenbaum, “A Framework for Image Segmentation Using Shape Models and Kernel Space Shape Priors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 8, pp. 1385-1399, Aug. 2008.
[15] S. Dambreville, R. Sandhu, A. Yezzi, and A. Tannenbaum, “Robust 3D Pose Estimation and Efficient 2D Region-Based Segmentation from a 3D Shape Prior,” Proc. European Conf. Computer Vision, 2008.
[16] E. Debreuve, M. Gastaud, M. Barlaud, and G. Aubert, “Using the Shape Gradient for Active Contour Segmentation: From the Continuous to the Discrete Formulation,” J. Math. Imaging and Vision, vol. 28, no. 1, pp. 47-66, 2007.
[17] J. Deutscher and I. Reid, “Articulated Body Motion Capture by Stochastic Search,” Int'l J. Computer Vision, vol. 61, no. 2, pp. 185-205, 2004.
[18] M. Dhome, M. Richetin, and J.T. Lapreste, “Determination of the Attitude of 3D Objects from a Single Perspective View,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 12, pp. 1265-1278, Dec. 1989.
[19] T. Drummond and R. Cipolla, “Real-Time Tracking of Multiple Articulated Structures in Multiple Views,” Proc. European Conf. Computer Vision, 2000.
[20] R.C. Gonzalez and R.E. Woods, Digital Image Processing. Addison-Wesley Longman, 2001.
[21] F. Han and S. Zhu, “Bayesian Reconstruction of 3D Shapes and Scenes from a Single Image,” Proc. Int'l Workshop High Level Knowledge in 3D Modeling and Motion, vol. 2, 2003.
[22] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[23] S. Kichenassamy, S. Kumar, P. Olver, A. Tannenbaum, and A. Yezzi, “Conformal Curvature Flow: From Phase Transitions to Active Vision,” Archives for Rational Mechanics and Analysis, vol. 134, pp. 275-301, 1996.
[24] P. Kohli, J. Rihan, M. Bray, and P. Torr, “Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph Cuts,” Int'l J. Computer Vision, vol. 79, no. 3, pp. 285-298, 2008.
[25] J. Kwok and I. Tsang, “The Pre-Image Problem in Kernel Methods,” IEEE Trans. Neural Networks, vol. 15, no. 6, pp. 1517-1525, Nov. 2004.
[26] M. Leventon, E. Grimson, and O. Faugeras, “Statistical Shape Influence in Geodesic Active Contours,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1316-1324, 2000.
[27] Y. Ma, S. Soatto, J. Kosecka, and S. Sastry, An Invitation to 3D Vision. Springer, 2001.
[28] E. Marchand, P. Bouthemy, and F. Chaumette, “A 2D-3D Model-Based Approach to Real-Time Visual Tracking,” Image and Vision Computing, vol. 19, no. 13, pp. 941-955, Nov. 2001.
[29] J. Mercer, “Functions of Positive and Negative Type and Their Connection with the Theory of Integral Equations,” Philosophical Trans. Royal Soc. London, vol. 209, pp. 415-446, 1909.
[30] O. Michailovich, Y. Rathi, and A. Tannenbaum, “Image Segmentation Using Active Contours Driven by the Bhattacharyya Gradient Flow,” IEEE Trans. Image Processing, vol. 16, pp. 2787-2801, Nov. 2007.
[31] S. Mika, B. Scholkopf, A.J. Smola, K.R. Muller, M. Scholz, and G. Ratsch, “Kernel PCA and De-Noising in Feature Spaces,” Advances in Neural Information Processing Systems, vol. 11, MIT Press, 1999.
[32] N. Paragios and R. Deriche, “Geodesic Active Regions: A New Paradigm to Deal with Frame Partition Problems in Computer Vision,” J. Visual Comm. and Image Representation, vol. 13, pp. 249-268, 2002.
[33] L. Quan and Z.D. Lan, “Linear N-Point Camera Pose Determination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 774-780, Aug. 1999.
[34] Y. Rathi, S. Dambreville, and A. Tannenbaum, “Statistical Shape Analysis Using Kernel PCA,” Proc. SPIE, pp. 425-432, 2006.
[35] Y. Rathi, N. Vaswani, A. Tannenbaum, and T. Yezzi, “Particle Filtering for Geometric Active Contours with Application to Tracking Moving and Deforming Objects,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[36] T. Riklin-Raviv, N. Kiryati, and N. Sochen, “Prior-Based Segmentation by Projective Registration and Level Sets,” Proc. Int'l Conf. Computer Vision, pp. 204-211, 2005.
[37] B. Rosenhahn, T. Brox, and J. Weickert, “Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking,” Int'l J. Computer Vision, vol. 73, no. 3, pp. 243-262, 2007.
[38] B. Rosenhahn, U. Kersting, K. Powel, and H.P. Seidel, “Cloth X-Ray: Mocop of People Wearing Textiles,” Proc. Ann. Symp. German Assoc. for Pattern Recognition, pp. 495-504, 2006.
[39] B. Rosenhahn, C. Perwass, and G. Sommer, “Pose Estimation of Free-Form Contours,” Int'l J. Computer Vision, vol. 62, no. 3, pp. 267-289, 2005.
[40] D. Rother and G. Sapiro, “Seeing 3D Objects in a Single 2D Image,” Proc. Int'l Conf. Computer Vision, 2009.
[41] R. Sandhu, S. Dambreville, and A. Tannenbaum, “Particle Filtering for Registration of 2D and 3D Point Sets with Stochastic Dynamics,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[42] R. Sandhu, S. Dambreville, A. Yezzi, and A. Tannenbaum, “Nonrigid 2D-3D Pose Estimation and 2D Image Segmentation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[43] C. Schmaltz, B. Rosenhahn, T. Brox, D. Cremers, J. Weickert, L. Wietzke, and G. Sommer, “Region-Based Pose Tracking,” Pattern Recognition and Image Analysis, pp. 56-63, 2007.
[44] D. Snow, P. Viola, and R. Zabih, “Exact Voxel Occupancy with Graph Cuts,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 345-352, 2000.
[45] M. Sofka, G. Yang, and C. Stewart, “Simultaneous Covariance Driven Correspondence (CDC) and Transformation Estimation in the Expectation Maximization Framework,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[46] M. Taron, N. Paragios, and M. Jolly, “Registration with Uncertainties and Statistical Modeling of Shapes with Variable Metric Kernels,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 99-113, Jan. 2009.
[47] A. Tsai, T. Yezzi, W. Wells, C. Tempany, D. Tucker, A. Fan, E. Grimson, and A. Willsky, “A Shape-Based Approach to the Segmentation of Medical Imagery Using Level Sets,” IEEE Trans. Image Processing, vol. 22, no. 2, pp. 137-153, Feb. 2003.
[48] G. Unal, A. Yezzi, S. Soatto, and G. Slabaugh, “A Variational Approach to Problems in Calibration of Multiple Cameras,” IEEE Trans. Pattern Analysis and Machine Intelligence., vol. 29, no. 8, pp. 1322-1338, Aug. 2007.
[49] A. Yezzi and S. Soatto, “Stereoscopic Segmentation,” Int'l J. Computer Vision, vol. 53, no. 3, pp. 31-43, 2003.
[50] A. Yezzi and S. Soatto, “Structure from Motion for Scenes without Features,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 171-178, Jun 2003.
6 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool