This Article 
 Bibliographic References 
 Add to: 
Nonrigid Motion Analysis Based on Dynamic Refinement of Finite Element Models
May 2000 (vol. 22 no. 5)
pp. 526-543

Abstract—In this paper, we propose new algorithms for accurate nonrigid motion tracking. Given an initial model representing general knowledge of the object, a set of sparse correspondences, and incomplete or missing information about geometry or material properties, we can recover dense motion vectors using finite element models. The method is based on the iterative analysis of the differences between the actual and predicted behavior. Large differences indicate that an object's properties are not captured properly by the model describing it. Feedback from the images during the motion allows the refinement of the model by minimizing the error between the expected and true position of the object's points. These errors are due to flaws in the model parameter estimation such as geometry and material properties. Unknown parameters are recovered using an iterative descent search for the best nonlinear finite element model that approximates nonrigid motion of the given object. During this search process, we not only estimate material properties, but also infer dense point correspondences from our initial set of sparse correspondences. Thus, during tracking, the model is refined which, in turn, improves tracking quality. As a result, we obtain a more precise description of nonrigid motion. Experimental results demonstrate the success of the proposed algorithm. The method was applied to man-made elastic materials and human skin to recover unknown elasticity, to complex 3D objects to find details of their geometry, and to a hand motion analysis application. Our work demonstrates the possibility of accurate quantitative analysis of nonrigid motion in range image sequences with objects consisting of multiple materials and 3D volumes.

[1] J.R. Brauer, What Every Engineer Should Know About Finite Element Analysis. New York: Marcel Dekker, 1993.
[2] L.V. Tsap, D.B. Goldgof, S. Sarkar, and W.-C. Huang, “Efficient Nonlinear FEM Modeling of Nonrigid Objects via Optimization of Mesh Models,” Computer Vision and Image Understanding, vol. 69, no. 3, pp. 330-350, Mar. 1998.
[3] D. Metaxas and D. Terzopoulos, "Constrained Deformable Superquadrics and Nonrigid Motion Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 337-343, 1991.
[4] D. DeCarlo and D. Metaxas, "Adaptive Shape Evolution Using Blending," IEEE Proc. Int'l Conf. Computer Vision, pp. 834-839, 1995.
[5] W. Neuenschwander, P. Fua, G. Szekely, and O. Kubler, “Deformable Velcro Surfaces,” Proc. Fifth Int'l Conf. Computer Vision, pp. 828-833, June 1995.
[6] C. Nastar and N. Ayache, “Frequency-Based Nonrigid Motion Analysis: Application to Four Dimensional Medical Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 11, pp. 1,067-1,079, Nov. 1996.
[7] Y. Kita, “Elastic-Model Driven Analysis of Several Views of a Deformable Cylindrical Object,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 12, pp. 1,150-1,162, Dec. 1996.
[8] S. Sclaroff and A.P. Pentland, “Physically-Based Combinations of Views: Representing Rigid and Nonrigid Motion,” Proc. IEEE Workshop on Motion of Non-Rigid and Articulated Objects, pp. 158-164, Nov. 1994.
[9] A. Young and L. Axel, "Non-Rigid Heart Wall Motion Using MR Tagging," Proc. Computer Vision and Pattern Recognition '92, pp. 399-404, 1992.
[10] D. Metaxas and E. Koh, “Efficient Shape Representation Using Deformable Models with Locally Adaptive Finite Elements,” Proc. SPIE Geometric Methods in Computer Vision II, vol. 2,031, pp. 160-171, July 1993.
[11] M.H. Davis, A. Khotanzad, D.P. Flamig, and S.E. Harms, “A Physics-Based Coordinate Transformation for 3-D Image Matching,” IEEE Trans. Medical Imaging, vol. 16, no. 3, pp. 317-328, June 1997.
[12] S. Basu and A. Pentland, "A Three-Dimensional Model of Human Lip Motions Trained from Video," Proc. IEEE Non-Rigid and Articulated Motion Workshop at CVPR 97, IEEE Press, Piscataway, N.J., 1997, pp. 46-53.
[13] W.-C. Huang and D.B. Goldgof, “Point Correspondence Recovery in Nonrigid Motion Using Nonlinear Finite Element Modeling,” Proc. Asian Conf. Computer Vision, pp. 256-259, Nov. 1993.
[14] Y.C. Fung, Biomechanics: Mechanical Properties of Living Tissues. second ed., New York: Springer-Verlag, 1993.
[15] W.F. Larrabee, “A Finite Element Model of Skin Deformation. I. Biomechanics of Skin and Soft Tissue: A Review,” Laryngoscope, vol. 96, pp. 399-405, Apr. 1986.
[16] J.A. Weiss, J.A. Painter, and E.P. France, “Finite Element Mesh Generation of the Knee and Its Soft Tissues from CT Data,” Proc. Second World Congress Biomechanics, p. 149, July 1995.
[17] D. Metaxas and D. Terzopoulos, “Shape and Non-Rigid Motion Estimation through Physics-Based Synthesis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 580-591, June 1993.
[18] J.M. Rehg and T. Kanade, Visual Tracking of High DOF Articulated Structures: An Application to Human Hand Tracking Proc. European Conf. Computer Vision, vol. 2, pp. 35-46, 1994.
[19] J.M. Rehg and T. Kanade, “Model-Based Tracking of Self-Occluding Articulated Objects,” Proc. Fifth Int'l Conf. Computer Vision, pp. 612–617, June 1995.
[20] J.J. Kuch and T.S. Huang, "Vision-Based Hand Modeling and Tracking," Proc. IEEE Int'l Conf. Computer Vision,Cambridge, Mass., June 1995.
[21] H. Delingette, “Adaptive and Deformable Models Based on Simplex Meshes,” Proc. IEEE Workshop Motion of Non-Rigid and Articulated Objects, pp. 152-157, Nov. 1994.
[22] J.K. Aggarwal and Q. Cai, “Human Motion Analysis: A Review,” Proc. IEEE Nonrigid and Articulated Motion Workshop, pp. 90–102, June 1997.
[23] W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C.Cambridge, England: Cambridge Univ. Press, 1988.
[24] D.S. Burnett, Finite Element Analysis. Addison-Wesley, 1988.
[25] L.V. Tsap, D.B. Goldgof, S. Sarkar, and P. Powers, “A Vision-Based Technique for Objective Assessment of Burn Scars,” IEEE Trans. Medical Imaging, vol. 17, no. 4, pp. 620-633, Aug. 1998.
[26] T.J.R. Hughes, “Numerical Implementation of Constitutive Models,” Theoretical Foundation for Large-Scale Computations for Nonlinear Material Behavior. Dordrecht, The Netherlands: Martinus Nijhoff, 1984.
[27] ANSYS User's Manual for Revision 5.3, Swanson Analysis System, Inc., Houston, Pa., 1996.
[28] B.A. Todd and J.G. Thacker, “Three-Dimensional Computer Model of the Human Buttocks, in Vivo,” J. Rehabilitation Research, vol. 31, no. 2, pp. 111-119, 1994.
[29] M.B. Silver-Thorn, J.W. Steege, and D.S. Childress, “A Review of Prosthetic Interface Stress Investigations,” J. Rehabilitation Research and Development, vol. 33, no. 3, pp. 253-266, 1996.
[30] W.M. Vannah and D.S. Childress, “Modeling the Mechanics of Narrowly Contained Soft Tissues: The Effects of Specification of Poisson's Ratio,” J. Rehabilitation Research and Development, vol. 30, no. 2, pp. 205-209, 1993.
[31] M. Zhang, M. Lord, A.R. Turner-Smith, and V.C. Roberts, “Development of a Nonlinear Finite Element Modeling of the Below-Knee Prosthetic Socket Interface,” Medical Eng. and Physics, vol. 17, no. 8, pp. 559-566, 1995.
[32] T. Maeno, K. Kobayashi, and N. Yamazaki, “Effect of Geometry and Property of the Finger Tissue on Stress/Strain Distribution Near the Tactile Receptors,” Proc. 10th Conf. European Soc. Biomechanics, p. 312, 1996.
[33] C.W.J. Oomens, “A Mixture Approach to the Mechanics of Skin and Subcutis,” PhD thesis, Technische Hogeschool Twente, The Netherlands, 1985.
[34] J.G. Thacker, “The Elastic Properties of Human Skin in Vivo,” PhD thesis, Univ. of Virginia, 1976.
[35] R.M. Alexander, The Human Machine. New York: Columbia Univ. Press, 1992.
[36] G.B. Bridgeman, Bridgeman's Complete Guide to Drawing from Life. New York: Weathervane Books, 1979.
[37] L.V. Tsap, D.B. Goldgof, and S. Sarkar, “Model-Based Force-Driven Nonrigid Motion Recovery from Sequences of Range Images without Point Correspondences,” Image and Vision Computing J., vol. 17, no. 14, pp. 997-1,008, Nov. 1999.
[38] The National Institute for Occupational Safety and Health (NIOSH), NIOSH Home Page 1996.

Index Terms:
Physically-based vision, deformable models, nonrigid motion analysis, biomedical applications, finite element analysis.
Leonid V. Tsap, Dmitry B. Goldgof, Sudeep Sarkar, "Nonrigid Motion Analysis Based on Dynamic Refinement of Finite Element Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 5, pp. 526-543, May 2000, doi:10.1109/34.857007
Usage of this product signifies your acceptance of the Terms of Use.