The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - Nov. (2013 vol.35)
pp: 2592-2607
G. Carneiro , Australian Centre for Visual Technol., Univ. of Adelaide, Adelaide, SA, Australia
J. C. Nascimento , Inst. de Sist. e Robot., Inst. Super. Tecnico, Lisbon, Portugal
ABSTRACT
We present a new statistical pattern recognition approach for the problem of left ventricle endocardium tracking in ultrasound data. The problem is formulated as a sequential importance resampling algorithm such that the expected segmentation of the current time step is estimated based on the appearance, shape, and motion models that take into account all previous and current images and previous segmentation contours produced by the method. The new appearance and shape models decouple the affine and nonrigid segmentations of the left ventricle to reduce the running time complexity. The proposed motion model combines the systole and diastole motion patterns and an observation distribution built by a deep neural network. The functionality of our approach is evaluated using a dataset of diseased cases containing 16 sequences and another dataset of normal cases comprised of four sequences, where both sets present long axis views of the left ventricle. Using a training set comprised of diseased and healthy cases, we show that our approach produces more accurate results than current state-of-the-art endocardium tracking methods in two test sequences from healthy subjects. Using three test sequences containing different types of cardiopathies, we show that our method correlates well with interuser statistics produced by four cardiologists.
INDEX TERMS
Shape, Motion segmentation, Image segmentation, Training, Computational modeling, Tracking, Imaging,discriminative classifiers, Left ventricle segmentation, deep belief networks, particle filters, dynamical model
CITATION
G. Carneiro, J. C. Nascimento, "Combining Multiple Dynamic Models and Deep Learning Architectures for Tracking the Left Ventricle Endocardium in Ultrasound Data", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 11, pp. 2592-2607, Nov. 2013, doi:10.1109/TPAMI.2013.96
REFERENCES
[1] J.A. Noble and D. Boukerroui, "Ultrasound Image Segmentation: A Survey," IEEE Trans. Medical Imaging, vol. 25, no. 8, pp. 987-1010, Aug. 2006.
[2] J. Bosch, S. Mitchell, B. Lelieveldt, F. Nijland, O. Kamp, M. Sonka, and J. Reiber, "Automatic Segmentation of Echocardiographic Sequences by Active Appearance Motion Models," IEEE Trans. Medical Imaging, vol. 21, no. 11, pp. 1374-1383, Nov. 2002.
[3] E. Bardinet, L. Cohen, and N. Ayache, "Tracking and Motion Analysis of the Left Ventricle with Deformable Superquadratics," Medical Image Analysis, vol. 2, no. 1, pp. 129-149, 1996.
[4] D. Geiger, A. Gupta, L.A. Costa, and J. Vlontzos, "Dynamic Programming for Detecting, Tracking, and Matching Deformable Contours," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 17, pp. 294-302, Mar. 1995.
[5] T. McInerney and D. Terzopoulos, "A Dynamic Finite Element Surface Model for Segmentation and Tracking in Multidimensional Medical Images with Application to 4-D Image Analysis," Computerized Medical Imaging and Graphics, vol. 1, no. 19, pp. 69-83, 1995.
[6] J. Montagnat and H. Delingette, "4D Deformable Models with Temporal Constraints: Application to 4D Cardiac Image Segmentation," Medical Image Analysis, vol. 9, pp. 87-100, 2005.
[7] D. Comaniciu, X. Zhou, and S. Krishnan, "Robust Real-Time Myocardial Border Tracking for Echocardiography: An Information Fusion Approach," IEEE Trans. Medical Imaging, vol. 23, no. 7, pp. 849-860, July 2004.
[8] M. Lynch, O. Ghita, and P. Whelan, "Segmentation of the Left Ventricle of the Heart in 3-D+T MRI Data Using an Optimized Nonrigid Temporal Model," IEEE Trans. Medical Imaging, vol. 27, no. 2, pp. 195-203, Feb. 2008.
[9] N. Paragios, "4-D Deformable Models with Temporal Constraints: Application to 4-D Cardiac Image Segmentation," Medical Image Analysis, vol. 9, no. 1, pp. 87-100, 2005.
[10] J. Sénégas, T. Netsch, C. Cocosco, G. Lund, and A. Stork, "Segmentation of Medical Images with a Shape and Motion Model: A Bayesian Perspective," Proc. European Conf. Computer Vision Workshops CVAMIA and MMBIA, pp. 157-168, 2004.
[11] D. Terzopoulos and R. Szeliski, Tracking with Kalman Snakes. MIT Press, 1993.
[12] G. Jacob, J.A. Noble, C. Behrenbruch, A.D. Kelion, and A.P. Banning, "A Shape-Space-Based Approach to Tracking Myocardial Borders and Quantifying Regional Left-Ventricular Function Applied in Echocardiography," IEEE Trans. Medical Imaging, vol. 21, no. 3, pp. 226-238, Mar. 2002.
[13] J. Dias and J. Leitão, "Wall Position and Thickness Estimation from Sequences of Echocardiograms Images," IEEE Trans. Medical Imaging, vol. 15, no. 1, pp. 25-38, Feb. 1996.
[14] N. Friedland and D. Adam, "Automatic Ventricular Cavity Boundary Detection from Sequential Ultrasound Images Using Simulated Annealing," IEEE Trans. Medical Imaging, vol. 8, no. 4, pp. 344-353, Dec. 1989.
[15] J. Montagnat, M. Sermesant, H. Delingette, G. Maladain, and N. Ayache, "Anisotropic Filtering for Model-Based Segmentation of 4-D Cylindrical Echocardiographic Images," Pattern Recognition Letters, vol. 24, nos. 4/5, pp. 815-828, 2003.
[16] S.K. Setarehdan and J.J. Soraghan, "Automatic Cardiac LV Boundary Detection and Tracking Using Hybrid Fuzzy Temporal and Fuzzy Multiscale Edge Detection," IEEE Trans. Biomedical Eng., vol. 46, no. 11, pp. 1364-1378, Nov. 1999.
[17] M. Lorenzo-Valdes, G. Sanchez-Ortiz, A. Elkington, R. Mohiaddin, and D. Rueckert, "Segmentation of 4D Cardiac MR Images Using a Probabilistic Atlas and the EM Algorithm," Medical Image Analysis, vol. 8, pp. 255-265, 2004.
[18] W. Sun, M. Cetin, R. Chan, and A. Willsky, "Learning the Dynamics and Time-Recursive Boundary Detection of Deformable Objects," IEEE Trans. Image Processing, vol. 17, no. 11, pp. 2186-2200, Nov. 2008.
[19] G. Carneiro and J.C. Nascimento, "Multiple Dynamic Models for Tracking the Left Ventricle of the Heart from Ultrasound Data Using Particle Filters and Deep Learning Architectures," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[20] Z. Qian, D. Metaxas, and L. Axel, "Boosting and Nonparametric Based Tracking of Tagged MRI Cardiac Boundaries," Medical Image Computing and Computer-Assisted Intervention, vol. 9, pp. 636-644, 2006.
[21] W. Sun, M. Cetin, R. Chan, V. Reddy, G. Holmvang, V. Chandar, and A. Willsky, "Segmenting and Tracking the Left Ventricle by Learning the Dynamics in Cardiac Images," Proc. 19th Int'l Conf. Information Processing in Medical Imaging, vol. 19, pp. 553-565, 2005.
[22] L. Yang, B. Georgescu, Y. Zheng, P. Meer, and D. Comaniciu, "3D Ultrasound Tracking of the Left Ventricle Using One-Step Forward Prediction and Data Fusion of Collaborative Trackers," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[23] Y. Zhu, X. Papademetris, A. Sinusas, and J. Duncan, "Segmentation of the Left Ventricle from Cardiac MR Images Using a Subject-Specific Dynamical Model," IEEE Trans. Medical Imaging, vol. 29, no. 3, pp. 669-687, Mar. 2010.
[24] V. Chalana, D. Linker, D. Haynor, and Y. Kim, "A Multiple Active Contour Model for Cardiac Boundary Detection on Echocardiographic Sequences," IEEE Trans. Medical Imaging, vol. 15, no. 3, pp. 290-298, June 1996.
[25] M. Jolly, H. Xue, L. Grady, and J. Guehring, "Combining Registration and Minimum Surfaces for the Segmentation of the left Ventricle in Cardiac Cine MR Images," Medical Image Computing and Computer-Assisted Intervention, vol. 12, pp. 910-918, 2009.
[26] J.C. Nascimento and J.S. Marques, "Robust Shape Tracking with Multiple Models in Ultrasound Images," IEEE Trans. Imaging Processing, vol. 17, no. 3, pp. 392-406, Mar. 2008.
[27] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active Contour Models," Int'l J. Computer Vision, vol. 4, no. 1, pp. 321-331, 1987.
[28] O. Bernard, B. Touil, A. Gelas, R. Prost, and D. Friboulet, "A RBF-Based Multiphase Level Set Method for Segmentation in Echocardiography Using the Statistics of the Radiofrequency Signal," Proc. IEEE Int'l Conf. Image Processing, 2007.
[29] O. Bernard, D. Friboulet, P. Thevenaz, and M. Unser, "Variational B-Spline Level-Set: A Linear Filtering Approach for Fast Deformable Model Evolution," IEEE Trans. Image Processing, vol. 18, no. 6, pp. 1179-1991, June 2009.
[30] C. Corsi, G. Saracino, A. Sarti, and C. Lamberti, "Left Ventricular Volume Estimation for Real-Time Three-Dimensional Echocardiography," IEEE Trans. Medical Imaging, vol. 21, no. 9, pp. 1202-1208, Sept. 2002.
[31] D. Cremers, S. Osher, and S. Soatto, "Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation," Int'l J. Computer Vision, vol. 69, no. 3, pp. 335-351, 2006.
[32] E. Debreuve, M. Barlaud, G. Aubert, I. Laurette, and J. Darcourt, "Space-Time Segmentation Using Level Set Active Contours Applied to Myocardial Gated SPECT," IEEE Trans. Medical Imaging, vol. 20, no. 7, pp. 643-659, July 2001.
[33] N. Lin, W. Yu, and J. Duncan, "Combinative Multi-Scale Level Set Framework for Echocardiographic Image Segmentation," Medical Image Analysis, vol. 7, no. 4, pp. 529-537, 2003.
[34] R. Malladi, J. Sethian, and B. Vemuri, "Shape Modeling with Front Propagation: A Level Set Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 2 pp. 158-175, Feb. 1995.
[35] N. Paragios and R. Deriche, "Geodesic Active Regions and Level Set Methods for Supervised Texture Segmentation," Int'l J. Computer Vision, vol. 46, no. 3, pp. 223-247, 2002.
[36] N. Paragios, "A Level Set Approach for Shape-Driven Segmentation and Tracking of the Left Ventricle," IEEE Trans. Medical Imaging, vol. 21, no. 9, pp. 773-776, June 2003.
[37] A. Sarti, C. Corsi, E. Mazzini, and C. Lamberti, "Maximum Likelihood Segmentation of Ultrasound Images with Rayleigh Distribution," IEEE Trans. Ultrasonic, Ferroelectrics and Frequency Controller, vol. 52, no. 6, pp. 947-960, June 2005.
[38] T. Chen, J. Babb, P. Kellman, L. Axel, and D. Kim, "Semiautomated Segmentation of Myocardial Contours for Fast Strain Analysis in Cine Displacement-Encoded MRI," IEEE Trans. Medical Imaging, vol. 27, no. 8, pp. 1084-1094, Aug. 2008.
[39] Q. Duan, E.D. Angelini, and A. Laine, "Real Time Segmentation by Active Geometric Functions," Computer Methods and Programs Biomedicine, vol. 98, no. 3, pp. 223-230, 2010.
[40] M. Mignotte, J. Meunier, and J. Tardif, "Endocardial Boundary Estimation and Tracking in Echocardiographic Images Using Deformable Template and Markov Random Fields," Pattern Analysis and Applications, vol. 4, no. 4, pp. 256-271, 2001.
[41] V. Zagrodsky, V. Walimbe, C. Castro-Pareja, J.X. Qin, J.-M. Song, and R. Shekhar, "Registration-Assisted Segmentation of Real-Time 3-D Echocardiographic Data Using Deformable Models," IEEE Trans. Medical Imaging, vol. 24, no. 9, pp. 1089-1099, Sept. 2005.
[42] T. Cootes, C. Taylor, D. Cooper, and J. Graham, "Active Shape Models—Their Training and Application," Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
[43] T. Cootes, C. Beeston, G. Edwards, and C. Taylor, "A Unified Framework for Atlas Matching Using Active Appearance Models," Proc. 16th Int'l Conf. Information Processing in Medical Imaging, pp. 322-333, 1999.
[44] S. Mitchell, B. Lelieveldt, R. van der Geest, H. Bosch, J. Reiber, and M. Sonka, "Multistage Hybrid Active Appearance Model Matching: Segmentation of Left and Right Ventricles in Cardiac MR Images," IEEE Trans. Medical Imaging, vol. 20, no. 5, pp. 415-423, May 2001.
[45] G. Carneiro, B. Georgescu, S. Good, and D. Comaniciu, "Detection and Measurement of Fetal Anatomies from Ultrasound Images Using a Constrained Probabilistic Boosting Tree," IEEE Trans. Medical Imaging, vol. 27, no. 9, pp. 1342-1355, Sept. 2008.
[46] B. Georgescu, X.S. Zhou, D. Comaniciu, and A. Gupta, "Databased-Guided Segmentation of Anatomical Structures with Complex Appearance," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[47] J. Weng, A. Singh, and M. Chiu, "Learning-Based Ventricle Detection from Cardiac MR and CT images," IEEE Trans. Medical Imaging, vol. 16, no. 4, pp. 378-391, Aug. 1997.
[48] Y. Zheng, A. Barbu, B. Georgescu, M. Scheuering, and D. Comaniciu, "Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features," IEEE Trans. Medical Imaging, vol. 27, no. 11, pp. 1668-1681, Nov. 2008.
[49] Y. Freund and R. Schapire, "A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting," J. Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1997.
[50] M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, "A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking," IEEE Trans. Signal Processing, vol. 50, no. 2, pp. 174-188, Feb. 2002.
[51] R. Salakhutdinov and G. Hinton, "Learning a Non-Linear Embedding by Preserving Class Neighbourhood Structure," AI and Statistics, 2007.
[52] G. Carneiro, J.C. Nascimento, and A. Freitas, "Robust Left Ventricle Segmentation from Ultrasound Data Using Deep Neural Networks and Efficient Search Methods," Proc. IEEE Int'l Symp. Biomedical Imaging, from Nano to Macro, pp. 1085-1088, 2010.
[53] K. Okuma, A. Taleghani, N. de Freitas, J. Little, and D. Lowe, "A Boosted Particle Filter: Multitarget Detection and Tracking," Proc. European Conf. Computer Vision, 2004.
[54] Y. Bengio, "Learning Deep Architectures for AI," Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
[55] D. Rumelhart, G. Hinton, and R. Williams, "Learning Representations by Back-Propagating Errors," Nature, vol. 323, pp. 533-536, 1986.
[56] M. Carreira-Perpinan and G. Hinton, "On Contrastive Divergence Learning," Proc. Workshop Artificial Intelligence and Statistics, 2005.
[57] G. Hinton and R. Salakhutdinov, "Reducing the Dimensionality of Data with Neural Networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.
[58] G. Hinton, S. Osindero, and Y. Teh, "A Fast Learning Algorithm for Deep Belief Nets," Neural Computation, vol. 18, pp. 1527-1554, 2006.
[59] G. Hinton, http://videolectures.netnips09_hinton_dlmi /, 2013.
[60] R. Gonzalez and R. Woods, Digital Image Processing. Prentice Hall, 2008.
[61] R. Duda, P. Hart, and D. Stork, Pattern Classification. John Wiley and Sons, 2001.
[62] K. Fukunaga, Introduction to Statistical Pattern Recognition. Elsevier, 1990.
[63] A. Dempster, M. Laird, and D. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc., vol. 39, no. 1, pp. 1-38, 1977.
[64] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, Mar. 2004.
[65] P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 511-518, 2001.
[66] G. Carneiro, J.C. Nascimento, and A. Freitas, "The Segmentation of the Left Ventricle of the Heart from Ultrasound Data Using Deep Learning Architectures and Derivative-Based Search Methods," IEEE Trans. Image Processing, vol. 21, no. 3, pp. 968-982, Mar. 2012.
[67] A. Hammoude, "Computer-Assisted Endocardial Border Identification from a Sequence of Two-Dimensional Echocardiographic Images," PhD dissertation, Univ. of Washington, 1988.
[68] X.S. Zhou, D. Comaniciu, and A. Gupta, "An Information Fusion Framework for Robust Shape Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 115-129, Jan. 2005.
[69] I. Mikić, S. Krucinki, and J.D. Thomas, "Segmentation and Tracking in Echocardiographic Sequences: Active Contours Guided by Optical Flow Estimates," IEEE Trans. Medical Imaging, vol. 17, no. 2, pp. 274-284, Mar. 1998.
[70] V. Chalana and Y. Kim, "A Methodology for Evaluation of Boundary Detection Algorithms on Medical Images," IEEE Trans. Medical Imaging, vol. 16, no. 10, pp. 642-652, Oct. 1997.
[71] C. Alberola-Lopez, M. Martin-Fernandez, and J. Ruiz-Alzola, "Comments on: A Methodology for Evaluation of Boundary Detection Algorithms on Medical Images," IEEE Trans. Medical Imaging, vol. 23, no. 5, pp. 658-660, May 2004.
[72] J. Bland and A. Altman, "Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement," Lancet, vol. 1, no. 8476, pp. 307-310, 1986.
[73] J.C. Reiber, A.R. Viddeleer, G. Koning, M.J. Schalij, and P.E. Lange, "Left Ventricular Regression Equations from Single Plane Cine and Digital X-Ray Ventriculograms Revisited," Clinical Cardiology, vol. 12, no. 2, pp. 69-78, 1996.
[74] H. Sandler and H.T. Dodge, "The Use of Single Plane Angiocardiograms for the Calculation of Left Ventricular Volume in Man," Am. Heart J., vol. 75, no. 3, pp. 325-334, 1968.
[75] R. Fisher, "Questions and Answers Number 14," The Am. Statistician, vol. 2, no. 5, pp. 30-31, 1948.
[76] X. Zhu, "Semi-Supervised Learning Literature Survey," Technical Report 1530, Computer Sciences, Univ. of Wisconsin–Madison, 2005.
8 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool