Publication 2010 Issue No. 7 - July Abstract - A Combinatorial Solution for Model-Based Image Segmentation and Real-Time Tracking
A Combinatorial Solution for Model-Based Image Segmentation and Real-Time Tracking
July 2010 (vol. 32 no. 7)
pp. 1153-1164
 ASCII Text x Thomas Schoenemann, Daniel Cremers, "A Combinatorial Solution for Model-Based Image Segmentation and Real-Time Tracking," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 7, pp. 1153-1164, July, 2010.
 BibTex x @article{ 10.1109/TPAMI.2009.79,author = {Thomas Schoenemann and Daniel Cremers},title = {A Combinatorial Solution for Model-Based Image Segmentation and Real-Time Tracking},journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},volume = {32},number = {7},issn = {0162-8828},year = {2010},pages = {1153-1164},doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2009.79},publisher = {IEEE Computer Society},address = {Los Alamitos, CA, USA},}
 RefWorks Procite/RefMan/Endnote x TY - JOURJO - IEEE Transactions on Pattern Analysis and Machine IntelligenceTI - A Combinatorial Solution for Model-Based Image Segmentation and Real-Time TrackingIS - 7SN - 0162-8828SP1153EP1164EPD - 1153-1164A1 - Thomas Schoenemann, A1 - Daniel Cremers, PY - 2010KW - Image segmentationKW - trackingKW - elastic shape priorsKW - discrete optimizationKW - dynamic programmingKW - minimum ratio cyclesKW - real-time applications.VL - 32JA - IEEE Transactions on Pattern Analysis and Machine IntelligenceER -
Thomas Schoenemann, University of Bonn, Bonn
Daniel Cremers, University of Bonn, Bonn
We propose a combinatorial solution to determine the optimal elastic matching of a deformable template to an image. The central idea is to cast the optimal matching of each template point to a corresponding image pixel as a problem of finding a minimum cost cyclic path in the three-dimensional product space spanned by the template and the input image. We introduce a cost functional associated with each cycle, which consists of three terms: a data fidelity term favoring strong intensity gradients, a shape consistency term favoring similarity of tangent angles of corresponding points, and an elastic penalty for stretching or shrinking. The functional is normalized with respect to the total length to avoid a bias toward shorter curves. Optimization is performed by Lawler's Minimum Ratio Cycle algorithm parallelized on state-of-the-art graphics cards. The algorithm provides the optimal segmentation and point correspondence between template and segmented curve in computation times that are essentially linear in the number of pixels. To the best of our knowledge, this is the only existing globally optimal algorithm for real-time tracking of deformable shapes.

[1] R.E. Bellman, "On a Routing Problem," Quarterly Applied Math., vol. 16, pp. 87-90, 1958.
[2] A. Blake and M. Isard, Active Contours. Springer Verlag, 1998.
[3] V. Caselles, R. Kimmel, G. Sapiro, and C. Sbert, "Minimal Surfaces Based Object Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 4, pp. 394-398, Apr. 1997.
[4] T. Cootes and C.J. Taylor, "Active Shape Model Search Using Local Grey-Level Models: A Quantitative Evaluation," Proc. British Machine Vision Conf., pp. 639-648, 1993.
[5] J. Coughlan, A. Yuille, C. English, and D. Snow, "Efficient Deformable Template Detection and Localization without User Initialization," Computer Vision and Image Understanding, vol. 78, no. 3, pp. 303-319, 2000.
[6] D. Cremers, "Dynamical Statistical Shape Priors for Level Set Based Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1262-1273, Aug. 2006.
[7] D. Cremers, S.J. Osher, and S. Soatto, "Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation," Int'l J. Computer Vision, vol. 69, no. 3, pp. 335-351, Sept. 2006.
[8] D. Cremers, F.R. Schmidt, and F. Barthel, "Shape Priors in Variational Image Segmentation: Convexity, Lipschitz Continuity and Globally Optimal Solutions," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, June 2008.
[9] D. Cremers, F. Tischhäuser, J. Weickert, and C. Schnörr, "Diffusion Snakes: Introducing Statistical Shape Knowledge into the Mumford-Shah Functional," Int'l J. Computer Vision, vol. 50, no. 3, pp. 295-313, 2002.
[10] F. Dellaert and C. Thorpe, "Robust Car Tracking Using Kalman Filtering and Bayesian Templates," Proc. Conf. Intelligent Transportation Systems, 1997.
[11] J. Denzler and H. Niemann, "Active Rays: Polar-Transformed Active Contours for Real-Time Contour Tracking," Real-Time Imaging, vol. 5, pp. 203-213, 1999.
[12] A. Doucet, N. de Freitas, and N. Gordon, Sequential Monte Carlo Methods in Practice (Statistics for Engineering and Information Science). Springer Verlag, 2001.
[13] P.F. Felzenszwalb, "Representation and Detection of Deformable Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 2, pp. 208-220, Feb. 2005.
[14] P.F. Felzenszwalb and D. Huttenlocher, "Pictorial Structures for Object Recognition," Int'l J. Computer Vision, vol. 61, no. 1, pp. 55-79, 2005.
[15] M.A. Fischler and R.A. Eschlager, "The Representation and Matching of Pictorial Structures," IEEE Trans. Computers, vol. 22, no. 1, pp. 67-92, Jan. 1973.
[16] L.R. Ford, "Network Flow Theory," Paper P-923, The Rand Corporation, 1956.
[17] W. Förstner and E. Gülch, "A Fast Operator for Detection and Precise Localization of Distinct Points, Corners and Circular Features," Proc. Intercommission Conf. Fast Processing of Photogrammetric Data, pp. 281-305, 1987.
[18] Y. Gdalyahu and D. Weinshall, "Flexible Syntactic Matching of Curves and Its Application to Automatic Hierarchical Classification of Silhouettes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1312-1328, Dec. 1999.
[19] U. Grenander, Y. Chow, and D.M. Keenan, Hands: A Pattern Theoretic Study of Biological Shapes. Springer Verlag, 1991.
[20] L. Gui, J. Thiran, and N. Paragios, "Joint Object Segmentation and Behaviour Classification in Image Sequences," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007.
[21] G.D. Hager and P.N. Belhumeur, "Real-Time Tracking of Image Regions with Changes in Geometry and Illumination," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 403-410, 1996.
[22] C. Harris and M. Stephens, "A Combined Corner and Edge Detector," Proc. Fourth Alvey Vision Conf., pp. 147-151, 1988.
[23] A. Jalba, M. Wilkinson, and J. Roerdink, "CPM: A Deformable Model for Shape Recovery and Segmentation Based on Charged Particles," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1320-1335, Oct. 2004.
[24] I.H. Jermyn and H. Ishikawa, "Globally Optimal Regions and Boundaries as Minimum Ratio Weight Cycles," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1075-1088, Oct. 2001.
[25] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active Contour Models," Int'l J. Computer Vision, vol. 1, no. 4, pp. 321-331, 1988.
[26] L.J. Latecki and R. Lakämper, "Shape Similarity Measure Based on Correspondence of Visual Parts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1185-1190, Oct. 2000.
[27] E.L. Lawler, "Optimal Cycles in Doubly Weighted Linear Graphs," Proc. Int'l Symp. Theory of Graphs, pp. 209-213, 1966.
[28] V. Lempitsky, A. Blake, and C. Rother, "Image Segmentation by Branch-and-Mincut," Proc. European Conf. Computer Vision, Oct. 2008.
[29] M. Leventon, W. Grimson, and O. Faugeras, "Statistical Shape Influence in Geodesic Active Contours," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 316-323, 2000.
[30] D. Lowe, "Object Recognition from Local Scale-Invariant Features," Proc. IEEE Int'l Conf. Computer Vision, Sept. 1999.
[31] M. Maes, "Polygonal Shape Recognition Using String-Matching Techniques," Pattern Recognition, vol. 24, no. 5, pp. 433-440, 1991.
[32] R. McConnell, R. Kwok, J.C. Curlander, W. Kober, and S.S. Pang, "$\psi - s$ Correlation and Dynamic Time Warping: Two Methods for Tracking Ice Floes in SAR Images," IEEE Trans. Geosciences and Remote Sensing, vol. 29, no. 11, pp. 1004-1012, Nov. 1991.
[33] E.F. Moore, "The Shortest Path through a Maze," Proc. Int'l Symp. Theory of Switching, pp. 285-292, 1959.
[34] D. Mumford and J. Shah, "Optimal Approximations by Piecewise Smooth Functions and Associated Variational Problems," Comm. Pure and Applied Math., vol. 42, pp. 577-685, 1989.
[35] D. Ramanan and C. Sminchisescu, "Training Deformable Models for Localization," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 206-213, June 2006.
[36] M. Rousson and D. Cremers, "Efficient Kernel Density Estimation of Shape and Intensity Priors for Level Set Segmentation," Proc. Int'l Conf. Medical Image Computing and Computer Assisted Intervention, vol. 1, pp. 757-764, 2005.
[37] M. Rousson and N. Paragios, "Shape Priors for Level Set Representations," Proc. European Conf. Computer Vision, pp. 78-92, 2002.
[38] T. Schoenemann and D. Cremers, "Globally Optimal Image Segmentation with an Elastic Shape Prior," Proc. IEEE Int'l Conf. Computer Vision, Oct. 2007.
[39] T. Schoenemann and D. Cremers, "Globally Optimal Shape-Based Tracking in Real-Time," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, June 2008.
[40] T. Schoenemann and D. Cremers, "Matching Non-Rigidly Deformable Shapes across Images: A Globally Optimal Solution," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, June 2008.
[41] T. Schoenemann, F.R. Schmidt, and D. Cremers, "Image Segmentation with Elastic Shape Priors via Global Geodesics in Product Spaces," Proc. British Machine Vision Conf., Sept. 2008.
[42] J. Shi and C. Tomasi, "Good Features to Track," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, June 1994.
[43] A. Tsai, A. Yezzi, W. Wells, C. Tempany, D. Tucker, A. Fan, E. Grimson, and A. Willsky, "Model-Based Curve Evolution Technique for Image Segmentation," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 463-468, 2001.
[44] X. Xie and M. Mirmehdi, "MAC: Magnetostatic Active Contour Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp. 632-646, Apr. 2008.
[45] C. Xu and J. Prince, "Generalized Gradient Vector Flow External Forces for Active Contours," Signal Processing, vol. 71, no. 2, pp. 131-139, 1998.

Index Terms:
Image segmentation, tracking, elastic shape priors, discrete optimization, dynamic programming, minimum ratio cycles, real-time applications.
Citation:
Thomas Schoenemann, Daniel Cremers, "A Combinatorial Solution for Model-Based Image Segmentation and Real-Time Tracking," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 7, pp. 1153-1164, July 2010, doi:10.1109/TPAMI.2009.79