This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Mean Shift: A Robust Approach Toward Feature Space Analysis
May 2002 (vol. 24 no. 5)
pp. 603-619

A general nonparametric technique is proposed for the analysis of a complex multimodal feature space and to delineate arbitrarily shaped clusters in it. The basic computational module of the technique is an old pattern recognition procedure, the mean shift. We prove for discrete data the convergence of a recursive mean shift procedure to the nearest stationary point of the underlying density function and, thus, its utility in detecting the modes of the density. The relation of the mean shift procedure to the Nadaraya-Watson estimator from kernel regression and the robust M-estimators of location is also established. Algorithms for two low-level vision tasks, discontinuity preserving smoothing and image segmentation, are described as applications. In these algorithms, the only user set parameter is the resolution of the analysis and either gray level or color images are accepted as input. Extensive experimental results illustrate their excellent performance.

[1] G. Aggarwal, S. Ghosal, and P. Dubey, “Efficient Query Modification for Image Retrieval,” Proc. 2000 IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 255-261, June 2000.
[2] R. Bajcsy, S.W. Lee, and A. Leonardis, “Detection of Diffuse and Specular Interface Reflections and Inter-Reflections by Color Image Segmentation,” Int'l J. Computer Vision, vol. 17, pp. 241-272, 1996.
[3] D. Barash, “Bilateral Filtering and Anisotropic Diffusion: Towards a Unified Viewpoint,” IEEE Trans. Pattern Analysis and Pattern Analysis, to appear.
[4] D.P. Bertsekas, Nonlinear Programming. Athena Scientific, 1995.
[5] M. Black, G. Sapiro, D. Marimont, and D. Heeger, “Robust Anisotropic Diffusion,” IEEE Trans. Image Processing, vol. 7, no. 3, pp. 421-432, 1998.
[6] G.R. Bradski, “Computer Vision Face Tracking as a Component of a Perceptual User Interface,” Proc. IEEE Workshop Applications of Computer Vision, pp. 214-219, Oct. 1998.
[7] Y. Cheng, Mean Shift, Mode Seeking, and Clustering IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 790-799, Aug. 1995.
[8] E. Choi and P. Hall, “Data Sharpening as a Prelude to Density Estimation,” Biometrika, vol. 86, pp. 941-947, 1999.
[9] C.K. Chu, I.K. Glad, F. Godtliebsen, and J.S. Maron, “Edge-Preserving Smoothers for Image Processing,” J. Am. Statistical Assoc., vol. 93, pp. 526-541, 1998.
[10] D. Comaniciu, “Nonparametric Robust Methods for Computer Vision,” PhD thesis, Dept. of Electrical and Computer Eng., Rutgers Univ., 1999. Available at.
[11] D. Comaniciu and P. Meer, Robust Analysis of Feature Spaces: Color Image Segmentation Proc. 1997 IEEE Conf. Computer Vision and Pattern Recognition, pp. 750-755, 1997.
[12] D. Comaniciu and P. Meer, “Distribution Free Decomposition of Multivariate Data,” Pattern Analysis and Applications, vol. 2, pp. 22-30, 1999.
[13] D. Comaniciu and P. Meer, Mean Shift Analysis and Applications Proc. IEEE Seventh Int'l Conf. Computer Vision, vol. 2, pp. 1197-1203, Sept. 1999.
[14] D. Comaniciu, V. Ramesh, and P. Meer, Real-Time Tracking of Non-Rigid Objects Using Mean Shift Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 142-149, 2000.
[15] D. Comaniciu, V. Ramesh, and P. Meer, “The Variable Bandwidth Mean Shift and Data-Driven Scale Selection,” Proc Eighth Int'l Conf. Computer Vision, vol. I, pp. 438-445, July 2001.
[16] C. Connolly, “The Relationship between Colour Metrics and the Appearance of Three-Dimensional Coloured Objects,” Color Research and Applications, vol. 21, pp. 331-337, 1996.
[17] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. Wiley, 1973.
[18] A. Elgammal, D. Harwood, and L. Davis, “Non-Parametric Model for Background Subtraction,” Proc. European Conf. Computer Vision, vol. II, pp. 751-767, June 2000.
[19] B. Fischl and E.L. Schwartz, “Adaptive Nonlocal Filtering: A Fast Alternative to Anisotropic Diffusion for Image Enhancement,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 1, pp. 42-48, Jan. 1999.
[20] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[21] K. Fukunaga and L.D. Hostetler,“The estimation of the gradient of a density function, with applications in pattern recognition,” IEEE Trans. Information Theory, vol. 21, pp. 32-40, 1975.
[22] J. Guo, J. Kim, and C. Kuo, “Fast and Accurate Moving Object Extraction Technique for MPEG-4 Object Based Video Coding,” Proc. SPIE Visual Comm. and Image Processing, vol. 3653, pp. 1210-1221, 1999.
[23] G. Guy and G. Medioni, “Inference of Surfaces, 3D Curves, and Junctions from Sparse, Noisy, 3D Data,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 11, pp. 1265-1277, Nov. 1997.
[24] W. Härdle, Applied Nonparameteric Regression. Cambridge Univ. Press, 1991.
[25] M. Herbin, N. Bonnet, and P. Vautrot, “A Clustering Method Based on the Estimation of the Probability Density Function and on the Skeleton by Influence Zones,” Pattern Recognition Letters, vol. 17, pp. 1141-1150, 1996.
[26] P.J. Huber, Robust Statistical Procedures, second ed. SIAM, 1996.
[27] J.N. Hwang and S.R. Lay, and A. Lippman, “Nonparametric Multivariate Density Estimation: A Comparative Study,” IEEE Trans. Signal Processing, vol. 42, pp. 2795-2810, 1994.
[28] A.K. Jain and R.C. Dubes, Algorithms for Clustering Data. Englewood Cliffs, N.J.: Prentice Hall, 1988.
[29] A.K. Jain, R.P.W. Duin, and J. Mao, Statistical Pattern Recognition: A Review IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4-37, Jan. 2000.
[30] L. Kauffman and P. Rousseeuw, Finding Groups in Data: An Introduction to Cluster Analysis. J. Wiley&Sons, 1990.
[31] D.T. Kuan, A.A. Sawchuk, T.C. Strand, and P. Chavel, “Adaptive Noise Smoothing Filter for Images with Signal Dependent Noise,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 2, pp. 165-177, Mar. 1985.
[32] G. Li, “Robust Regression,” Exploring Data Tables, Trends, and Shapes, D.C. Hoaglin, F. Mosteller, and J.W. Tukey, eds., pp. 281-343. Wiley, 1985.
[33] L. Liu and S. Sclaroff, “Deformable Shape Detection and Description via Model-Based Region Grouping,” Proc. 1999 IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 21-27, June 1999.
[34] W.Y. Ma and B.S. Manjunath, “EdgeFlow: A Technique for Boundary Detection and Image Segmentation,” IEEE Trans. Image Processing, vol. 9, no. 8, pp. 1375-1388, 2000.
[35] B.A. Maxwell and S.S. Shafer, “Segmentation and Interpretation of Multicolored Objects with Highlights,” Computer Vision and Image Understanding, vol. 77, pp. 1-24, 2000.
[36] A. Montanvert, P. Meer, and A. Rosenfeld, Hierarchical Image Analysis Using Irregular Tesselations IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 4, pp. 307-316, Apr. 1991.
[37] J. Monteil and A. Beghdadi, “A New Interpretation and Improvement of Nonlinear Anisotropic Diffusion for Image Enhancement,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 9, pp. 940-946, Sept. 1999.
[38] Y. Ohta, T. Kanade, and T. Sakai, “Color Information for Region Segmentation,” Computer Graphics and Image Processing, vol. 13, pp. 222-241, 1980.
[39] J. Pan, F. McInnes, and M. Jack, “Fast Clustering Algorithms for Vector Quantization,” Pattern Recognition, vol. 29, pp. 511-518, 1996.
[40] T. Pappas, “An Adaptive Clustering Algorithm for Image Segmentation,” IEEE Trans. Signal Processing, vol. 40, Apr. 1992.
[41] N. Paragios and R. Deriche, Geodesic Active Contours for Supervised Texture Segmentation Proc. Conf. Computer Vision Pattern Recognition, pp. 422-427, 1999.
[42] B. Park and J. Marron, “Comparison of Data-Driven Bandwidth Selectors,” J. Am. Statistical Assoc., vol. 85, pp. 66-72, 1990.
[43] E.J. Pauwels and G. Frederix, “Nonparametric Clustering for Image Segmentation and Grouping,” Image Understanding, vol. 75, no. 1, pp. 73-85, 2000.
[44] P. Perona and J. Malik, "Scale-Space and Edge Detection Using Anisotropic Diffusion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 629639, July 1990.
[45] K. Popat and R.W. Picard, Cluster-Based Probability Model and Its Application to Image and Texture Processing IEEE Trans. Image Processing, vol. 6, no. 2, pp. 268-284, 1997.
[46] W.K. Pratt, Digital Image Processing. Wiley-Interscience, 1991.
[47] D. Ridder, J. Kittler, O. Lemmers, and R. Duin, “The Adaptive Subspace Map for Texture Segmentation,” Proc. 2000 Int'l Conf. Pattern Recognition, pp. 216-220, Sept. 2000.
[48] T. Risse, “Hough Transform for Line Recognition: Complexity of Evidence Accumulation and Cluster Detection,” Computer Vision Graphics and Image Processing, vol. 46, pp. 327-345, 1989.
[49] S.J. Roberts, “Parametric and Non-Parametric Unsupervised Cluster Analysis,” Pattern Recognition, vol. 30, pp. 261-272, 1997.
[50] P. Saint-Marc, J. Chen, and G. Medioni, "Adaptive Smoothing: A General Tool for Early Vision," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, 1991.
[51] D.W. Scott, Multivariate Density Estimation. Wiley, 1992.
[52] R. Sedgewick, Algorithms in C++. Addison-Wesley, 1992.
[53] S. Sheather, M. Jones, “A Reliable Data-Based Bandwidth Selection Method for Kernel Density Estimation,” J. Royal Statistics Soc. B, vol. 53, pp. 683-690, 1991.
[54] B. W. Silverman, Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986.
[55] J. Simonoff, Smoothing Methods in Statistics. Springer-Verlag, 1996.
[56] Special Issue on Partial Differential Equations and Geometry-Driven Diffusion in Image Processing and Analysis, IEEE Trans. Image Processing, vol. 7, Mar. 1998.
[57] M. Tabb and N. Ahuja, “Multiscale Image Segmentation by Integrated Edge and Region Detection,” IEEE Trans. Image Processing, vol. 6, no. 5, pp. 642-655, 1997.
[58] C.-K. Tang, G. Medioni, and M.-S. Lee, “Epipolar Geometry Estimation by Tensor Voting in 8D,” Proc. IEEE Int'l Conf. Computer Vision, pp. 502-509, 1999.
[59] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” Proc. Sixth Int'l Conf. Computer Vision, pp. 839-846, Jan. 1998.
[60] A. Touzani and J.G. Postaire, “Clustering by Mode Boundary Detection,” Pattern Recognition Letters, vol. 9, pp. 1-12, 1989.
[61] T. Tuytelaars, L. Van Gool, M. Proesmans, and T. Moons, “The Cascaded Hough Transform as an Aid in Aerial Image Interpretation,” Proc. Sixth Int'l Conf. Computer Vision, pp. 67-72, Jan. 1998.
[62] M.P. Wand and M. Jones, Kernel Smoothing. Chapman and Hall, 1995.
[63] R. Wilson and M. Spann, “A New Approach to Clustering,” Pattern Recognition, vol. 23, no. 12, pp. 1413-1425, 1990.
[64] C. Wren, A. Azarbayejani, T. Darrell, and A.P. Pentland, Pfinder: Real-Time Tracking of the Human Body IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780-785, July 1997.
[65] G. Wyszecki and W.S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae. second ed. Wiley, 1982.
[66] S.C. Zhu and A. Yuille, “Region Competition: Unifying Snakes, Region Growing and Bayes/MDL for Multiband Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, pp. 884-900, 1996.
[67] X. Zhuang, Y. Huang, K. Palaniappan, and Y. Zhao, “Gaussian Mixture Density Modeling Decomposition, and Applications,” IEEE Trans. Image Processing, vol. 5, pp. 1293-1302, 1996.

Index Terms:
Mean shift, clustering, image segmentation, image smoothing, feature space, low-level vision
Citation:
D. Comaniciu, P. Meer, "Mean Shift: A Robust Approach Toward Feature Space Analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002, doi:10.1109/34.1000236
Usage of this product signifies your acceptance of the Terms of Use.