The Community for Technology Leaders
RSS Icon
Issue No.05 - May (2009 vol.31)
pp: 953-960
Yuanhong Li , Wayne State University, Detroit
Ming Dong , Wayne State University, Detroit
Jing Hua , Wayne State University, Detroit
In this paper, we propose a novel approach of simultaneous localized feature selection and model detection for unsupervised learning. In our approach, local feature saliency, together with other parameters of Gaussian mixtures, are estimated by Bayesian variational learning. Experiments performed on both synthetic and real-world data sets demonstrate that our approach is superior over both global feature selection and subspace clustering methods.
Unsupervised, localized, feature selection, Bayesian.
Yuanhong Li, Ming Dong, Jing Hua, "Simultaneous Localized Feature Selection and Model Detection for Gaussian Mixtures", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 5, pp. 953-960, May 2009, doi:10.1109/TPAMI.2008.261
[1] A. Jain and D. Zongker, “Feature Selection: Evaluation, Application, and Small Sample Performance,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp.153-158, Feb. 1997.
[2] Y. Li, M. Dong, and J. Hua, “Localized Feature Selection for Clustering,” Pattern Recognition Letters, vol. 29, pp.10-18, 2008.
[3] S. Avidan, “Joint Feature-Basis Subset Selection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2004.
[4] Y. Wu and A. Zhang, “Feature Selection for Classifying High-Dimensional Numerical Data,” Proc. IEEE Conf.. Computer Vision and Pattern Recognition, June 2004.
[5] H. Peng, F. Long, and C. Ding, “Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp.1226-1238, Aug. 2005.
[6] S.K. Singhi and H. Liu, “Feature Subset Selection Bias for Classification Learning,” Proc. 23rd Int'l Conf. Machine Learning, pp.849-856, 2006.
[7] M. Dong and R. Kothari, “Feature Subset Selection Using a New Definitionof Classifiability,” Pattern Recognition Letters, vol. 23, pp.1215-1225, 2003.
[8] S. Chang, N. Dasgupta, and L. Carin, “A Bayesian Approach to Unsupervised Feature Selection and Density Estimation Using Expectation Propagation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp.1043-1050, June 2005.
[9] J.G. Dy and C.E. Brodley, “Feature Selection for Unsupervised Learning,” J.Machine Learning Research, vol. 5, pp.845-889, 2004.
[10] P. Mitra, C. Murthy, and S. Pal, “Unsupervised Feature Selection Using Feature Similarity,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp.301-312, Mar. 2002.
[11] M.H. Law, M.A.T. Figueiredo, and A.K. Jain, “Simultaneous Feature Selection and Clustering Using Mixture Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp.1154-1166, Sept. 2004.
[12] H. Liu and L. Yu, “Toward Integrating Feature Selection Algorithms for Classification and Clustering,” IEEE Trans. Knowledge and Data Eng., vol. 17, no. 4, pp.491-502, Apr. 2005.
[13] M. Dash, K. Choi, P. Scheuermann, and H. Liu, “Feature Selection for Clustering—A Filter Solution,” Proc. IEEE Conf. Information and Data Mining, pp.115-122, 2002.
[14] C. Constantinopoulos, M.K. Titsias, and A. Likas, “Bayesian Feature and Model Selection for Gaussian Mixture Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp.1013-1018, June 2006.
[15] M. Dash and H. Liu, “Feature Selection for Clustering,” Proc. Pacific-Asia Conf. Knowledge Discovery and Data Mining, pp.110-121, 2000.
[16] A.E. Raftery and N. Dean, “Variable Selection for Model-Based Clustering,” J. Am. Statistical Assoc., vol. 101, no. 473, pp.168-178, 2006.
[17] H. Zha, X. He, C. Ding, M. Gu, and H. Simon, “Bipartite Graph Partitioning and Data Clustering,” Proc. ACM Int'l Conf. Knowledge Management, pp.25-32, Nov. 2001.
[18] M. Rege, M. Dong, and F. Fotouhi, “Co-Clustering Documents and Words Using Bipartite Isoperimetric Graph Partitioning,” Proc. IEEE Int'l Conf. Data Mining, 2006.
[19] L. Parsons, E. Haque, and H. Liu, “Subspace Clustering for High Dimensional Data: A Review,” SIGKDD Explore Newsletter, vol. 6, no. 1, pp.90-105, 2004.
[20] Q. Ke and T. Kanade, “Robust Subspace Clustering by Combined Use of NND Metric and SVD Algorithm,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp.592-599, June 2004.
[21] C. Baumgartner, C. Plant, K. Kailing, H.-P. Kriegel, and P. Kröger, “Subspace Selection for Clustering High-Dimensional Data,” Proc. Fourth IEEE Int'l Conf. Data Mining, pp.11-18, 2004.
[22] R. Agrawal, J. Gehrke, D. Gunopulos, and P. Raghavan, “Automatic Subspace Clustering of High Dimensional Data for Data Mining Applications,” Proc. ACM-SIGMOD Int'l Conf. Management of Data, pp.94-105, 1998.
[23] C.C. Aggarwal, C.M. Procopiuc, J.L. Wolf, P.S. Yu, and J.S. Park, “Fast Algorithms for Projected Clustering,” Proc. ACM-SIGMOD Int'l Conf. Management of Data, pp.61-72, 1999.
[24] J.H. Friedman and J.J. Meulman, “Clustering Objects on Subsets of Attributes,” J. Royal Statistical Soc.: Series B, vol. 66, no. 4, pp.815-849, 2004.
[25] M.W. Graham and D.J. Miller, “Unsupervised Learning of Parsimonious Mixtures on Large Spaces with Integrated Feature and Component Selection,” IEEE Trans. Signal Processing, vol. 54, no. 4, pp.1289-1303, 2006.
[26] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. John Willey & Sons, Inc., 2000.
[27] A. Asuncion and D. Newman, UCI Machine Learning Repository,, 2007.
[28] C.M. Bishop, Pattern Recognition and Machine Learning, chapter 10, first ed. Springer, 2006.
664 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool