Text classification poses some specific challenges. One such challenge is its high dimensionality where each document (data point) contains only a small subset of them. In this paper, we propose Semi-supervised Impurity based Subspace Clustering (SISC) in conjunction with k-Nearest Neighbor approach, based on semi-supervised subspace clustering that considers the high dimensionality as well as the sparse nature of them in text data. SISC finds clusters in the subspaces of the high dimensional text data where each text document has fuzzy cluster membership. This fuzzy clustering exploits two factors - chi square statistic of the dimensions and the impurity measure within each cluster. Empirical evaluation on real world data sets reveals the effectiveness of our approach as it significantly outperforms other state-of-the-art text classification and subspace clustering algorithms.