Our aim is to group data objects, to which the same class labels could be assigned, using a clustering method for high dimensional data sets. In order to group data objects by clustering, we compute the degree of the influence of each attribute for class labels. To find important attributes having large influence on class labels, we use the feature extraction method which we have developed. We can construct a set of data objects which have single class labels with high accuracy by finding sensitive attributes for the class label. Next, we group data objects which have unique class labels by clustering methods. From experimental simulation, we show the effectiveness of the important attribute detection by performing clustering of transformed benchmark data sets as two class classification problems.