Classifying high-dimensional numerical data is a very challenging problem. In high dimensional feature spaces, the performance of supervised learning methods suffer from the curse of dimensionality, which degrades both classification accuracy and efficiency. To address this issue, we present an efficient feature selection method to facilitate classifying high-dimensional numerical data. Our method employs balanced information gain to measure the contribution of each feature (for data classification); and it calculates feature correlation with a novel extension of balanced information gain. By integrating feature contribution and correlation, our feature selection approach uses a forward sequential selection algorithm to select uncorrelated features with large balanced information gain. Extensive experiments have been carried out on image and gene microarray datasets to demonstrate the effectiveness and robustness of the presented method.