This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Applications
June 2007 (vol. 29 no. 6)
pp. 1035-1051
RELIEF is considered one of the most successful algorithms for assessing the quality of features. In this paper, we propose a set of new feature weighting algorithms that perform significantly better than RELIEF, without introducing a large increase in computational complexity. Our work starts from a mathematical interpretation of the seemingly heuristic RELIEF algorithm as an online method solving a convex optimization problem with a margin-based objective function. This interpretation explains the success of RELIEF in real application and enables us to identify and address its following weaknesses. RELIEF makes an implicit assumption that the nearest neighbors found in the original feature space are the ones in the weighted space and RELIEF lacks a mechanism to deal with outlier data. We propose an iterative RELIEF (I-RELIEF) algorithm to alleviate the deficiencies of RELIEF by exploring the framework of the Expectation-Maximization algorithm. We extend I-RELIEF to multiclass settings by using a new multiclass margin definition. To reduce computational costs, an online learning algorithm is also developed. Convergence analysis of the proposed algorithms is presented. The results of large-scale experiments on the UCI and microarray data sets are reported, which demonstrate the effectiveness of the proposed algorithms, and verify the presented theoretical results.
Index Terms:
Feature weighting, feature selection, RELIEF, iterative algorithm, DNA microarray, classification.
Citation:
Yijun Sun, "Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Applications," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1035-1051, June 2007, doi:10.1109/TPAMI.2007.1093
Usage of this product signifies your acceptance of the Terms of Use.