Feature selection based on mutual information and redundancy-synergy coefficient

INTRODUCTION Feature subset selection (FSS) is a data miningfundamental problem to select out relevant featuresand cast away irrelevant and redundant featuresfrom an original feature set (Liu and Motoda, 1998).If a feature subset satisfies the FSS measure and hasthe minimal size, it is regarded as the optimal fea-ture subset. Complete search strategy is the way toobtain an optimal feature subset. Branch and Bound(Narendra and Fukunaga, 1977), Focus (Almuallimand Dietterich, 199…

Feature selection based on mutual information and redundancy-synergy coefficient

Mutual information is an important information measure for feature subset. In this paper, a hashing mechanism is proposed to calculate the mutual information on the feature subset. Redundancy-synergy coefficient, a novel redundancy and synergy measure of features to express the class feature, is defined by mutual information. The information maximi- zation rule was applied to derive the heuristic feature subset selection method based on mutual information and redun- dancy-synergy coefficient. Our experiment results showed the good performance of the new feature selection method.