Mutual information-based method for selecting informative feature sets

• We evaluate the performance of eight different mutual information-based feature selection methods.
• Based on the analysis of the evaluation results, we propose a novel mutual information-based feature selection method.
• The effectiveness of the proposed method is demonstrated by the experimental results on UCI datasets and object recognition.

Feature selection is one of the fundamental problems in pattern recognition and data mining. A popular and effective approach to feature selection is based on information theory, namely the mutual information of features and class variable. In this paper we compare eight different mutual information-based feature selection methods. Based on the analysis of the comparison results, we propose a new mutual information-based feature selection method. By taking into account both the class-dependent and class-independent correlation among features, the proposed method selects a less redundant and more informative set of features. The advantage of the proposed method over other methods is demonstrated by the results of experiments on UCI datasets (Asuncion and Newman, 2010 [1]) and object recognition.