Feature selection by integrating two groups of feature evaluation criteria

Feature selection is a preprocessing step in many application areas that are relevant to expert and intelligent systems, such as data mining and machine learning. Feature selection criteria that are based on information theory can be generally sorted into two categories. The criteria in the first group focus on minimizing feature redundancy, whereas those in the second group aim to maximize new classification information. However, both groups of feature evaluation criteria fail to balance the importance of feature redundancy and new classification information. Therefore, we propose a hybrid feature selection method named Minimal Redundancy-Maximal New Classification Information (MR-MNCI) that integrates the two groups of feature selection criteria. Moreover, according to the characteristics of the two groups of selection criteria, we adopt class-dependent feature redundancy and class-independent feature redundancy. To evaluate MR-MNCI, seven competing feature selection methods are compared with our method on 12 real-world data sets. Our method achieves the best classification performance in terms of average classification accuracy and highest classification accuracy.