Feature selection considering the composition of feature relevancy

Feature selection plays a critical role in classification problems. Feature selection methods intend to retain relevant features and eliminate redundant features. This work focuses on feature selection methods based on information theory. By analyzing the composition of feature relevancy, we believe that a good feature selection method should maximize new classification information while minimizing feature redundancy. Therefore, a novel feature selection method named Composition of Feature Relevancy (CFR) is proposed. To evaluate CFR, we conduct experiments on eight real-world data sets and two different classifiers (Naïve-Bayes and Support Vector Machine). Our method outperforms five other competing methods in terms of average classification accuracy and highest classification accuracy.