Feature selection method with joint maximal information entropy between features and class

Feature selection remains a popular method for quantity reduction of attributes of high-dimensional data, to reduce computational costs in classifications. A new feature selection method based on the joint maximal information entropy between features and class (FS-JMIE) is proposed in this paper. Firstly, the joint maximal information entropy (JMIE) is defined to measure a feature subset. Next, a binary particle swarm optimization (BPSO) algorithm is introduced to search the optimal feature subset. Finally, classification is performed on UCI corpora to verify the performance of our proposed method compared to the traditional mutual information (MI) method, CHI method, as well as a binary version of particle swarm optimization-support vector machines (BPSO-SVMs) feature selection. Experiments show that FS-JMIE achieves an equal or better performance than MI, CHI, and BPSO-SVM. Further, FS-JMIE manifests relatively better robustness to the number of classes. Moreover, the method shows higher consistency and better time-efficiency than BPSO-SVM.