A novel feature selection method considering feature interaction

Interacting features are those that appear to be irrelevant or weakly relevant with the class individually, but when it combined with other features, it may highly correlate to the class. Discovering feature interaction is a challenging task in feature selection. In this paper, a novel feature selection algorithm considering feature interaction is proposed. Firstly, feature relevance, feature redundancy and feature interaction have been redefined in the framework of information theory. Then the interaction weight factor which can reflect the information of whether a feature is redundant or interactive is proposed. Afterwards, we bring forward an Interaction Weight based Feature Selection algorithm (IWFS). To evaluate the performance of the proposed algorithm, we compare IWFS with other five representative feature selection algorithms, including CFS, INTERACT, FCBF, MRMR and Relief-F, in terms of the classification accuracies and the number of selected features with three different types of classifiers including C4.5, IB1 and PART. The results on the six synthetic datasets show that IWFS can effectively identify irrelevant and redundant features while reserving interactive ones. The results on the eight real world datasets indicate that IWFS not only efficiently reduces the dimensionality of feature space, but also offers the highest average accuracy for all the three classification algorithms.