A relative decision entropy-based feature selection approach

• We proposed a novel heuristic feature selection algorithm in rough sets.
• We presented a new information entropy model – relative decision entropy.
• We proved that relative decision entropy is monotonic with respect to the partial order of partitions.
• We applied our feature selection algorithm to intrusion detection.
• The effectiveness of our algorithm was shown on KDD-99 data set and some other data sets.

Rough set theory has been proven to be an effective tool for feature selection. To avoid the exponential computation in exhaustive methods, many heuristic feature selection algorithms have been proposed in rough sets. However, these algorithms still suffer from high computational cost. In this paper, we propose a novel heuristic feature selection algorithm (called FSMRDE) in rough sets. To measure the significance of features in FSMRDE, we propose a new model of relative decision entropy, which is an extension of Shannon׳s information entropy in rough sets. Moreover, to test the effectiveness of FSMRDE, we apply it to intrusion detection and other application domains. Experimental results show that by using the relative decision entropy-based feature significance as heuristic information, FSMRDE is efficient for feature selection. In particular, FSMRDE is able to achieve good scalability for large data sets.