Online feature importance ranking based on sensitivity analysis

Online learning is a growing branch of data mining which allows all traditional data mining techniques to be applied on a online stream of data in real time. In this paper, we present a fast and efficient online sensitivity based feature ranking method (SFR) which is updated incrementally. We take advantage of the concept of global sensitivity and rank features based on their impact on the outcome of the classification model. In the feature selection part, we use a two-stage filtering method in order to first eliminate highly correlated and redundant features and then eliminate irrelevant features in the second stage. One important advantage of our algorithm is its generality, which means the method works for correlated feature spaces without preprocessing. It can be implemented along with any single-pass online classification method with separating hyperplane such as SVMs. The proposed method is primarily developed for online tasks, however, we achieve very significant experimental results in comparison with popular batch feature ranking/selection methods. We also perform experiments to compare the method with available online feature ranking methods. Empirical results suggest that our method can be successfully implemented in batch learning or online mode.