On-line incremental feature weighting in evolving fuzzy classifiers

In this paper, we present an approach for addressing the problem of dynamic dimension reduction during on-line training, evolution and updating of evolving fuzzy classifiers (EFC). The basic idea of our approach is that, instead of permanently changing the list of most important features with newly loaded data blocks, we generalize the concept of incremental feature selection to an incremental feature weighting approach: features are assigned weights in [0,1] according to their importance level. These weights are permanently updated during on-line mode and guarantee a smooth learning process in the evolving fuzzy classifiers, as they change softly and continuously over time. In some cases, when the weights become (approximately) 0, an automatic switching off of some features and therefore a (soft) dimension reduction is achieved. Two novel incremental feature weighting strategies are proposed in this paper, one based on a leave-one-feature-out, the other based on a feature-wise separability criterion. We will describe the integration concept of the feature weights in the evolving fuzzy classifiers, using single and multi-model architecture, where FLEXFIS-Class SM and FLEXFIS-Class MM serve as training engines. The whole approach of integrated incremental feature weighting in evolving fuzzy classifiers will be evaluated based on high-dimensional on-line real-world classification scenarios and based on data from the Internet. The results will show that incremental feature weighting in EFC in fact helps to reduce curse of dimensionality and therefore guides the evolving fuzzy classifiers to a higher on-line predictive power.