An adaptive streaming active learning strategy based on instance weighting

•The concept of the “sufficient weight” that we use as an uncertainty measure.•An efficient uncertainty-based active learning for classification.•An adaptive uncertainty threshold for the streaming active learning.

This paper addresses stream-based active learning for classification. We propose a new query strategy based on instance weighting that improves the performance of the active learner compared to the commonly used uncertainty strategies. The proposed strategy computes the smallest weight that should be associated with new instance, so that the classifier changes its prediction regarding this instance. If a small weight is sufficient to change the predicted label, then the classifier was uncertain about its prediction, and the true label is queried from a labeller. In order to determine whether the sufficient weight is “small enough”, we propose an adaptive uncertainty threshold which is suitable for the streaming setting. The proposed adaptive threshold allows the stream-based active learner to achieve an accuracy which is similar to that of a fully supervised learner, while querying much less labels. Experiments on several public and real world data prove the effectiveness of the proposed method.