A comparative study on concept drift detectors

• We evaluated eight different concept drift detectors.
• A 2k factorial design was used to indicate the best parameters for each method.
• Tests compared accuracy, evaluation time, false alarm and miss detection rates.
• A Mahalanobis distance is proposed as a metric to compare drift methods.
• DDM was the method that presented the best average results in all tested datasets.

In data stream environments, drift detection methods are used to identify when the context has changed. This paper evaluates eight different concept drift detectors (ddm, eddm, pht, stepd, dof, adwin, Paired Learners, and ecdd) and performs tests using artificial datasets affected by abrupt and gradual concept drifts, with several rates of drift, with and without noise and irrelevant attributes, and also using real-world datasets. In addition, a 2k2k factorial design was used to indicate the parameters that most influence performance which is a novelty in the area. Also, a variation of the Friedman non-parametric statistical test was used to identify the best methods. Experiments compared accuracy, evaluation time, as well as false alarm and miss detection rates. Additionally, we used the Mahalanobis distance to measure how similar the methods are when compared to the best possible detection output. This work can, to some extent, also be seen as a research survey of existing drift detection methods.