Performance evaluation of anomaly-detection algorithms for mouse dynamics

Mouse dynamics—the analysis of mouse operating behaviors to identify users—has been proposed for detecting impostors. Since many anomaly-detection algorithms have been proposed for this task, it is natural to ask how well these algorithms perform and how they compare with each other (e.g., to identify promising research directions). This paper presents a performance-evaluation study of a range of anomaly-detection algorithms in mouse dynamics on an equal basis. We collected a mouse-dynamics data set consisting of 17,400 samples from 58 subjects, developed a repeatable evaluation methodology, and implemented and evaluated 17 detectors from the mouse-dynamics and pattern-recognition literatures. Performance is measured in terms of detection accuracy, sensitivity to training sample size, usability with respect to sample length, and scalability with respect to the number of users (user space). The six top-performing detectors achieve equal-error rates between 8.81% and 11.63% with a detection time of 6.1 s; detector performance improves as training sample size and sample length increase and becomes saturated gradually; detector performance decreases as user space becomes large, but only small fluctuations with the error range are apparent when the space size exceeds a certain number. Along with the shared data and evaluation methodology, the results constitute a benchmark for comparing detectors and measuring progress.