An object-based comparative methodology for motion detection based on the F-Measure

The majority of visual surveillance algorithms rely on effective and accurate motion detection. However, most evaluation techniques described in literature do not address the complexity and range of the issues which underpin the design of a good evaluation methodology. In this paper, we explore the problems associated with both the optimising the operating point of any motion detection algorithms and the objective performance comparison of competing algorithms. In particular, we develop an object-based approach based on the F-Measure—a single-valued ROC-like measure which enables a straight-forward mechanism for both optimising and comparing motion detection algorithms. Despite the advantages over pixel-based ROC approaches, a number of important issues associated with parameterising the evaluation algorithm need to be addressed. The approach is illustrated by a comparison of three motion detection algorithms including the well-known Stauffer and Grimson algorithm, based on results obtained on two datasets.