Stand-alone quality estimation of background subtraction algorithms

Foreground segmentation is a key stage in multiple computer vision applications, where existing algorithms are commonly evaluated making use of ground-truth data. Reference-free or stand-alone evaluations that estimate segmented foreground quality are an alternative methodology to overcome the limitations inherent to ground-truth based evaluations. In this work, we survey and explore existing stand-alone measures proposed in related research areas to determine good object properties for estimating the segmentation quality in background subtraction algorithms. We propose a new taxonomy for stand-alone evaluation measures and analyze 21 proposals. We demonstrate the utility of the selected measures to evaluate the segmentation masks of eight background subtraction algorithms. The experiments are performed over a large heterogeneous dataset with varied challenges (CDNET2014) and identify which properties of the measures are the most effective to estimate quality. The experiments also demonstrate that qualitative performance levels can be distinguished and background subtraction algorithms can be ranked without the need of ground-truth.