Using the Agoraset dataset: Assessing for the quality of crowd video analysis methods

In this paper, we present a simulation-based crowd video dataset to be used for evaluation of low-level video crowd analysis methods, such as tracking or segmentation. Most of the time, an exact ground truth associated to real videos is difficult and time-consuming to produce, prone to errors, and these difficulties rise exponentially with the apparent density of the crowd in the image. We propose a synthetic crowd dataset to help researchers evaluate their methods against an objective and temporally dense synthetic ground truth. This dataset, named Agoraset, is presented in detail. The associated ground-truth and metrics are also described, together with a discussion on the use of this new kind of dataset in the field of pattern recognition. We believe this dataset to be the first bridge between simulation and pattern recognition in the field of dense crowd analysis. A discussion on the range of validity and limitations of the use of synthetic datasets in the contest of video processing is also proposed.