Automatic visual detection of human behavior: A review from 2000 to 2014

•We review the topic of automatic detection of human behaviors from video.•We survey 193 papers using six major scientific publishers from 2000 to 2014.•Papers were classified into three subjects: techniques, datasets and applications.•We provide a roadmap to guide future research in this area.

Due to advances in information technology (e.g., digital video cameras, ubiquitous sensors), the automatic detection of human behaviors from video is a very recent research topic. In this paper, we perform a systematic and recent literature review on this topic, from 2000 to 2014, covering a selection of 193 papers that were searched from six major scientific publishers. The selected papers were classified into three main subjects: detection techniques, datasets and applications. The detection techniques were divided into four categories (initialization, tracking, pose estimation and recognition). The list of datasets includes eight examples (e.g., Hollywood action). Finally, several application areas were identified, including human detection, abnormal activity detection, action recognition, player modeling and pedestrian detection. Our analysis provides a road map to guide future research for designing automatic visual human behavior detection systems.