Understanding human interactions with track and body synergies (TBS) captured from multiple views

This paper presents a new two-stage multi-view framework for the analysis of human interactions and activities. The analysis is performed in a distributed multi-view vision system that synergistically integrates track- and body-level processing. The proposed framework is geared toward versatile and easily-deployable systems that do not require careful camera calibration. The main contributions of the paper are as follows; (1) context-dependent view switching for occlusion handling, (2) a method for switching the two-stage analysis between the track- and body-level processing, and (3) a hypothesis–verification paradigm for top-down feedback that exploits the spatio-temporal constraints inherent in human interaction. An experimental evaluation shows the efficacy of the proposed system for analyzing multi-person interactions.