Multi-agent activity recognition using observation decomposedhidden Markov models

To automatically recognize multi-agent activities is a highly challenging task due to the complexity of the interactions between agents. The difficulties in this task stem from two aspects: firstly, the feature vectors derived from input data are of large dimensionality and variable length. Secondly, an efficient mapping of agents from input data to pre-defined activity models, known as agent assignment, is required. This paper presents a new method to model and classify multi-agent activities based on the proposed observation decomposed hidden Markov models (ODHMMs). To handle the feature vectors, we decomposed each original feature vector into a set of sub-feature vectors to keep the explored feature space consistent. Agent assignment is realized using a newly introduced parameter, which represents the ‘role’ of each agent. The experimental results show that the proposed method can successfully classify three-person activities with high accuracy and is less sensitive to incomplete data input.