Common-sense reasoning for human action recognition

This paper presents a novel method that leverages reasoning capabilities in a computer vision system dedicated to human action recognition. The proposed methodology is decomposed into two stages. First, a machine learning based algorithm – known as bag of words – gives a first estimate of action classification from video sequences, by performing an image feature analysis. Those results are afterward passed to a common-sense reasoning system, which analyses, selects and corrects the initial estimation yielded by the machine learning algorithm. This second stage resorts to the knowledge implicit in the rationality that motivates human behaviour. Experiments are performed in realistic conditions, where poor recognition rates by the machine learning techniques are significantly improved by the second stage in which common-sense knowledge and reasoning capabilities have been leveraged. This demonstrates the value of integrating common-sense capabilities into a computer vision pipeline.

Graphical abstractFigure optionsDownload full-size imageDownload high-quality image (80 K)Download as PowerPoint slideHighlights► Action recognition approach which combines statistical & knowledge based reasoning. ► First fully context & scenario independent framework combing computer vision and AI. ► Inclusion of artificial intelligence strategies, based on common sense reasoning. ► Results outperform state-of-art in computer vision under real-life conditions. ► Computer vision authors should not focus only on machine learning but also on AI.