3D human action recognition model based on image set and regularized multi-task leaning

With in-depth research of action recognition, the conceptions and algorithms of multi-view had been proposed and demonstrated its superiority by researchers, but these algorithms usually neglected the relatedness of multi-views. For the sake of overcome the shortage, this paper proposes a 3D human action recognition model based on image set and regularized multi-task learning. Specifically, we first extract dense trajectory feature for each camera, and then propose to construct the shared codebook by k-means for all cameras, after that, Bag-of-Word (BOW) weight scheme is employed to code dense trajectory feature by the shared codebook for each camera, respectively. Furthermore, we formulate the 3D human action recognition into regularized multi-task learning problem penalized by image set and relevant and irrelevant decomposition to discover the underlying relationship among different tasks and different views, and consequently boost the performances. Large scale experimental results on three public multi-view action3D datasets - IXMAX, UCLA and CVS-MV-RGBD-SINGLE, show that multi-task learning approach is very helpful for discovering the latent relationships among different tasks, which can significantly improve performance over the single-task learning method. Moreover, the image set regularized term, which is utilized to combine different views, can further improve the performance. In a word, the performance of our proposed method is comparable to the state-of-the-art methods.