Approaches for automatic low-dimensional human shape refinement with priors or generic cues using RGB-D data

•A novel and accurate method to refine low dimensional human shape using RGB-D data is proposed.•Uses of multiple modalities do not carry any features from the shape provider.•Combines low and high level observations jointly in multi-layer graph structure•Extensive experiments showed that it outperforms compared suitable algorithms.•Also, an existing method is extended by fusing more generic cues for this purpose.

Some human detection or tracking algorithms output a low-dimensional representation of the human body, such as a bounding box. Even though this representation is enough for some tasks, a more accurate and detailed point-wise representation of the human body is more useful for pose estimation and action recognition. The refinement process can produce a point-wise mask of the human body from its low-dimensional representation. In this paper, we tackle the problem of refining low-dimensional human shapes using RGB-D data with a novel and accurate method for this purpose. This algorithm combines low-level cues such as shape and color, and high level observations such as the estimated ground plane, in a multi-layer graph cut framework. In our algorithm, shape prior information is learned by training a classifier. Unlike some existing work, our method does not utilize or carry features from the internal steps of the methods which provide the bounding box, so our method can work on the outputs of any similar shape providers. Extensive experiments demonstrate that the proposed technique significantly outperforms other suitable methods. Moreover, a previously published refinement method is extended by incorporating more generic cues to serve this purpose.