Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data

•We apply Random Forests for regression (RDF-R) to 3D Hand Pose Estimation.•RDF-R is generalized by hierarchical mode selection using constraints (RDF-R+).•We test Classification Forests (RDF-C), RDF-R and RDF-R+ with 4 different datasets.•The proposed method, RDF-R+ outperforms RDF-C and RDF-R.

The emergence of inexpensive 2.5D depth cameras has enabled the extraction of the articulated human body pose. However, human hand skeleton extraction still stays as a challenging problem since the hand contains as many joints as the human body model. The small size of the hand also makes the problem more challenging due to resolution limits of the depth cameras. Moreover, hand poses suffer from self-occlusion which is considerably less likely in a body pose. This paper describes a scheme for extracting the hand skeleton using random regression forests in real-time that is robust to self- occlusion and low resolution of the depth camera. In addition to that, the proposed algorithm can estimate the joint positions even if all of the pixels related to a joint are out of the camera frame. The performance of the new method is compared to the random classification forests based method in the literature. Moreover, the performance of the joint estimation is further improved using a novel hierarchical mode selection algorithm that makes use of constraints imposed by the skeleton geometry. The performance of the proposed algorithm is tested on datasets containing synthetic and real data, where self-occlusion is frequently encountered. The new algorithm which runs in real time using a single depth image is shown to outperform previous methods.