Domain adaptation with low-rank alignment for weakly supervised hand pose recovery

- The key contribution is domain adaptation for weakly supervised hand pose recovery. Both the training samples and testing samples are represented in a unified space and aligning parameters are computed in this space. These parameters are used to align the testing samples to the domain of training samples.
- The second contribution is domain adaptation with low-rank representation. Low-rank representation is sparse and the distributions of training samples and testing samples can be observed clearly. In this way, the process of alignment can be achieved in low-rank feature space.
- The third contribution is the mapping between 2D depth images and 3D hand poses are computed by a neural network with 2 hidden layers. In this way, their relationship is described on a non-linear manner.

Human hand pose recovery (HPR) in depth images is usually conducted by constructing mappings between 2D depth images and 3D hand poses. It is a challenging task since the feature spaces of 2D images and 3D poses are different. Therefore, a large number of labeled data is required for training, especially for popular frameworks such as deep learning. In this paper, we propose an HPR method with weak supervision. It is based on neural network and domain adaptation is introduced to enhance the trained model. To achieve domain adaptation, we propose low-rank alignment, which aligns the testing samples to the distribution of labeled samples. In this process, autoencoders are used to extract 2D image features and low-rank representation is used to describe this feature space. Therefore, the proposed method is named as Domain Adaptation with Low-Rank Alignment (DALA). In this way, we obtain a robust and non-linear mapping from 2D images to 3D poses. Experiments are conducted on two challenging benchmark datasets MSRA and ICVL. Both the results on a single dataset and across datasets show the outstanding performance of DALA.