Hands and face tracking for VR applications

In this paper, we present a robust real-time 3D tracking system of human hands and face. This system can be used as a perceptual interface for virtual reality activities in a workbench environment. The main advantage of our system is that the human, placed in front of the virtual reality device, does not need any type of marker or special suit. The system includes a colour segmentation module to detect in real-time the skin-colour pixels present in the images. The results of this skin-colour segmentation will be skin-colour blobs, these are the inputs of a data association module. This module labels the blobs pixels using a set of hypothesis from previous frames. The 2D-tracking results are used for the 3D reconstruction of hands and face in order to obtain the 3D positions of these limbs. Finally, we present several results using the H-ANIM standard to show the system's output performance.