3D hand tracking for human computer interaction

We propose a real-time model-based 3D hand tracker that combines image regions and the signal from an off-the-shelf 3-axis accelerometer placed on the user's hand. The visual regions allow the tracker to cope with occlusions, motion blur and background clutter, while the latter aids with the inherent silhouette-pose ambiguities. The accelerometer and tracker are synchronised by casting the calibration problem as one of principal component analysis. Based on the assumption that, often, the number of possible hand configurations is limited by the activity the hand is engaging in, we use a multiclass pose classifier to distinguish between a number of activity dependent articulated hand configurations. We demonstrate the benefits of our method, both qualitatively and quantitatively, on a variety of video sequences and hand configurations and show a proof-of-concept human computer interface based on our system.

Figure optionsDownload high-quality image (59 K)Download as PowerPoint slideHighlights
► Use a level set, region and model based 3D tracker.
► Combine with an accelerometer to account for the pose-silhouette ambiguities.
► Use a multiclass pose classifier to distinguish between activity dependent hand configuration.
► Works in real time.