Combining multiple depth-based descriptors for hand gesture recognition

•The hand is reliably extracted from the scene by jointly using color and depth data.•Features extracted from depth data allow a reliable hand gesture recognition.•Multiple features capturing different properties of the gestures are combined together.•The proposed approach is able to obtain a very high accuracy in real-time.

Depth data acquired by current low-cost real-time depth cameras provide a more informative description of the hand pose that can be exploited for gesture recognition purposes. Following this rationale, this paper introduces a novel hand gesture recognition scheme based on depth information. The hand is firstly extracted from the acquired data and divided into palm and finger regions. Then four different sets of feature descriptors are extracted, accounting for different clues like the distances of the fingertips from the hand center and from the palm plane, the curvature of the hand contour and the geometry of the palm region. Finally a multi-class SVM classifier is employed to recognize the performed gestures. Experimental results demonstrate the ability of the proposed scheme to achieve a very high accuracy on both standard datasets and on more complex ones acquired for experimental evaluation. The current implementation is also able to run in real-time.