Hand gesture recognition from multibeam sonar imagery*

Divers perform demanding tasks in a complex and hazardous underwater environment, which prevents them from carrying special devices that may allow them to communicate with their robotic diving buddies. In this world of natural human-robot interaction in the underwater environment, envisioned by the FP7 Cognitive Robotics project CADDY, hand detection and gesture interpretation is a prerequisite. While hand gesture recognition is most often performed with cameras (mono and stereo), their use in the underwater environment is compromised due to water turbidity and lack of sunlight at greater depths. This paper deals with this lack of performance by introducing the concept of using high resolution multibeam sonars (often referred to as acoustic cameras) for diver hand gesture recognition. In order to ensure reliable communication between the diver and the robot, it is of great importance that the classification precision is as high as possible. This paper presents results of hand gesture recognition which is performed by using two approaches: convex hull method and the support vector machine (SVM). A novel approach that fuses the two methods is introduced as a way of increasing the precision of classification. The results obtained on more than 1000 real sonar samples show that the precision using the convex hull method is around 92%, and using the SVM around 94%, while fusing the two approaches provides around 99% classification precision.