Context-based hand gesture recognition for the operating room

•A sterile system for navigating MRI images in the operating room is presented.•A novel method for continuous gesture recognition has been discussed.•Contextual cues from a neurobiopsy procedure were integrated into the system.•The system has been shown to significantly improve task completion performance.•The system was significantly more natural than current methods of MRI navigation.

A sterile, intuitive context-integrated system for navigating MRIs through freehand gestures during a neurobiopsy procedure is presented. Contextual cues are used to determine the intent of the user to improve continuous gesture recognition, and the discovery and exploration of MRIs. One of the challenges in gesture interaction in the operating room is to discriminate between intentional and non-intentional gestures. This problem is also referred as spotting. In this paper, a novel method for training gesture spotting networks is presented. The continuous gesture recognition system was shown to successfully detect gestures 92.26% of the time with a reliability of 89.97%. Experimental results show that significant improvements in task completion time were obtained through the effect of context integration.