User-adaptive hand gesture recognition system with interactive training

Our paper proposes a vision-based hand gesture recognition system with interactive training, aimed to achieve a user-independent application by on-line supervised training. Usual recognition systems involve a preliminary off-line training phase, separated from the recognition phase. If the system recognizes unknown (non-trainer) users the recognition rate of gesture classes could decrease. The recognition has to be suspended and all gestures need to be retrained with an improved training set, resulting in inconveniences. Our new approach introduces an on-line training method embedded into the recognition process, being interactively controlled by the user and adapting to his/her gestures. Our main goal is that any non-trainer users be able to use the system instantly and if the recognition accuracy decreases only the faulty detected gestures be retrained realizing fast adaptation. We implement the proposed system as a camera-projector system in which users can directly interact with the projected image by hand gestures, realizing an augmented reality tool in a multi-user environment. The emphasis is on the novel approach of dynamic and quick follow-up training capabilities instead of handling large pre-trained databases. We also conducted tests on several users in real environments for a practical application.