Behavioral-model-based freehand tracking in a Selection-Move-Release system

•A practical cognitive-behavioral model, TPTM model, is proposed.•The manner the TPTM model guides the design of freehand tracking is shown.•We provided a method to sample and a way to determine the sampling number.

The development of three dimensional (3D) freehand tracking is motivated by the need for 3D freehand-based human computer interface in the Selection-Move-Release (SMR) system. The high dimensionality of articulated hand models is one of the major obstacles for tracking 3D hand in real time. A practical Task-Phase-Trajectory-Mentality (TPTM) model for the SMR system is built and a novel freehand tracking algorithm is introduced by using a camera that is calibrated by a charge-coupled device. The TPTM model is proposed and explained based on the cognitive theory in the proposed method. The manner by which the TPTM model guides the realtime design of freehand tracking is also shown. The proposed method reduces the high dimensionality down to eight degrees of freedom (DOF), optimizes the dynamic number of particles for each frame, and achieves realtime hand tracking and high accuracy of 3D full DOF hand gestures. The algorithm presented in this paper is successfully applied to a 3D virtual teaching system.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide