An evaluation of 3D head pose estimation using the Microsoft Kinect v2

• We evaluate the Kinect v2 sensor for use in non-invasive head pose estimation.
• High definition face tracking is used to extract markerless head pose estimates.
• Results are compared against a marker-based (Vicon) record of ground truth.
• Where facial features remain visible, pose estimation accuracy is state of the art.
• Kinect v2 is a powerful tool for real-time non-invasive measurement of head posture.

The Kinect v2 sensor supports real-time non-invasive 3D head pose estimation. Because the sensor is small, widely available and relatively cheap it has great potential as a tool for groups interested in measuring head posture. In this paper we compare the Kinect's head pose estimates with a marker-based record of ground truth in order to establish its accuracy. During movement of the head and neck alone (with static torso), we find average errors in absolute yaw, pitch and roll angles of 2.0 ± 1.2°, 7.3 ± 3.2° and 2.6 ± 0.7°, and in rotations relative to the rest pose of 1.4 ± 0.5°, 2.1 ± 0.4° and 2.0 ± 0.8°. Larger head rotations where it becomes difficult to see facial features can cause estimation to fail (10.2 ± 6.1% of all poses in our static torso range of motion tests) but we found no significant changes in performance with the participant standing further away from Kinect – additionally enabling full-body pose estimation – or without performing face shape calibration, something which is not always possible for younger or disabled participants. Where facial features remain visible, the sensor has applications in the non-invasive assessment of postural control, e.g. during a programme of physical therapy. In particular, a multi-Kinect setup covering the full range of head (and body) movement would appear to be a promising way forward.