An RGB–D based image set classification for robust face recognition from Kinect data

The paper proposes a method for robust face recognition from low quality Kinect acquired images which have a wide range of variations in head pose, illumination, facial expressions, sunglass disguise and occlusions by hand. Multiple Kinect images of a person are considered as an image set and face recognition from these images is formulated as an RGB–D image set classification problem. The Kinect acquired raw depth data is used for pose estimation and an automatic cropping of the face region. Based upon the estimated poses, the face images of a set are divided into multiple image subsets. An efficient block based covariance matrix representation is proposed to model images in an image subset on Riemannian manifold (Lie group). For classification, SVM models are separately learnt for each image subset on the Lie group of Riemannian manifold and a fusion strategy is introduced to combine results from all image subsets. The proposed technique has been evaluated on a combination of three large data sets containing over 35,000 RGB–D images under challenging conditions. The proposed RGB–D based image set classification incurs low computational cost and achieves an identification rate as high as 99.5%.