Combining appearance and motion for face and gender recognition from videos

While many works consider moving faces only as collections of frames and apply still image-based methods, recent developments indicate that excellent results can be obtained using texture-based spatiotemporal representations for describing and analyzing faces in videos. Inspired by the psychophysical findings which state that facial movements can provide valuable information to face analysis, and also by our recent success in using LBP (local binary patterns) for combining appearance and motion for dynamic texture analysis, this paper investigates the combination of facial appearance (the shape of the face) and motion (the way a person is talking and moving his/her facial features) for face analysis in videos. We propose and study an approach for spatiotemporal face and gender recognition from videos using an extended set of volume LBP features and a boosting scheme. We experiment with several publicly available video face databases and consider different benchmark methods for comparison. Our extensive experimental analysis clearly assesses the promising performance of the LBP-based spatiotemporal representations for describing and analyzing faces in videos.