The impact of weak ground truth and facial expressiveness on affect detection accuracy from time-continuous videos of facial expressions

In this paper we address two issues concerning real-world time-continuous emotion detection from videos of users’ faces: (i) the impact of weak ground truth on the emotion detection accuracy and (ii) the impact of the users’ facial expressiveness on the emotion detection accuracy. We implemented an appearance-based emotion detection algorithm that uses Gabor features and a k nearest neighbors classifier. We tested the performance of this algorithm on two datasets with different ground truth strengths (a firm ground truth dataset and a weak ground truth dataset). Then we split the dataset into three subsets reflecting different levels of users’ facial expressiveness (low, mid and high) and performed separate emotion detection.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide