Exploiting multi-expression dependences for implicit multi-emotion video tagging

• We propose a novel multiple emotional tagging method based on multi-expressions.
• We propose a novel multi-expression recognition by exploring dependencies among expressions.
• We model the relationships between expressions and emotions using a Bayesian Network.

In this paper, a novel approach of implicit multiple emotional video tagging is proposed, which considers the relations between the users' facial expressions and emotions as well as the relations among multiple expressions. First, the audiences' expressions are inferred through a multi-expression recognition model, which consists of an image-driven expression measurement recognition and a Bayesian network representing the co-existence and mutual exclusion relations among multi-expressions. Second, the videos' multi-emotion tags are obtained from the recognized expressions by another Bayesian Network, capturing the relations between expressions and emotions. Results of the experiments conducted on the JAFFE and NVIE databases demonstrate that the performance of expression recognition is improved by considering the relations among multiple expressions. Furthermore, the relations between expressions and emotions help improve emotional tagging, as our approach outperforms the traditional expression-based or image-driven implicit tagging methods.