User and context adaptive neural networks for emotion recognition

Recognition of emotional states of users in human–computer interaction (HCI) has been shown to be highly dependent on individual human characteristics and way of behavior. Multimodality is a key issue in achieving more accurate results; however, fusing different modalities is a difficult issue in emotion analysis. Emotion recognition systems are generally either rule-based or extensively trained through emotionally colored HCI data sets. In either case, such systems need to take into account, i.e., adapt their knowledge to, the specific user or context of interaction. Neural networks fit well with the adaptation requirement, by collecting and analyzing data from specific environments. An effective approach is presented in this paper, which uses neural network architectures to both detect the need for adaptation of their knowledge, and adapt it through an efficient adaptation procedure. An experimental study with emotion datasets generate in the framework of the EC IST Humaine Network of Excellence.