An array of physical sensors and an adaptive regression strategy for emotion recognition in a noisy scenario

Several studies demonstrate that since emotions are spontaneously manifested through different measurable quantities (e.g. vocal and facial expressions), this makes possible a sort of automatic estimation of emotion from objective measurements. However, the reliability of such estimations is strongly influenced by the availability of the different sensor modalities used to monitor the affective status of a subject, and furthermore the extraction of objective parameters is sometime thwarted in a noisy and disturbed environment. This paper introduces a personalized emotion estimation based on a heterogeneous array of physical sensors for the measurement of vocal, facial, and physiological (electro-cardiogram and electro-dermal) activities. As a proof of concept, changes in the levels of both emotion reactiveness and pleasantness are estimated under critical operative conditions. The estimator model takes advantage from the time-varying selection of the most relevant non-spurious sensors features and the adaptation of the k-nearest neighbour paradigm to the continuous identification of the most affine model templates. The model, once trained, demonstrated to autonomously embed new sensorial input and adapt to unwanted/unpredicted sensor noise or emotion alteration. The proposed approach has been successfully tested on the RECOLA database, a multi-sensorial corpus of spontaneous emotional interactions in French.